2024-03-29T08:47:05.033 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T08:47:05.033 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm} --owner scheduled_vshankar@teuthology --verbose --name vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/orig.config.yaml 2024-03-29T08:47:05.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T08:47:05.174 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169 branch: wip-vshankar-testing-20240326.105515-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm} email: vshankar@redhat.com first_in_suite: false job_id: '7628169' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-20240326.105515-reef ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240326.105515-reef sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7327 sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 sleep_before_teardown: 0 subset: 990/1024 suite: fs suite_branch: wip-vshankar-testing-20240326.105515-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHjuiw4ET3PmkH1wWnugZbpBQ11jBn6EVX0hktynHK1DwKhuOOn59FfDYFbktPv+hJf2YQy+VF9dgCgM4ZmInxo= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAdC+qenmuQZ2Ai2w2X9Q44TgT72N3dd0UgBMDYJpKaq1BCq99ERVhZj4/lvVfPK/WygeKTtsZP/EXl4Fppfs0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-28_11:21:59 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-29T08:47:05.174 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa; will attempt to use it 2024-03-29T08:47:05.175 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks 2024-03-29T08:47:05.175 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T08:47:05.176 INFO:teuthology.task.internal:Checking packages... 2024-03-29T08:47:05.201 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'd0e392d9aee635f2b65768c460c7a151ead7a5e2' 2024-03-29T08:47:05.201 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T08:47:05.201 INFO:teuthology.packaging:ref: None 2024-03-29T08:47:05.201 INFO:teuthology.packaging:tag: None 2024-03-29T08:47:05.201 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240326.105515-reef 2024-03-29T08:47:05.201 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:47:05.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20240326.105515-reef 2024-03-29T08:47:05.438 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1015.gd0e392d9 2024-03-29T08:47:05.439 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T08:47:05.448 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T08:47:05.448 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T08:47:05.753 INFO:teuthology.task.internal:Saving configuration 2024-03-29T08:47:05.769 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T08:47:06.003 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T08:47:06.024 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 08:41:48.058747', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHjuiw4ET3PmkH1wWnugZbpBQ11jBn6EVX0hktynHK1DwKhuOOn59FfDYFbktPv+hJf2YQy+VF9dgCgM4ZmInxo='} 2024-03-29T08:47:06.042 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 08:41:48.056934', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAdC+qenmuQZ2Ai2w2X9Q44TgT72N3dd0UgBMDYJpKaq1BCq99ERVhZj4/lvVfPK/WygeKTtsZP/EXl4Fppfs0='} 2024-03-29T08:47:06.042 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T08:47:06.110 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-29T08:47:06.110 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-29T08:47:06.110 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T08:47:06.196 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa3ec5630d0>, signals=[15]) 2024-03-29T08:47:06.197 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T08:47:06.204 INFO:teuthology.task.internal:Opening connections... 2024-03-29T08:47:06.204 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T08:47:06.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:47:06.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T08:47:06.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:47:06.362 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T08:47:06.370 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-03-29T08:47:06.391 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-03-29T08:47:06.391 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-03-29T08:47:06.454 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="Red Hat Enterprise Linux" 2024-03-29T08:47:06.455 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="8.6 (Ootpa)" 2024-03-29T08:47:06.455 INFO:teuthology.orchestra.run.smithi099.stdout:ID="rhel" 2024-03-29T08:47:06.455 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="fedora" 2024-03-29T08:47:06.455 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="8.6" 2024-03-29T08:47:06.455 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el8" 2024-03-29T08:47:06.456 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-29T08:47:06.456 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-03-29T08:47:06.456 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-29T08:47:06.456 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://www.redhat.com/" 2024-03-29T08:47:06.456 INFO:teuthology.orchestra.run.smithi099.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-29T08:47:06.456 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T08:47:06.456 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T08:47:06.457 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-29T08:47:06.457 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-29T08:47:06.457 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-29T08:47:06.457 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-29T08:47:06.458 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-03-29T08:47:06.480 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-03-29T08:47:06.501 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-03-29T08:47:06.501 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-03-29T08:47:06.561 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Red Hat Enterprise Linux" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8.6 (Ootpa)" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:ID="rhel" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="fedora" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8.6" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.redhat.com/" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-29T08:47:06.562 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T08:47:06.563 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-03-29T08:47:06.563 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-03-29T08:47:06.563 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-03-29T08:47:06.563 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-03-29T08:47:06.563 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-03-29T08:47:06.579 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T08:47:06.592 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T08:47:06.658 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T08:47:06.658 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-03-29T08:47:06.660 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-03-29T08:47:06.679 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T08:47:06.696 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T08:47:06.696 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-03-29T08:47:06.724 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-03-29T08:47:06.758 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T08:47:06.805 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T08:47:06.824 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-29T08:47:06.824 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-29T08:47:06.824 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T08:47:06.825 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T08:47:06.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:47:06.844 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:06.844 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-29T08:47:06.844 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-03-29T08:47:06.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:47:06.845 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:06.845 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-29T08:47:06.846 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-03-29T08:47:06.905 INFO:teuthology.orchestra.run.smithi133.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-29T08:47:06.906 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:06.906 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-29T08:47:06.906 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:06.910 INFO:teuthology.orchestra.run.smithi099.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-03-29T08:47:06.913 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:06.913 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-29T08:47:06.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:07.127 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum upgrade -y linux-firmware 2024-03-29T08:47:07.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum upgrade -y linux-firmware 2024-03-29T08:47:08.590 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2024-03-29T08:47:08.590 INFO:teuthology.orchestra.run.smithi133.stdout:Unable to read consumer identity 2024-03-29T08:47:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-29T08:47:08.682 INFO:teuthology.orchestra.run.smithi099.stdout:Unable to read consumer identity 2024-03-29T08:47:08.922 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T08:47:08.922 INFO:teuthology.orchestra.run.smithi133.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-29T08:47:08.922 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T08:47:08.973 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T08:47:08.973 INFO:teuthology.orchestra.run.smithi099.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-03-29T08:47:08.973 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T08:47:10.558 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-03-29T08:47:10.909 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2024-03-29T08:47:11.467 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 19 MB/s | 16 MB 00:00 2024-03-29T08:47:12.403 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2024-03-29T08:47:15.083 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 537 kB/s | 24 kB 00:00 2024-03-29T08:47:15.899 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-03-29T08:47:15.900 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-03-29T08:47:15.900 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-03-29T08:47:16.026 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 409 kB/s | 24 kB 00:00 2024-03-29T08:47:16.050 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-29T08:47:16.050 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:16.269 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-29T08:47:16.269 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-29T08:47:16.271 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi133.front.sepia.ceph.com to complete... 2024-03-29T08:47:16.844 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-29T08:47:16.845 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-03-29T08:47:16.845 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-29T08:47:16.963 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-29T08:47:16.963 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:47:17.181 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ 2024-03-29T08:47:17.181 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/ --input-file=- 2024-03-29T08:47:17.184 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi099.front.sepia.ceph.com to complete... 2024-03-29T08:47:31.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29 08:47:31 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-29T08:47:31.659 INFO:teuthology.orchestra.run.smithi133.stderr:FINISHED --2024-03-29 08:47:31-- 2024-03-29T08:47:31.659 INFO:teuthology.orchestra.run.smithi133.stderr:Total wall clock time: 15s 2024-03-29T08:47:31.659 INFO:teuthology.orchestra.run.smithi133.stderr:Downloaded: 1 files, 383M in 15s (25.4 MB/s) 2024-03-29T08:47:31.662 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-29T08:47:31.663 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-29T08:47:33.470 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying... ######################################## 2024-03-29T08:47:33.564 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing... ######################################## 2024-03-29T08:47:33.722 INFO:teuthology.orchestra.run.smithi133.stdout:Updating / installing... 2024-03-29T08:47:55.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29 08:47:55 URL:https://1.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/8/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [401601944/401601944] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-29T08:47:55.595 INFO:teuthology.orchestra.run.smithi099.stderr:FINISHED --2024-03-29 08:47:55-- 2024-03-29T08:47:55.595 INFO:teuthology.orchestra.run.smithi099.stderr:Total wall clock time: 38s 2024-03-29T08:47:55.595 INFO:teuthology.orchestra.run.smithi099.stderr:Downloaded: 1 files, 383M in 38s (10.1 MB/s) 2024-03-29T08:47:55.598 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-29T08:47:55.598 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-29T08:47:57.422 INFO:teuthology.orchestra.run.smithi099.stdout:Verifying... ######################################## 2024-03-29T08:47:57.514 INFO:teuthology.orchestra.run.smithi099.stdout:Preparing... ######################################## 2024-03-29T08:47:57.670 INFO:teuthology.orchestra.run.smithi099.stdout:Updating / installing... 2024-03-29T08:48:14.459 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-29T08:48:16.773 INFO:teuthology.orchestra.run.smithi133.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T08:48:36.589 INFO:teuthology.orchestra.run.smithi133.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T08:48:37.992 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-29T08:48:39.408 INFO:teuthology.orchestra.run.smithi099.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T08:49:04.120 INFO:teuthology.orchestra.run.smithi099.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-29T08:50:47.753 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-29T08:50:47.753 DEBUG:teuthology.orchestra.run.smithi133:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-29T08:50:47.812 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-29T08:50:47.812 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-29T08:50:47.812 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-29T08:50:47.812 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-29T08:50:47.812 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-29T08:50:47.812 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-29T08:50:47.813 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-29T08:50:47.814 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-29T08:50:47.815 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-29T08:50:47.816 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-29T08:50:47.817 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-29T08:50:47.818 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-29T08:50:47.819 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-29T08:50:47.820 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-29T08:50:47.821 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-29T08:50:47.822 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-29T08:50:47.823 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-29T08:50:47.824 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-29T08:50:47.825 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-29T08:50:47.826 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-29T08:50:47.827 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-29T08:50:47.828 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-29T08:50:47.829 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-29T08:50:47.830 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-29T08:50:47.831 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-29T08:50:47.832 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-29T08:50:47.833 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-29T08:50:47.834 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-29T08:50:47.835 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-29T08:50:47.836 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-29T08:50:47.837 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-29T08:50:47.838 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-29T08:50:47.839 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-29T08:50:47.840 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-29T08:50:47.841 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-29T08:50:47.842 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-29T08:50:47.843 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-29T08:50:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-29T08:50:47.845 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-29T08:50:47.846 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-29T08:50:47.847 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-29T08:50:47.848 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-29T08:50:47.849 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-29T08:50:47.850 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-29T08:50:47.851 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-29T08:50:47.852 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-29T08:50:47.853 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-29T08:50:47.854 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-29T08:50:47.855 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-29T08:50:47.856 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-29T08:50:47.857 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-29T08:50:47.858 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-29T08:50:47.859 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-29T08:50:47.860 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-29T08:50:47.861 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-29T08:50:47.862 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-29T08:50:47.863 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-29T08:50:47.864 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-29T08:50:47.865 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-29T08:50:47.866 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-29T08:50:47.867 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-29T08:50:47.868 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-29T08:50:47.869 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-29T08:50:47.870 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-29T08:50:47.871 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-29T08:50:47.872 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-29T08:50:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-29T08:50:47.874 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-29T08:50:47.875 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-29T08:50:47.876 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-29T08:50:47.877 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-29T08:50:47.878 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-29T08:50:47.879 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-29T08:50:47.880 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-29T08:50:47.881 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-29T08:50:47.882 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-29T08:50:47.883 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-29T08:50:47.884 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-29T08:50:47.885 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-29T08:50:47.886 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-29T08:50:47.887 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-29T08:50:47.888 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-29T08:50:47.889 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-29T08:50:47.890 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-29T08:50:47.891 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-29T08:50:47.892 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-29T08:50:47.893 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-29T08:50:47.894 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-29T08:50:47.895 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-29T08:50:47.896 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-29T08:50:47.897 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-29T08:50:47.898 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-29T08:50:47.899 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-29T08:50:47.900 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-29T08:50:47.901 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-29T08:50:47.902 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-29T08:50:47.903 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-29T08:50:47.904 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-29T08:50:47.905 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-29T08:50:47.906 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-29T08:50:47.907 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-29T08:50:47.908 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-29T08:50:47.909 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-29T08:50:47.910 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-29T08:50:47.911 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-29T08:50:47.912 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-29T08:50:47.913 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-29T08:50:47.914 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-29T08:50:47.915 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-29T08:50:47.916 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-29T08:50:47.917 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-29T08:50:47.918 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-29T08:50:47.919 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-29T08:50:47.920 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-29T08:50:47.921 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-29T08:50:47.922 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-29T08:50:47.923 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-29T08:50:47.924 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-29T08:50:47.925 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-29T08:50:47.926 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-29T08:50:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-29T08:50:47.928 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-29T08:50:47.929 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-29T08:50:47.930 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-29T08:50:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-29T08:50:47.932 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-29T08:50:47.933 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-29T08:50:47.934 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-29T08:50:47.935 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-29T08:50:47.936 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-29T08:50:47.937 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-29T08:50:47.938 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-29T08:50:47.939 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-29T08:50:47.940 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-29T08:50:47.941 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-29T08:50:47.942 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-29T08:50:47.943 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-29T08:50:47.944 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-29T08:50:47.945 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-29T08:50:47.946 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-29T08:50:47.947 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-29T08:50:47.948 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-29T08:50:47.949 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-29T08:50:47.950 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-29T08:50:47.951 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-29T08:50:47.952 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-29T08:50:47.952 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-29T08:50:47.952 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-29T08:50:47.952 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-29T08:50:47.952 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T08:50:47.956 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-29T08:50:47.956 DEBUG:teuthology.orchestra.run.smithi133:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-29T08:50:47.979 INFO:teuthology.orchestra.run.smithi133.stdout:postinstall scriptlet (using /bin/sh): 2024-03-29T08:50:47.979 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-29T08:50:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T08:50:47.979 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-03-29T08:50:47.979 INFO:teuthology.orchestra.run.smithi133.stdout:for file in vmlinuz System.map config; do 2024-03-29T08:50:47.979 INFO:teuthology.orchestra.run.smithi133.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout: fi 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:done 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-29T08:50:47.980 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-03-29T08:50:47.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-03-29T08:50:48.055 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-03-29T08:50:48.055 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.02 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 123.el8 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Group : System Environment/Base 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 9265450 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Relocations : (not relocatable) 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : Red Hat, Inc. 2024-03-29T08:50:48.056 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : Red Hat, Inc. 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T08:50:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T08:50:48.058 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T08:50:48.058 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 6.8.0-g1df0c7e8a235 2024-03-29T08:50:48.058 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T08:50:48.860 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-03-29T08:50:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-03-29T08:50:50.912 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-03-29T08:50:50.929 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.dbAILdafZR 2024-03-29T08:50:50.929 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dbAILdafZR 2024-03-29T08:50:51.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.dbAILdafZR 2024-03-29T08:50:51.161 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.dbAILdafZR is 6KB 2024-03-29T08:50:51.210 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.dbAILdafZR 2024-03-29T08:50:51.226 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T08:50:51.298 INFO:teuthology.orchestra.run.smithi133.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-29T08:50:51.298 INFO:teuthology.orchestra.run.smithi133.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235.conf 2024-03-29T08:50:51.298 INFO:teuthology.orchestra.run.smithi133.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-29T08:50:51.298 INFO:teuthology.orchestra.run.smithi133.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-29T08:50:51.298 INFO:teuthology.orchestra.run.smithi133.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-29T08:50:51.299 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235 2024-03-29T08:50:51.446 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-03-29T08:51:15.160 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-29T08:51:15.161 DEBUG:teuthology.orchestra.run.smithi099:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-29T08:51:15.221 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-29T08:51:15.222 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-29T08:51:15.223 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-29T08:51:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-29T08:51:15.225 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-29T08:51:15.226 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-29T08:51:15.227 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-29T08:51:15.228 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-29T08:51:15.229 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-29T08:51:15.230 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-29T08:51:15.231 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-29T08:51:15.232 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-29T08:51:15.233 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-29T08:51:15.234 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-29T08:51:15.235 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-29T08:51:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-29T08:51:15.237 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-29T08:51:15.238 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-29T08:51:15.239 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-29T08:51:15.240 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-29T08:51:15.241 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-29T08:51:15.242 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-29T08:51:15.243 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-29T08:51:15.244 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-29T08:51:15.245 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-29T08:51:15.246 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-29T08:51:15.247 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-29T08:51:15.248 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-29T08:51:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-29T08:51:15.250 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-29T08:51:15.251 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-29T08:51:15.252 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-29T08:51:15.253 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-29T08:51:15.254 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-29T08:51:15.255 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-29T08:51:15.256 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-29T08:51:15.257 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-29T08:51:15.258 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-29T08:51:15.259 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-29T08:51:15.260 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-29T08:51:15.261 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-29T08:51:15.262 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-29T08:51:15.263 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-29T08:51:15.264 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-29T08:51:15.265 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-29T08:51:15.266 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-29T08:51:15.267 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-29T08:51:15.268 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-29T08:51:15.269 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-29T08:51:15.270 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-29T08:51:15.271 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-29T08:51:15.272 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-29T08:51:15.273 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-29T08:51:15.274 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-29T08:51:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-29T08:51:15.276 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-29T08:51:15.277 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-29T08:51:15.278 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-29T08:51:15.279 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-29T08:51:15.280 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-29T08:51:15.281 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-29T08:51:15.282 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-29T08:51:15.283 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-29T08:51:15.284 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-29T08:51:15.285 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-29T08:51:15.286 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-29T08:51:15.287 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-29T08:51:15.288 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-29T08:51:15.289 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-29T08:51:15.290 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-29T08:51:15.291 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-29T08:51:15.292 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-29T08:51:15.293 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-29T08:51:15.294 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-29T08:51:15.295 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-29T08:51:15.296 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-29T08:51:15.297 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-29T08:51:15.298 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-29T08:51:15.299 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-29T08:51:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-29T08:51:15.301 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-29T08:51:15.302 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-29T08:51:15.303 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-29T08:51:15.304 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-29T08:51:15.305 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-29T08:51:15.306 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-29T08:51:15.307 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-29T08:51:15.308 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-29T08:51:15.309 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-29T08:51:15.310 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-29T08:51:15.311 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-29T08:51:15.312 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-29T08:51:15.313 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-29T08:51:15.314 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-29T08:51:15.315 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-29T08:51:15.316 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-29T08:51:15.317 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-29T08:51:15.318 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-29T08:51:15.319 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-29T08:51:15.320 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-29T08:51:15.321 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-29T08:51:15.322 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-29T08:51:15.323 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-29T08:51:15.324 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-29T08:51:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-29T08:51:15.326 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-29T08:51:15.327 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-29T08:51:15.328 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-29T08:51:15.329 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-29T08:51:15.330 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-29T08:51:15.331 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-29T08:51:15.332 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-29T08:51:15.333 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-29T08:51:15.334 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-29T08:51:15.335 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-29T08:51:15.336 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-29T08:51:15.337 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-29T08:51:15.338 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-29T08:51:15.339 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-29T08:51:15.340 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-29T08:51:15.341 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-29T08:51:15.342 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-29T08:51:15.343 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-29T08:51:15.344 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-29T08:51:15.345 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-29T08:51:15.346 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-29T08:51:15.347 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-29T08:51:15.348 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-29T08:51:15.349 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-29T08:51:15.350 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-29T08:51:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-29T08:51:15.352 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-29T08:51:15.353 INFO:teuthology.orchestra.run.smithi099.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T08:51:15.357 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-29T08:51:15.357 DEBUG:teuthology.orchestra.run.smithi099:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-29T08:51:15.380 INFO:teuthology.orchestra.run.smithi099.stdout:postinstall scriptlet (using /bin/sh): 2024-03-29T08:51:15.380 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-29T08:51:15.380 INFO:teuthology.orchestra.run.smithi099.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:for file in vmlinuz System.map config; do 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout: fi 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:done 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-29T08:51:15.381 INFO:teuthology.orchestra.run.smithi099.stdout:fi 2024-03-29T08:51:15.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-03-29T08:51:15.455 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.02 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 123.el8 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Group : System Environment/Base 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 9265450 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-03-29T08:51:15.456 INFO:teuthology.orchestra.run.smithi099.stdout:Relocations : (not relocatable) 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : Red Hat, Inc. 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : Red Hat, Inc. 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T08:51:15.457 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-03-29T08:51:15.458 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-29T08:51:15.458 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 6.8.0-g1df0c7e8a235 2024-03-29T08:51:15.458 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-29T08:51:16.224 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-03-29T08:51:18.154 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-03-29T08:51:18.155 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-03-29T08:51:18.172 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.EPneZ63JkC 2024-03-29T08:51:18.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EPneZ63JkC 2024-03-29T08:51:18.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.EPneZ63JkC 2024-03-29T08:51:18.388 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.EPneZ63JkC is 6KB 2024-03-29T08:51:18.436 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.EPneZ63JkC 2024-03-29T08:51:18.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-03-29T08:51:18.527 INFO:teuthology.orchestra.run.smithi099.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-03-29T08:51:18.527 INFO:teuthology.orchestra.run.smithi099.stdout:2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235.conf 2024-03-29T08:51:18.527 INFO:teuthology.orchestra.run.smithi099.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-03-29T08:51:18.527 INFO:teuthology.orchestra.run.smithi099.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-03-29T08:51:18.527 INFO:teuthology.orchestra.run.smithi099.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-03-29T08:51:18.528 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.8.0-g1df0c7e8a235 2024-03-29T08:51:18.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-03-29T08:51:21.450 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-29T08:51:21.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-29T08:51:21.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:51:48.758 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-29T08:51:48.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-29T08:51:48.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:51:56.062 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-03-29T08:52:05.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-29T08:52:05.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:52:23.455 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-03-29T08:52:23.455 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-29T08:52:32.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-29T08:52:32.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:52:35.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-29T08:52:35.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:52:38.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-03-29T08:52:50.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-29T08:52:53.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-29T08:52:53.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:52:53.986 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-03-29T08:52:54.433 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-29T08:52:54.433 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-29T08:52:54.433 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-03-29T08:52:54.474 INFO:teuthology.orchestra.run.smithi133.stdout:6.8.0-g1df0c7e8a235 2024-03-29T08:52:54.474 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-29T08:52:54.474 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T08:52:54.474 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-29T08:52:55.475 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-29T08:52:55.475 DEBUG:teuthology.orchestra.run.smithi133:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T08:52:55.607 INFO:teuthology.orchestra.run.smithi133.stdout:ttyS1 2024-03-29T08:52:55.631 DEBUG:teuthology.parallel:result is None 2024-03-29T08:53:02.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-29T08:53:02.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:53:05.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-03-29T08:53:20.919 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-29T08:53:20.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:53:21.256 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-03-29T08:53:21.683 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-29T08:53:21.683 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-29T08:53:21.683 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-03-29T08:53:21.725 INFO:teuthology.orchestra.run.smithi099.stdout:6.8.0-g1df0c7e8a235 2024-03-29T08:53:21.725 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-29T08:53:21.725 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T08:53:21.725 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-03-29T08:53:22.726 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-29T08:53:22.727 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T08:53:22.853 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-03-29T08:53:22.879 DEBUG:teuthology.parallel:result is None 2024-03-29T08:53:22.880 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T08:53:22.892 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T08:53:22.892 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T08:53:22.896 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T08:53:22.942 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T08:53:22.949 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T08:53:22.973 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T08:53:22.973 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T08:53:22.979 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T08:53:23.051 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T08:53:23.059 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T08:53:23.059 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T08:53:23.063 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T08:53:23.128 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:53:23.150 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:53:23.154 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:53:23.177 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:53:23.180 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T08:53:23.189 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T08:53:23.189 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T08:53:23.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T08:53:23.257 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T08:53:23.266 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T08:53:23.266 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T08:53:23.270 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T08:53:23.310 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T08:53:23.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T08:53:23.495 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T08:53:23.563 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T08:53:23.643 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T08:53:23.643 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T08:53:23.706 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T08:53:23.775 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T08:53:23.864 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T08:53:23.930 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T08:53:24.012 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T08:53:24.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T08:53:24.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-03-29T08:53:24.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-03-29T08:53:24.186 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T08:53:24.241 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T08:53:24.469 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T08:53:24.477 INFO:teuthology.task.internal:Starting timer... 2024-03-29T08:53:24.477 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T08:53:24.511 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T08:53:24.542 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-29T08:53:24.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-03-29T08:53:24.653 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-03-29T08:53:24.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-03-29T08:53:24.783 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-03-29T08:53:24.787 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T08:53:24.787 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-03-29T08:53:24.832 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-03-29T08:53:24.832 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-03-29T08:53:24.886 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-03-29T08:53:24.886 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2024-03-29T08:53:24.886 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T08:53:24.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:53:24.958 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T08:53:25.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:53:25.026 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T08:53:25.026 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-03-29T08:53:25.092 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-03-29T08:53:25.159 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T08:53:25.169 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-29T08:53:25.175 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-29T08:53:25.303 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T08:53:25.320 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T08:53:25.321 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi099.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T09:04:40.601 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2024-03-29T09:04:40.620 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-29T09:04:40.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T09:04:40.718 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-03-29T09:04:40.823 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-03-29T09:04:40.824 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-29T09:04:40.825 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T09:04:40.912 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-03-29T09:04:41.031 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-29T09:04:41.032 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T09:04:41.042 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T09:04:41.042 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T09:04:41.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T09:04:41.047 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T09:04:41.047 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T09:04:41.113 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T09:04:41.142 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-29T09:04:41.145 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T09:04:41.174 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-29T09:04:41.238 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-03-29T09:04:41.266 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-03-29T09:04:41.273 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-03-29T09:04:41.298 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T09:04:41.301 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-03-29T09:04:41.329 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T09:04:41.333 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-29T09:04:41.366 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-29T09:04:41.426 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2024-03-29T09:04:41.433 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T09:04:41.433 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-03-29T09:04:41.433 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.433 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.434 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.434 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.451 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2024-03-29T09:04:41.458 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T09:04:41.458 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-03-29T09:04:41.458 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.459 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.459 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.459 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:04:41.460 INFO:teuthology.run_tasks:Running task install... 2024-03-29T09:04:41.468 DEBUG:teuthology.task.install:project ceph 2024-03-29T09:04:41.469 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}} 2024-03-29T09:04:41.469 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'} 2024-03-29T09:04:41.469 INFO:teuthology.task.install:Using flavor: default 2024-03-29T09:04:41.475 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T09:04:41.475 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-29T09:04:41.476 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'tag': None, 'wait_for_package': False} 2024-03-29T09:04:41.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T09:04:41.479 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'tag': None, 'wait_for_package': False} 2024-03-29T09:04:41.479 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T09:04:41.629 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/ 2024-03-29T09:04:41.630 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1015.gd0e392d9 2024-03-29T09:04:41.740 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T09:04:41.740 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:04:41.740 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T09:04:41.783 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/ 2024-03-29T09:04:41.783 INFO:teuthology.task.install.rpm:Package version is 18.2.2-1015.gd0e392d9 2024-03-29T09:04:41.805 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T09:04:41.806 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e392d9aee635f2b65768c460c7a151ead7a5e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T09:04:41.890 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-29T09:04:41.890 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:04:41.890 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-29T09:04:41.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T09:04:41.956 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-29T09:04:41.956 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e392d9aee635f2b65768c460c7a151ead7a5e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-29T09:04:42.004 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T09:04:42.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-29T09:04:42.067 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-03-29T09:04:42.071 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-03-29T09:04:42.154 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-29T09:04:42.218 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-03-29T09:04:42.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-03-29T09:04:42.517 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2024-03-29T09:04:42.667 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-29T09:04:44.063 INFO:teuthology.orchestra.run.smithi133.stdout:61 files removed 2024-03-29T09:04:44.066 INFO:teuthology.orchestra.run.smithi099.stdout:61 files removed 2024-03-29T09:04:44.122 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T09:04:44.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-29T09:04:44.572 INFO:teuthology.orchestra.run.smithi099.stdout:Updating Subscription Management repositories. 2024-03-29T09:04:44.573 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2024-03-29T09:04:46.590 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 199 kB/s | 80 kB 00:00 2024-03-29T09:04:46.641 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 261 kB/s | 80 kB 00:00 2024-03-29T09:04:46.982 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 28 kB/s | 9.9 kB 00:00 2024-03-29T09:04:47.017 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 29 kB/s | 9.9 kB 00:00 2024-03-29T09:04:47.375 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2024-03-29T09:04:47.409 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2024-03-29T09:04:47.567 INFO:teuthology.orchestra.run.smithi099.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2024-03-29T09:04:47.576 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-03-29T09:04:48.543 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2024-03-29T09:04:49.463 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 16 MB 00:01 2024-03-29T09:04:52.735 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 412 kB/s | 24 kB 00:00 2024-03-29T09:04:53.619 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 393 kB/s | 24 kB 00:00 2024-03-29T09:04:54.244 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 66 MB 00:01 2024-03-29T09:04:54.437 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 12 MB/s | 8.8 MB 00:00 2024-03-29T09:04:57.714 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 MB/s | 60 MB 00:01 2024-03-29T09:05:04.710 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-03-29T09:05:08.080 INFO:teuthology.orchestra.run.smithi099.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 60 MB 00:01 2024-03-29T09:05:09.872 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 MB/s | 66 MB 00:01 2024-03-29T09:05:24.679 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-29T09:05:24.681 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-29T09:05:24.701 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-29T09:05:24.702 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-29T09:05:24.704 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-29T09:05:24.706 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-29T09:05:24.708 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-29T09:05:24.713 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-29T09:05:24.714 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-29T09:05:24.941 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout:=================================================================================================================================== 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout:=================================================================================================================================== 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 6.4 k 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 5.5 M 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 854 k 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 166 k 2024-03-29T09:05:24.956 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 1.6 M 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 138 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 1.8 M 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 7.4 M 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 49 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 13 M 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 47 M 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 263 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 173 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 261 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 18 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-29T09:05:24.957 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 31 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 778 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 139 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 197 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 371 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 364 k 2024-03-29T09:05:24.958 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 106 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 95 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 3.5 M 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 181 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 3.7 M 2024-03-29T09:05:24.959 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 3.6 M 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 22 M 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 23 k 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 2.3 M 2024-03-29T09:05:24.960 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 245 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 4.8 M 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 18 M 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 15 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 24 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 142 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-29T09:05:24.961 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 474 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 6.5 M 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-29T09:05:24.962 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 45 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 129 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-29T09:05:24.963 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-29T09:05:24.964 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-29T09:05:24.965 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-29T09:05:24.966 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout:Enabling module streams: 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql 10 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout:=================================================================================================================================== 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout:Install 130 Packages 2024-03-29T09:05:24.967 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 16 Packages 2024-03-29T09:05:24.968 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:05:24.968 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 213 M 2024-03-29T09:05:24.968 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-03-29T09:05:25.294 INFO:teuthology.orchestra.run.smithi099.stdout:(1/146): ceph-18.2.2-1015.gd0e392d9.el8.x86_64. 43 kB/s | 6.4 kB 00:00 2024-03-29T09:05:25.466 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-29T09:05:25.467 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-03-29T09:05:25.487 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-03-29T09:05:25.489 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-03-29T09:05:25.494 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-03-29T09:05:25.494 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-03-29T09:05:25.495 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-03-29T09:05:25.499 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-03-29T09:05:25.501 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-03-29T09:05:25.602 INFO:teuthology.orchestra.run.smithi099.stdout:(2/146): ceph-fuse-18.2.2-1015.gd0e392d9.el8.x8 2.7 MB/s | 854 kB 00:00 2024-03-29T09:05:25.727 INFO:teuthology.orchestra.run.smithi099.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 1.3 MB/s | 166 kB 00:00 2024-03-29T09:05:25.731 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-03-29T09:05:25.746 INFO:teuthology.orchestra.run.smithi133.stdout:=================================================================================================================================== 2024-03-29T09:05:25.746 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout:=================================================================================================================================== 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 6.4 k 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 5.5 M 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 854 k 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 166 k 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 1.6 M 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 138 k 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 1.8 M 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 7.4 M 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 49 k 2024-03-29T09:05:25.747 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 13 M 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 47 M 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 263 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 173 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 261 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 18 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 31 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 778 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 139 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-03-29T09:05:25.748 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 812 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 10.23-4.module+el8.9.0+21317+d2c6bf8a rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 197 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 371 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 364 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 106 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 95 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 3.5 M 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 181 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-03-29T09:05:25.749 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 3.7 M 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 3.6 M 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-03-29T09:05:25.750 INFO:teuthology.orchestra.run.smithi133.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 22 M 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 23 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 2.3 M 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 245 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 4.8 M 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 18 M 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:18.2.2-1015.gd0e392d9.el8 ceph-noarch 15 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 24 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-03-29T09:05:25.751 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 142 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 474 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 6.5 M 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-03-29T09:05:25.752 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-03-29T09:05:25.753 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 45 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:18.2.2-1015.gd0e392d9.el8 ceph 129 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-03-29T09:05:25.754 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 16.8-9.el8 codeready-builder-for-rhel-8-x86_64-rpms 48 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-03-29T09:05:25.755 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-03-29T09:05:25.756 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql 10 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:05:25.757 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-03-29T09:05:25.758 INFO:teuthology.orchestra.run.smithi133.stdout:=================================================================================================================================== 2024-03-29T09:05:25.758 INFO:teuthology.orchestra.run.smithi133.stdout:Install 130 Packages 2024-03-29T09:05:25.758 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 16 Packages 2024-03-29T09:05:25.758 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:05:25.758 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 213 M 2024-03-29T09:05:25.758 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-03-29T09:05:25.986 INFO:teuthology.orchestra.run.smithi099.stdout:(4/146): ceph-base-18.2.2-1015.gd0e392d9.el8.x8 6.6 MB/s | 5.5 MB 00:00 2024-03-29T09:05:26.163 INFO:teuthology.orchestra.run.smithi133.stdout:(1/146): ceph-18.2.2-1015.gd0e392d9.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2024-03-29T09:05:26.203 INFO:teuthology.orchestra.run.smithi099.stdout:(5/146): ceph-mgr-18.2.2-1015.gd0e392d9.el8.x86 7.2 MB/s | 1.6 MB 00:00 2024-03-29T09:05:26.370 INFO:teuthology.orchestra.run.smithi099.stdout:(6/146): ceph-mds-18.2.2-1015.gd0e392d9.el8.x86 3.6 MB/s | 2.3 MB 00:00 2024-03-29T09:05:26.813 INFO:teuthology.orchestra.run.smithi133.stdout:(2/146): ceph-fuse-18.2.2-1015.gd0e392d9.el8.x8 1.3 MB/s | 854 kB 00:00 2024-03-29T09:05:26.880 INFO:teuthology.orchestra.run.smithi133.stdout:(3/146): ceph-immutable-object-cache-18.2.2-101 2.4 MB/s | 166 kB 00:00 2024-03-29T09:05:26.953 INFO:teuthology.orchestra.run.smithi099.stdout:(7/146): ceph-common-18.2.2-1015.gd0e392d9.el8. 12 MB/s | 22 MB 00:01 2024-03-29T09:05:27.045 INFO:teuthology.orchestra.run.smithi099.stdout:(8/146): ceph-mon-18.2.2-1015.gd0e392d9.el8.x86 5.7 MB/s | 4.8 MB 00:00 2024-03-29T09:05:27.146 INFO:teuthology.orchestra.run.smithi099.stdout:(9/146): ceph-selinux-18.2.2-1015.gd0e392d9.el8 242 kB/s | 24 kB 00:00 2024-03-29T09:05:27.747 INFO:teuthology.orchestra.run.smithi133.stdout:(4/146): ceph-mds-18.2.2-1015.gd0e392d9.el8.x86 2.6 MB/s | 2.3 MB 00:00 2024-03-29T09:05:28.048 INFO:teuthology.orchestra.run.smithi133.stdout:(5/146): ceph-base-18.2.2-1015.gd0e392d9.el8.x8 2.6 MB/s | 5.5 MB 00:02 2024-03-29T09:05:28.412 INFO:teuthology.orchestra.run.smithi099.stdout:(10/146): ceph-radosgw-18.2.2-1015.gd0e392d9.el 9.0 MB/s | 13 MB 00:01 2024-03-29T09:05:28.456 INFO:teuthology.orchestra.run.smithi133.stdout:(6/146): ceph-mgr-18.2.2-1015.gd0e392d9.el8.x86 2.2 MB/s | 1.6 MB 00:00 2024-03-29T09:05:28.496 INFO:teuthology.orchestra.run.smithi099.stdout:(11/146): cephfs-mirror-18.2.2-1015.gd0e392d9.e 3.1 MB/s | 261 kB 00:00 2024-03-29T09:05:28.579 INFO:teuthology.orchestra.run.smithi099.stdout:(12/146): libcephfs-devel-18.2.2-1015.gd0e392d9 372 kB/s | 31 kB 00:00 2024-03-29T09:05:28.797 INFO:teuthology.orchestra.run.smithi099.stdout:(13/146): libcephfs2-18.2.2-1015.gd0e392d9.el8. 3.5 MB/s | 778 kB 00:00 2024-03-29T09:05:28.888 INFO:teuthology.orchestra.run.smithi099.stdout:(14/146): libcephsqlite-18.2.2-1015.gd0e392d9.e 1.5 MB/s | 142 kB 00:00 2024-03-29T09:05:28.980 INFO:teuthology.orchestra.run.smithi099.stdout:(15/146): librados-devel-18.2.2-1015.gd0e392d9. 1.5 MB/s | 139 kB 00:00 2024-03-29T09:05:29.139 INFO:teuthology.orchestra.run.smithi099.stdout:(16/146): libradosstriper1-18.2.2-1015.gd0e392d 2.9 MB/s | 474 kB 00:00 2024-03-29T09:05:29.374 INFO:teuthology.orchestra.run.smithi133.stdout:(7/146): ceph-mon-18.2.2-1015.gd0e392d9.el8.x86 3.6 MB/s | 4.8 MB 00:01 2024-03-29T09:05:29.823 INFO:teuthology.orchestra.run.smithi133.stdout:(8/146): ceph-common-18.2.2-1015.gd0e392d9.el8. 5.6 MB/s | 22 MB 00:03 2024-03-29T09:05:29.898 INFO:teuthology.orchestra.run.smithi133.stdout:(9/146): ceph-selinux-18.2.2-1015.gd0e392d9.el8 326 kB/s | 24 kB 00:00 2024-03-29T09:05:30.364 INFO:teuthology.orchestra.run.smithi099.stdout:(17/146): librgw2-18.2.2-1015.gd0e392d9.el8.x86 5.3 MB/s | 6.5 MB 00:01 2024-03-29T09:05:30.448 INFO:teuthology.orchestra.run.smithi099.stdout:(18/146): python3-ceph-argparse-18.2.2-1015.gd0 537 kB/s | 45 kB 00:00 2024-03-29T09:05:30.540 INFO:teuthology.orchestra.run.smithi099.stdout:(19/146): python3-ceph-common-18.2.2-1015.gd0e3 1.4 MB/s | 129 kB 00:00 2024-03-29T09:05:30.640 INFO:teuthology.orchestra.run.smithi099.stdout:(20/146): python3-cephfs-18.2.2-1015.gd0e392d9. 1.9 MB/s | 197 kB 00:00 2024-03-29T09:05:30.749 INFO:teuthology.orchestra.run.smithi099.stdout:(21/146): python3-rados-18.2.2-1015.gd0e392d9.e 3.3 MB/s | 371 kB 00:00 2024-03-29T09:05:30.849 INFO:teuthology.orchestra.run.smithi099.stdout:(22/146): python3-rbd-18.2.2-1015.gd0e392d9.el8 3.6 MB/s | 364 kB 00:00 2024-03-29T09:05:30.941 INFO:teuthology.orchestra.run.smithi099.stdout:(23/146): python3-rgw-18.2.2-1015.gd0e392d9.el8 1.1 MB/s | 106 kB 00:00 2024-03-29T09:05:31.057 INFO:teuthology.orchestra.run.smithi099.stdout:(24/146): rbd-fuse-18.2.2-1015.gd0e392d9.el8.x8 818 kB/s | 95 kB 00:00 2024-03-29T09:05:31.791 INFO:teuthology.orchestra.run.smithi099.stdout:(25/146): rbd-mirror-18.2.2-1015.gd0e392d9.el8. 4.7 MB/s | 3.5 MB 00:00 2024-03-29T09:05:31.891 INFO:teuthology.orchestra.run.smithi099.stdout:(26/146): rbd-nbd-18.2.2-1015.gd0e392d9.el8.x86 1.8 MB/s | 181 kB 00:00 2024-03-29T09:05:31.975 INFO:teuthology.orchestra.run.smithi099.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1015.g 279 kB/s | 23 kB 00:00 2024-03-29T09:05:32.067 INFO:teuthology.orchestra.run.smithi099.stdout:(28/146): ceph-mgr-cephadm-18.2.2-1015.gd0e392d 1.5 MB/s | 138 kB 00:00 2024-03-29T09:05:32.426 INFO:teuthology.orchestra.run.smithi099.stdout:(29/146): ceph-mgr-dashboard-18.2.2-1015.gd0e39 5.0 MB/s | 1.8 MB 00:00 2024-03-29T09:05:32.890 INFO:teuthology.orchestra.run.smithi133.stdout:(10/146): ceph-radosgw-18.2.2-1015.gd0e392d9.el 3.7 MB/s | 13 MB 00:03 2024-03-29T09:05:32.990 INFO:teuthology.orchestra.run.smithi133.stdout:(11/146): cephfs-mirror-18.2.2-1015.gd0e392d9.e 2.6 MB/s | 261 kB 00:00 2024-03-29T09:05:33.057 INFO:teuthology.orchestra.run.smithi133.stdout:(12/146): libcephfs-devel-18.2.2-1015.gd0e392d9 465 kB/s | 31 kB 00:00 2024-03-29T09:05:33.276 INFO:teuthology.orchestra.run.smithi099.stdout:(30/146): ceph-osd-18.2.2-1015.gd0e392d9.el8.x8 2.7 MB/s | 18 MB 00:06 2024-03-29T09:05:33.283 INFO:teuthology.orchestra.run.smithi133.stdout:(13/146): libcephfs2-18.2.2-1015.gd0e392d9.el8. 3.4 MB/s | 778 kB 00:00 2024-03-29T09:05:33.383 INFO:teuthology.orchestra.run.smithi133.stdout:(14/146): libcephsqlite-18.2.2-1015.gd0e392d9.e 1.4 MB/s | 142 kB 00:00 2024-03-29T09:05:33.450 INFO:teuthology.orchestra.run.smithi133.stdout:(15/146): librados-devel-18.2.2-1015.gd0e392d9. 2.0 MB/s | 139 kB 00:00 2024-03-29T09:05:33.592 INFO:teuthology.orchestra.run.smithi133.stdout:(16/146): libradosstriper1-18.2.2-1015.gd0e392d 3.3 MB/s | 474 kB 00:00 2024-03-29T09:05:33.801 INFO:teuthology.orchestra.run.smithi099.stdout:(31/146): ceph-test-18.2.2-1015.gd0e392d9.el8.x 7.1 MB/s | 47 MB 00:06 2024-03-29T09:05:33.851 INFO:teuthology.orchestra.run.smithi099.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1015.gd0 427 kB/s | 245 kB 00:00 2024-03-29T09:05:33.876 INFO:teuthology.orchestra.run.smithi099.stdout:(33/146): ceph-mgr-rook-18.2.2-1015.gd0e392d9.e 658 kB/s | 49 kB 00:00 2024-03-29T09:05:33.977 INFO:teuthology.orchestra.run.smithi099.stdout:(34/146): ceph-mgr-diskprediction-local-18.2.2- 4.8 MB/s | 7.4 MB 00:01 2024-03-29T09:05:34.002 INFO:teuthology.orchestra.run.smithi099.stdout:(35/146): ceph-prometheus-alerts-18.2.2-1015.gd 98 kB/s | 15 kB 00:00 2024-03-29T09:05:34.027 INFO:teuthology.orchestra.run.smithi099.stdout:(36/146): ceph-volume-18.2.2-1015.gd0e392d9.el8 1.7 MB/s | 263 kB 00:00 2024-03-29T09:05:34.061 INFO:teuthology.orchestra.run.smithi099.stdout:(37/146): cephfs-top-18.2.2-1015.gd0e392d9.el8. 310 kB/s | 18 kB 00:00 2024-03-29T09:05:34.094 INFO:teuthology.orchestra.run.smithi099.stdout:(38/146): cephadm-18.2.2-1015.gd0e392d9.el8.noa 1.4 MB/s | 173 kB 00:00 2024-03-29T09:05:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:(17/146): ceph-osd-18.2.2-1015.gd0e392d9.el8.x8 3.2 MB/s | 18 MB 00:05 2024-03-29T09:05:34.211 INFO:teuthology.orchestra.run.smithi099.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 404 kB/s | 59 kB 00:00 2024-03-29T09:05:34.228 INFO:teuthology.orchestra.run.smithi099.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 584 kB/s | 75 kB 00:00 2024-03-29T09:05:34.267 INFO:teuthology.orchestra.run.smithi133.stdout:(18/146): python3-ceph-argparse-18.2.2-1015.gd0 673 kB/s | 45 kB 00:00 2024-03-29T09:05:34.287 INFO:teuthology.orchestra.run.smithi099.stdout:(41/146): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2024-03-29T09:05:34.321 INFO:teuthology.orchestra.run.smithi099.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 473 kB/s | 44 kB 00:00 2024-03-29T09:05:34.334 INFO:teuthology.orchestra.run.smithi133.stdout:(19/146): python3-ceph-common-18.2.2-1015.gd0e3 1.9 MB/s | 129 kB 00:00 2024-03-29T09:05:34.346 INFO:teuthology.orchestra.run.smithi099.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2024-03-29T09:05:34.371 INFO:teuthology.orchestra.run.smithi099.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 661 kB/s | 33 kB 00:00 2024-03-29T09:05:34.397 INFO:teuthology.orchestra.run.smithi099.stdout:(45/146): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2024-03-29T09:05:34.409 INFO:teuthology.orchestra.run.smithi133.stdout:(20/146): python3-cephfs-18.2.2-1015.gd0e392d9. 2.6 MB/s | 197 kB 00:00 2024-03-29T09:05:34.430 INFO:teuthology.orchestra.run.smithi099.stdout:(46/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2024-03-29T09:05:34.456 INFO:teuthology.orchestra.run.smithi099.stdout:(47/146): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2024-03-29T09:05:34.481 INFO:teuthology.orchestra.run.smithi099.stdout:(48/146): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2024-03-29T09:05:34.506 INFO:teuthology.orchestra.run.smithi099.stdout:(49/146): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2024-03-29T09:05:34.526 INFO:teuthology.orchestra.run.smithi133.stdout:(21/146): python3-rados-18.2.2-1015.gd0e392d9.e 3.1 MB/s | 371 kB 00:00 2024-03-29T09:05:34.531 INFO:teuthology.orchestra.run.smithi099.stdout:(50/146): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2024-03-29T09:05:34.582 INFO:teuthology.orchestra.run.smithi099.stdout:(51/146): python3-logutils-0.3.5-11.el8.noarch. 979 kB/s | 49 kB 00:00 2024-03-29T09:05:34.607 INFO:teuthology.orchestra.run.smithi099.stdout:(52/146): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2024-03-29T09:05:34.632 INFO:teuthology.orchestra.run.smithi099.stdout:(53/146): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2024-03-29T09:05:34.657 INFO:teuthology.orchestra.run.smithi099.stdout:(54/146): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2024-03-29T09:05:34.683 INFO:teuthology.orchestra.run.smithi099.stdout:(55/146): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2024-03-29T09:05:34.732 INFO:teuthology.orchestra.run.smithi099.stdout:(56/146): python3-portend-2.6-1.el8.noarch.rpm 210 kB/s | 16 kB 00:00 2024-03-29T09:05:34.767 INFO:teuthology.orchestra.run.smithi099.stdout:(57/146): python3-kubernetes-11.0.0-6.el8.noarc 4.5 MB/s | 1.2 MB 00:00 2024-03-29T09:05:34.792 INFO:teuthology.orchestra.run.smithi099.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2024-03-29T09:05:34.817 INFO:teuthology.orchestra.run.smithi099.stdout:(59/146): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2024-03-29T09:05:34.851 INFO:teuthology.orchestra.run.smithi099.stdout:(60/146): python3-simplegeneric-0.8.1-17.el8.no 321 kB/s | 19 kB 00:00 2024-03-29T09:05:34.876 INFO:teuthology.orchestra.run.smithi099.stdout:(61/146): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2024-03-29T09:05:34.901 INFO:teuthology.orchestra.run.smithi099.stdout:(62/146): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2024-03-29T09:05:34.926 INFO:teuthology.orchestra.run.smithi099.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2024-03-29T09:05:34.952 INFO:teuthology.orchestra.run.smithi099.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2024-03-29T09:05:34.977 INFO:teuthology.orchestra.run.smithi099.stdout:(65/146): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2024-03-29T09:05:34.994 INFO:teuthology.orchestra.run.smithi099.stdout:(66/146): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2024-03-29T09:05:35.019 INFO:teuthology.orchestra.run.smithi099.stdout:(67/146): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2024-03-29T09:05:35.036 INFO:teuthology.orchestra.run.smithi099.stdout:(68/146): python3-zc-lockfile-2.0-2.el8.noarch. 548 kB/s | 23 kB 00:00 2024-03-29T09:05:35.053 INFO:teuthology.orchestra.run.smithi099.stdout:(69/146): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-03-29T09:05:35.060 INFO:teuthology.orchestra.run.smithi133.stdout:(22/146): ceph-test-18.2.2-1015.gd0e392d9.el8.x 9.2 MB/s | 47 MB 00:05 2024-03-29T09:05:35.078 INFO:teuthology.orchestra.run.smithi099.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2024-03-29T09:05:35.085 INFO:teuthology.orchestra.run.smithi133.stdout:(23/146): python3-rbd-18.2.2-1015.gd0e392d9.el8 652 kB/s | 364 kB 00:00 2024-03-29T09:05:35.177 INFO:teuthology.orchestra.run.smithi133.stdout:(24/146): librgw2-18.2.2-1015.gd0e392d9.el8.x86 4.1 MB/s | 6.5 MB 00:01 2024-03-29T09:05:35.180 INFO:teuthology.orchestra.run.smithi099.stdout:(71/146): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2024-03-29T09:05:35.203 INFO:teuthology.orchestra.run.smithi133.stdout:(25/146): python3-rgw-18.2.2-1015.gd0e392d9.el8 750 kB/s | 106 kB 00:00 2024-03-29T09:05:35.246 INFO:teuthology.orchestra.run.smithi099.stdout:(72/146): libverto-devel-0.3.0-5.el8.x86_64.rpm 93 kB/s | 18 kB 00:00 2024-03-29T09:05:35.253 INFO:teuthology.orchestra.run.smithi133.stdout:(26/146): rbd-fuse-18.2.2-1015.gd0e392d9.el8.x8 569 kB/s | 95 kB 00:00 2024-03-29T09:05:35.271 INFO:teuthology.orchestra.run.smithi099.stdout:(73/146): python3-jwt-1.6.1-2.el8.noarch.rpm 229 kB/s | 43 kB 00:00 2024-03-29T09:05:35.304 INFO:teuthology.orchestra.run.smithi133.stdout:(27/146): ceph-grafana-dashboards-18.2.2-1015.g 468 kB/s | 23 kB 00:00 2024-03-29T09:05:35.321 INFO:teuthology.orchestra.run.smithi099.stdout:(74/146): python3-oauthlib-2.1.0-1.el8.noarch.r 1.1 MB/s | 155 kB 00:00 2024-03-29T09:05:35.329 INFO:teuthology.orchestra.run.smithi133.stdout:(28/146): rbd-nbd-18.2.2-1015.gd0e392d9.el8.x86 2.4 MB/s | 181 kB 00:00 2024-03-29T09:05:35.354 INFO:teuthology.orchestra.run.smithi133.stdout:(29/146): ceph-mgr-cephadm-18.2.2-1015.gd0e392d 2.7 MB/s | 138 kB 00:00 2024-03-29T09:05:35.355 INFO:teuthology.orchestra.run.smithi099.stdout:(75/146): python3-pycparser-2.14-14.el8.noarch. 1.0 MB/s | 109 kB 00:00 2024-03-29T09:05:35.447 INFO:teuthology.orchestra.run.smithi099.stdout:(76/146): libaio-devel-0.3.112-1.el8.x86_64.rpm 149 kB/s | 19 kB 00:00 2024-03-29T09:05:35.472 INFO:teuthology.orchestra.run.smithi099.stdout:(77/146): python3-requests-oauthlib-1.0.0-1.el8 212 kB/s | 43 kB 00:00 2024-03-29T09:05:35.498 INFO:teuthology.orchestra.run.smithi099.stdout:(78/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 308 kB/s | 44 kB 00:00 2024-03-29T09:05:35.564 INFO:teuthology.orchestra.run.smithi099.stdout:(79/146): pcre2-devel-10.32-2.el8.x86_64.rpm 5.1 MB/s | 605 kB 00:00 2024-03-29T09:05:35.631 INFO:teuthology.orchestra.run.smithi099.stdout:(80/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2024-03-29T09:05:35.665 INFO:teuthology.orchestra.run.smithi099.stdout:(81/146): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2024-03-29T09:05:35.679 INFO:teuthology.orchestra.run.smithi133.stdout:(30/146): rbd-mirror-18.2.2-1015.gd0e392d9.el8. 6.9 MB/s | 3.5 MB 00:00 2024-03-29T09:05:35.690 INFO:teuthology.orchestra.run.smithi099.stdout:(82/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2024-03-29T09:05:35.729 INFO:teuthology.orchestra.run.smithi133.stdout:(31/146): ceph-mgr-dashboard-18.2.2-1015.gd0e39 4.5 MB/s | 1.8 MB 00:00 2024-03-29T09:05:35.754 INFO:teuthology.orchestra.run.smithi133.stdout:(32/146): ceph-mgr-modules-core-18.2.2-1015.gd0 3.2 MB/s | 245 kB 00:00 2024-03-29T09:05:35.780 INFO:teuthology.orchestra.run.smithi133.stdout:(33/146): ceph-mgr-rook-18.2.2-1015.gd0e392d9.e 980 kB/s | 49 kB 00:00 2024-03-29T09:05:35.790 INFO:teuthology.orchestra.run.smithi099.stdout:(83/146): libsepol-devel-2.9-3.el8.x86_64.rpm 694 kB/s | 87 kB 00:00 2024-03-29T09:05:35.813 INFO:teuthology.orchestra.run.smithi133.stdout:(34/146): ceph-prometheus-alerts-18.2.2-1015.gd 252 kB/s | 15 kB 00:00 2024-03-29T09:05:35.816 INFO:teuthology.orchestra.run.smithi099.stdout:(84/146): keyutils-libs-devel-1.5.10-9.el8.x86_ 261 kB/s | 48 kB 00:00 2024-03-29T09:05:35.841 INFO:teuthology.orchestra.run.smithi099.stdout:(85/146): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 383 kB/s | 58 kB 00:00 2024-03-29T09:05:35.863 INFO:teuthology.orchestra.run.smithi133.stdout:(35/146): ceph-volume-18.2.2-1015.gd0e392d9.el8 3.1 MB/s | 263 kB 00:00 2024-03-29T09:05:35.925 INFO:teuthology.orchestra.run.smithi099.stdout:(86/146): libcom_err-devel-1.45.6-4.el8.x86_64. 289 kB/s | 39 kB 00:00 2024-03-29T09:05:35.950 INFO:teuthology.orchestra.run.smithi099.stdout:(87/146): fuse-2.9.7-17.el8.x86_64.rpm 622 kB/s | 83 kB 00:00 2024-03-29T09:05:35.972 INFO:teuthology.orchestra.run.smithi133.stdout:(36/146): ceph-mgr-diskprediction-local-18.2.2- 12 MB/s | 7.4 MB 00:00 2024-03-29T09:05:35.997 INFO:teuthology.orchestra.run.smithi133.stdout:(37/146): cephadm-18.2.2-1015.gd0e392d9.el8.noa 939 kB/s | 173 kB 00:00 2024-03-29T09:05:36.056 INFO:teuthology.orchestra.run.smithi133.stdout:(38/146): cephfs-top-18.2.2-1015.gd0e392d9.el8. 94 kB/s | 18 kB 00:00 2024-03-29T09:05:36.071 INFO:teuthology.orchestra.run.smithi099.stdout:(88/146): libgfortran-8.5.0-20.el8.x86_64.rpm 2.7 MB/s | 645 kB 00:00 2024-03-29T09:05:36.125 INFO:teuthology.orchestra.run.smithi099.stdout:(89/146): libquadmath-8.5.0-20.el8.x86_64.rpm 861 kB/s | 172 kB 00:00 2024-03-29T09:05:36.159 INFO:teuthology.orchestra.run.smithi099.stdout:(90/146): libuuid-devel-2.32.1-43.el8.x86_64.rp 477 kB/s | 100 kB 00:00 2024-03-29T09:05:36.251 INFO:teuthology.orchestra.run.smithi099.stdout:(91/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-03-29T09:05:36.252 INFO:teuthology.orchestra.run.smithi133.stdout:(39/146): liboath-2.6.2-3.el8.x86_64.rpm 307 kB/s | 59 kB 00:00 2024-03-29T09:05:36.265 INFO:teuthology.orchestra.run.smithi133.stdout:(40/146): libunwind-1.3.1-3.el8.x86_64.rpm 360 kB/s | 75 kB 00:00 2024-03-29T09:05:36.276 INFO:teuthology.orchestra.run.smithi099.stdout:(92/146): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 482 kB/s | 73 kB 00:00 2024-03-29T09:05:36.332 INFO:teuthology.orchestra.run.smithi133.stdout:(41/146): gperftools-libs-2.7-9.el8.x86_64.rpm 853 kB/s | 306 kB 00:00 2024-03-29T09:05:36.334 INFO:teuthology.orchestra.run.smithi099.stdout:(93/146): NetworkManager-initscripts-updown-1.4 821 kB/s | 144 kB 00:00 2024-03-29T09:05:36.357 INFO:teuthology.orchestra.run.smithi133.stdout:(42/146): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 474 kB/s | 44 kB 00:00 2024-03-29T09:05:36.360 INFO:teuthology.orchestra.run.smithi099.stdout:(94/146): elfutils-libelf-devel-0.189-3.el8.x86 568 kB/s | 62 kB 00:00 2024-03-29T09:05:36.410 INFO:teuthology.orchestra.run.smithi099.stdout:(95/146): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.1 MB/s | 562 kB 00:00 2024-03-29T09:05:36.429 INFO:teuthology.orchestra.run.smithi133.stdout:(43/146): python3-beautifulsoup4-4.6.3-2.el8.1. 1.9 MB/s | 185 kB 00:00 2024-03-29T09:05:36.458 INFO:teuthology.orchestra.run.smithi133.stdout:(44/146): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2024-03-29T09:05:36.469 INFO:teuthology.orchestra.run.smithi099.stdout:(96/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 434 kB/s | 47 kB 00:00 2024-03-29T09:05:36.484 INFO:teuthology.orchestra.run.smithi133.stdout:(45/146): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2024-03-29T09:05:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:(46/146): python3-certifi-2018.10.15-7.el8.noar 204 kB/s | 16 kB 00:00 2024-03-29T09:05:36.561 INFO:teuthology.orchestra.run.smithi099.stdout:(97/146): openssl-devel-1.1.1k-12.el8_9.x86_64. 10 MB/s | 2.3 MB 00:00 2024-03-29T09:05:36.567 INFO:teuthology.orchestra.run.smithi133.stdout:(47/146): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2024-03-29T09:05:36.592 INFO:teuthology.orchestra.run.smithi133.stdout:(48/146): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2024-03-29T09:05:36.594 INFO:teuthology.orchestra.run.smithi099.stdout:(98/146): python3-cryptography-3.2.1-7.el8_9.x8 3.0 MB/s | 559 kB 00:00 2024-03-29T09:05:36.617 INFO:teuthology.orchestra.run.smithi133.stdout:(49/146): python3-google-auth-1.1.1-10.el8.noar 748 kB/s | 81 kB 00:00 2024-03-29T09:05:36.619 INFO:teuthology.orchestra.run.smithi099.stdout:(99/146): python3-packaging-16.8-9.el8.noarch.r 321 kB/s | 48 kB 00:00 2024-03-29T09:05:36.643 INFO:teuthology.orchestra.run.smithi133.stdout:(50/146): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2024-03-29T09:05:36.685 INFO:teuthology.orchestra.run.smithi133.stdout:(51/146): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2024-03-29T09:05:36.719 INFO:teuthology.orchestra.run.smithi133.stdout:(52/146): python3-logutils-0.3.5-11.el8.noarch. 643 kB/s | 49 kB 00:00 2024-03-29T09:05:36.720 INFO:teuthology.orchestra.run.smithi099.stdout:(100/146): python3-jsonpointer-1.10-11.el8.noar 126 kB/s | 20 kB 00:00 2024-03-29T09:05:36.737 INFO:teuthology.orchestra.run.smithi099.stdout:(101/146): python3-pyasn1-modules-0.3.7-6.el8.n 939 kB/s | 110 kB 00:00 2024-03-29T09:05:36.760 INFO:teuthology.orchestra.run.smithi133.stdout:(53/146): python3-kubernetes-11.0.0-6.el8.noarc 8.2 MB/s | 1.2 MB 00:00 2024-03-29T09:05:36.762 INFO:teuthology.orchestra.run.smithi099.stdout:(102/146): python3-prettytable-0.7.2-14.el8.noa 262 kB/s | 44 kB 00:00 2024-03-29T09:05:36.786 INFO:teuthology.orchestra.run.smithi133.stdout:(54/146): python3-more-itertools-7.2.0-3.el8.no 583 kB/s | 59 kB 00:00 2024-03-29T09:05:36.838 INFO:teuthology.orchestra.run.smithi099.stdout:(103/146): python3-werkzeug-0.12.2-4.el8.noarch 6.0 MB/s | 457 kB 00:00 2024-03-29T09:05:36.839 INFO:teuthology.orchestra.run.smithi133.stdout:(55/146): python3-natsort-7.1.1-2.el8.noarch.rp 500 kB/s | 60 kB 00:00 2024-03-29T09:05:36.863 INFO:teuthology.orchestra.run.smithi099.stdout:(104/146): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2024-03-29T09:05:36.878 INFO:teuthology.orchestra.run.smithi133.stdout:(56/146): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2024-03-29T09:05:36.896 INFO:teuthology.orchestra.run.smithi099.stdout:(105/146): python3-jsonpatch-1.21-2.el8.noarch. 151 kB/s | 27 kB 00:00 2024-03-29T09:05:36.903 INFO:teuthology.orchestra.run.smithi133.stdout:(57/146): python3-portend-2.6-1.el8.noarch.rpm 247 kB/s | 16 kB 00:00 2024-03-29T09:05:36.928 INFO:teuthology.orchestra.run.smithi133.stdout:(58/146): python3-repoze-lru-0.7-6.el8.noarch.r 382 kB/s | 34 kB 00:00 2024-03-29T09:05:36.953 INFO:teuthology.orchestra.run.smithi133.stdout:(59/146): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2024-03-29T09:05:36.979 INFO:teuthology.orchestra.run.smithi133.stdout:(60/146): python3-rsa-4.9-2.el8.noarch.rpm 852 kB/s | 64 kB 00:00 2024-03-29T09:05:37.004 INFO:teuthology.orchestra.run.smithi133.stdout:(61/146): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2024-03-29T09:05:37.005 INFO:teuthology.orchestra.run.smithi099.stdout:(106/146): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2024-03-29T09:05:37.030 INFO:teuthology.orchestra.run.smithi133.stdout:(62/146): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2024-03-29T09:05:37.047 INFO:teuthology.orchestra.run.smithi099.stdout:(107/146): bison-3.0.4-10.el8.x86_64.rpm 3.2 MB/s | 688 kB 00:00 2024-03-29T09:05:37.054 INFO:teuthology.orchestra.run.smithi133.stdout:(63/146): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2024-03-29T09:05:37.080 INFO:teuthology.orchestra.run.smithi133.stdout:(64/146): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2024-03-29T09:05:37.080 INFO:teuthology.orchestra.run.smithi099.stdout:(108/146): libtool-2.4.6-25.el8.x86_64.rpm 3.8 MB/s | 709 kB 00:00 2024-03-29T09:05:37.105 INFO:teuthology.orchestra.run.smithi133.stdout:(65/146): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2024-03-29T09:05:37.106 INFO:teuthology.orchestra.run.smithi099.stdout:(109/146): python3-markupsafe-0.23-19.el8.x86_6 387 kB/s | 39 kB 00:00 2024-03-29T09:05:37.130 INFO:teuthology.orchestra.run.smithi133.stdout:(66/146): python3-webob-1.8.5-1.el8.1.noarch.rp 3.2 MB/s | 251 kB 00:00 2024-03-29T09:05:37.156 INFO:teuthology.orchestra.run.smithi133.stdout:(67/146): python3-websocket-client-0.56.0-5.el8 813 kB/s | 61 kB 00:00 2024-03-29T09:05:37.164 INFO:teuthology.orchestra.run.smithi099.stdout:(110/146): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.7 MB/s | 103 kB 00:00 2024-03-29T09:05:37.181 INFO:teuthology.orchestra.run.smithi133.stdout:(68/146): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2024-03-29T09:05:37.181 INFO:teuthology.orchestra.run.smithi099.stdout:(111/146): uuid-1.6.2-43.el8.x86_64.rpm 638 kB/s | 64 kB 00:00 2024-03-29T09:05:37.206 INFO:teuthology.orchestra.run.smithi133.stdout:(69/146): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2024-03-29T09:05:37.207 INFO:teuthology.orchestra.run.smithi099.stdout:(112/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2024-03-29T09:05:37.265 INFO:teuthology.orchestra.run.smithi133.stdout:(70/146): xmlstarlet-1.6.1-20.el8.x86_64.rpm 836 kB/s | 70 kB 00:00 2024-03-29T09:05:37.307 INFO:teuthology.orchestra.run.smithi099.stdout:(113/146): python3-jinja2-2.10.1-3.el8.noarch.r 4.2 MB/s | 538 kB 00:00 2024-03-29T09:05:37.407 INFO:teuthology.orchestra.run.smithi133.stdout:(71/146): python3-packaging-16.8-9.el8.noarch.r 241 kB/s | 48 kB 00:00 2024-03-29T09:05:37.449 INFO:teuthology.orchestra.run.smithi133.stdout:(72/146): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2024-03-29T09:05:37.449 INFO:teuthology.orchestra.run.smithi099.stdout:(114/146): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2024-03-29T09:05:37.474 INFO:teuthology.orchestra.run.smithi133.stdout:(73/146): python3-jsonpointer-1.10-11.el8.noarc 98 kB/s | 20 kB 00:00 2024-03-29T09:05:37.482 INFO:teuthology.orchestra.run.smithi099.stdout:(115/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2024-03-29T09:05:37.516 INFO:teuthology.orchestra.run.smithi133.stdout:(74/146): python3-prettytable-0.7.2-14.el8.noar 403 kB/s | 44 kB 00:00 2024-03-29T09:05:37.549 INFO:teuthology.orchestra.run.smithi099.stdout:(116/146): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2024-03-29T09:05:37.574 INFO:teuthology.orchestra.run.smithi099.stdout:(117/146): python3-numpy-f2py-1.14.3-10.el8.x86 1.8 MB/s | 225 kB 00:00 2024-03-29T09:05:37.636 INFO:teuthology.orchestra.run.smithi133.stdout:(75/146): python3-jsonpatch-1.21-2.el8.noarch.r 170 kB/s | 27 kB 00:00 2024-03-29T09:05:37.650 INFO:teuthology.orchestra.run.smithi099.stdout:(118/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2024-03-29T09:05:37.675 INFO:teuthology.orchestra.run.smithi133.stdout:(76/146): python3-pyasn1-modules-0.3.7-6.el8.no 487 kB/s | 110 kB 00:00 2024-03-29T09:05:37.700 INFO:teuthology.orchestra.run.smithi133.stdout:(77/146): python3-pyasn1-0.3.7-6.el8.noarch.rpm 688 kB/s | 126 kB 00:00 2024-03-29T09:05:37.742 INFO:teuthology.orchestra.run.smithi099.stdout:(119/146): python3-mako-1.0.6-14.el8.noarch.rpm 940 kB/s | 157 kB 00:00 2024-03-29T09:05:37.833 INFO:teuthology.orchestra.run.smithi133.stdout:(78/146): python3-werkzeug-0.12.2-4.el8.noarch. 2.3 MB/s | 457 kB 00:00 2024-03-29T09:05:37.884 INFO:teuthology.orchestra.run.smithi133.stdout:(79/146): bison-3.0.4-10.el8.x86_64.rpm 3.2 MB/s | 688 kB 00:00 2024-03-29T09:05:37.909 INFO:teuthology.orchestra.run.smithi099.stdout:(120/146): openblas-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.0 MB 00:00 2024-03-29T09:05:37.926 INFO:teuthology.orchestra.run.smithi133.stdout:(80/146): flex-2.6.1-9.el8.x86_64.rpm 1.4 MB/s | 320 kB 00:00 2024-03-29T09:05:37.984 INFO:teuthology.orchestra.run.smithi099.stdout:(121/146): openblas-threads-0.3.15-6.el8.x86_64 12 MB/s | 5.2 MB 00:00 2024-03-29T09:05:37.984 INFO:teuthology.orchestra.run.smithi133.stdout:(81/146): python3-markupsafe-0.23-19.el8.x86_64 387 kB/s | 39 kB 00:00 2024-03-29T09:05:38.025 INFO:teuthology.orchestra.run.smithi099.stdout:(122/146): automake-1.16.1-8.el8.noarch.rpm 2.5 MB/s | 714 kB 00:00 2024-03-29T09:05:38.048 INFO:teuthology.orchestra.run.smithi133.stdout:(82/146): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.1 MB/s | 259 kB 00:00 2024-03-29T09:05:38.051 INFO:teuthology.orchestra.run.smithi099.stdout:(123/146): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2024-03-29T09:05:38.102 INFO:teuthology.orchestra.run.smithi133.stdout:(83/146): libtool-2.4.6-25.el8.x86_64.rpm 2.6 MB/s | 709 kB 00:00 2024-03-29T09:05:38.144 INFO:teuthology.orchestra.run.smithi133.stdout:(84/146): uuid-1.6.2-43.el8.x86_64.rpm 403 kB/s | 64 kB 00:00 2024-03-29T09:05:38.168 INFO:teuthology.orchestra.run.smithi099.stdout:(124/146): python3-pytz-2017.2-11.el8.noarch.rp 382 kB/s | 54 kB 00:00 2024-03-29T09:05:38.177 INFO:teuthology.orchestra.run.smithi133.stdout:(85/146): python3-pyOpenSSL-19.0.0-1.el8.noarch 801 kB/s | 103 kB 00:00 2024-03-29T09:05:38.210 INFO:teuthology.orchestra.run.smithi099.stdout:(125/146): jq-1.6-7.el8.x86_64.rpm 898 kB/s | 202 kB 00:00 2024-03-29T09:05:38.320 INFO:teuthology.orchestra.run.smithi133.stdout:(86/146): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2024-03-29T09:05:38.353 INFO:teuthology.orchestra.run.smithi133.stdout:(87/146): python3-jinja2-2.10.1-3.el8.noarch.rp 2.5 MB/s | 538 kB 00:00 2024-03-29T09:05:38.402 INFO:teuthology.orchestra.run.smithi099.stdout:(126/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 805 kB/s | 188 kB 00:00 2024-03-29T09:05:38.469 INFO:teuthology.orchestra.run.smithi099.stdout:(127/146): postgresql-10.23-4.module+el8.9.0+21 5.8 MB/s | 1.5 MB 00:00 2024-03-29T09:05:38.486 INFO:teuthology.orchestra.run.smithi133.stdout:(88/146): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2024-03-29T09:05:38.520 INFO:teuthology.orchestra.run.smithi133.stdout:(89/146): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2024-03-29T09:05:38.554 INFO:teuthology.orchestra.run.smithi133.stdout:(90/146): socat-1.7.4.1-1.el8.x86_64.rpm 1.3 MB/s | 323 kB 00:00 2024-03-29T09:05:38.577 INFO:teuthology.orchestra.run.smithi099.stdout:(128/146): postgresql-contrib-10.23-4.module+el 4.5 MB/s | 812 kB 00:00 2024-03-29T09:05:38.646 INFO:teuthology.orchestra.run.smithi133.stdout:(91/146): librdkafka-0.11.4-3.el8.x86_64.rpm 2.2 MB/s | 354 kB 00:00 2024-03-29T09:05:38.704 INFO:teuthology.orchestra.run.smithi133.stdout:(92/146): python3-mako-1.0.6-14.el8.noarch.rpm 1.0 MB/s | 157 kB 00:00 2024-03-29T09:05:38.755 INFO:teuthology.orchestra.run.smithi099.stdout:(129/146): python3-scipy-1.0.0-21.module+el8.9. 19 MB/s | 14 MB 00:00 2024-03-29T09:05:38.854 INFO:teuthology.orchestra.run.smithi133.stdout:(93/146): openblas-threads-0.3.15-6.el8.x86_64. 16 MB/s | 5.2 MB 00:00 2024-03-29T09:05:38.903 INFO:teuthology.orchestra.run.smithi099.stdout:(130/146): postgresql-server-10.23-4.module+el8 12 MB/s | 5.1 MB 00:00 2024-03-29T09:05:38.905 INFO:teuthology.orchestra.run.smithi133.stdout:(94/146): automake-1.16.1-8.el8.noarch.rpm 3.5 MB/s | 714 kB 00:00 2024-03-29T09:05:38.930 INFO:teuthology.orchestra.run.smithi133.stdout:(95/146): libpq-13.11-1.el8.x86_64.rpm 2.5 MB/s | 198 kB 00:00 2024-03-29T09:05:39.005 INFO:teuthology.orchestra.run.smithi133.stdout:(96/146): openblas-0.3.15-6.el8.x86_64.rpm 14 MB/s | 5.0 MB 00:00 2024-03-29T09:05:39.037 INFO:teuthology.orchestra.run.smithi099.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.7 MB/s | 232 kB 00:00 2024-03-29T09:05:39.073 INFO:teuthology.orchestra.run.smithi133.stdout:(97/146): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-03-29T09:05:39.098 INFO:teuthology.orchestra.run.smithi133.stdout:(98/146): python3-pytz-2017.2-11.el8.noarch.rpm 324 kB/s | 54 kB 00:00 2024-03-29T09:05:39.154 INFO:teuthology.orchestra.run.smithi099.stdout:(132/146): librbd1-18.2.2-1015.gd0e392d9.el8.x8 9.0 MB/s | 3.6 MB 00:00 2024-03-29T09:05:39.215 INFO:teuthology.orchestra.run.smithi133.stdout:(99/146): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.3 MB/s | 188 kB 00:00 2024-03-29T09:05:39.229 INFO:teuthology.orchestra.run.smithi099.stdout:(133/146): elfutils-libs-0.189-3.el8.x86_64.rpm 1.5 MB/s | 303 kB 00:00 2024-03-29T09:05:39.279 INFO:teuthology.orchestra.run.smithi099.stdout:(134/146): librados2-18.2.2-1015.gd0e392d9.el8. 5.2 MB/s | 3.7 MB 00:00 2024-03-29T09:05:39.298 INFO:teuthology.orchestra.run.smithi133.stdout:(100/146): postgresql-10.23-4.module+el8.9.0+21 7.5 MB/s | 1.5 MB 00:00 2024-03-29T09:05:39.304 INFO:teuthology.orchestra.run.smithi099.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 1.5 MB/s | 237 kB 00:00 2024-03-29T09:05:39.390 INFO:teuthology.orchestra.run.smithi133.stdout:(101/146): postgresql-contrib-10.23-4.module+el 4.5 MB/s | 812 kB 00:00 2024-03-29T09:05:39.471 INFO:teuthology.orchestra.run.smithi099.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 742 kB/s | 179 kB 00:00 2024-03-29T09:05:39.530 INFO:teuthology.orchestra.run.smithi099.stdout:(137/146): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2024-03-29T09:05:39.549 INFO:teuthology.orchestra.run.smithi133.stdout:(102/146): python3-scipy-1.0.0-21.module+el8.9. 25 MB/s | 14 MB 00:00 2024-03-29T09:05:39.555 INFO:teuthology.orchestra.run.smithi099.stdout:(138/146): libuuid-2.32.1-43.el8.x86_64.rpm 359 kB/s | 99 kB 00:00 2024-03-29T09:05:39.574 INFO:teuthology.orchestra.run.smithi133.stdout:(103/146): libverto-devel-0.3.0-5.el8.x86_64.rp 97 kB/s | 18 kB 00:00 2024-03-29T09:05:39.597 INFO:teuthology.orchestra.run.smithi099.stdout:(139/146): util-linux-user-2.32.1-43.el8.x86_64 819 kB/s | 103 kB 00:00 2024-03-29T09:05:39.674 INFO:teuthology.orchestra.run.smithi133.stdout:(104/146): python3-jwt-1.6.1-2.el8.noarch.rpm 340 kB/s | 43 kB 00:00 2024-03-29T09:05:39.691 INFO:teuthology.orchestra.run.smithi133.stdout:(105/146): python3-oauthlib-2.1.0-1.el8.noarch. 1.3 MB/s | 155 kB 00:00 2024-03-29T09:05:39.697 INFO:teuthology.orchestra.run.smithi099.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2024-03-29T09:05:39.722 INFO:teuthology.orchestra.run.smithi099.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2024-03-29T09:05:39.748 INFO:teuthology.orchestra.run.smithi099.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2024-03-29T09:05:39.775 INFO:teuthology.orchestra.run.smithi133.stdout:(106/146): postgresql-server-10.23-4.module+el8 11 MB/s | 5.1 MB 00:00 2024-03-29T09:05:39.817 INFO:teuthology.orchestra.run.smithi133.stdout:(107/146): python3-pycparser-2.14-14.el8.noarch 768 kB/s | 109 kB 00:00 2024-03-29T09:05:39.898 INFO:teuthology.orchestra.run.smithi099.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 3.5 MB/s | 711 kB 00:00 2024-03-29T09:05:39.932 INFO:teuthology.orchestra.run.smithi099.stdout:(144/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.8 MB/s | 333 kB 00:00 2024-03-29T09:05:39.933 INFO:teuthology.orchestra.run.smithi133.stdout:(108/146): python3-requests-oauthlib-1.0.0-1.el 176 kB/s | 43 kB 00:00 2024-03-29T09:05:39.959 INFO:teuthology.orchestra.run.smithi133.stdout:(109/146): libzstd-devel-1.4.4-1.el8.x86_64.rpm 309 kB/s | 44 kB 00:00 2024-03-29T09:05:39.982 INFO:teuthology.orchestra.run.smithi099.stdout:(145/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 5.7 MB/s | 1.5 MB 00:00 2024-03-29T09:05:39.984 INFO:teuthology.orchestra.run.smithi133.stdout:(110/146): libaio-devel-0.3.112-1.el8.x86_64.rp 89 kB/s | 19 kB 00:00 2024-03-29T09:05:40.085 INFO:teuthology.orchestra.run.smithi133.stdout:(111/146): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2024-03-29T09:05:40.099 INFO:teuthology.orchestra.run.smithi099.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.3 MB/s | 1.1 MB 00:00 2024-03-29T09:05:40.103 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-03-29T09:05:40.103 INFO:teuthology.orchestra.run.smithi099.stdout:Total 14 MB/s | 213 MB 00:15 2024-03-29T09:05:40.110 INFO:teuthology.orchestra.run.smithi133.stdout:(112/146): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2024-03-29T09:05:40.135 INFO:teuthology.orchestra.run.smithi133.stdout:(113/146): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-29T09:05:40.203 INFO:teuthology.orchestra.run.smithi133.stdout:(114/146): libselinux-devel-2.9-5.el8.x86_64.rp 1.7 MB/s | 200 kB 00:00 2024-03-29T09:05:40.228 INFO:teuthology.orchestra.run.smithi133.stdout:(115/146): keyutils-libs-devel-1.5.10-9.el8.x86 410 kB/s | 48 kB 00:00 2024-03-29T09:05:40.253 INFO:teuthology.orchestra.run.smithi133.stdout:(116/146): libsepol-devel-2.9-3.el8.x86_64.rpm 742 kB/s | 87 kB 00:00 2024-03-29T09:05:40.303 INFO:teuthology.orchestra.run.smithi133.stdout:(117/146): zlib-devel-1.2.11-18.el8_5.x86_64.rp 573 kB/s | 58 kB 00:00 2024-03-29T09:05:40.370 INFO:teuthology.orchestra.run.smithi133.stdout:(118/146): fuse-2.9.7-17.el8.x86_64.rpm 710 kB/s | 83 kB 00:00 2024-03-29T09:05:40.395 INFO:teuthology.orchestra.run.smithi133.stdout:(119/146): libcom_err-devel-1.45.6-4.el8.x86_64 231 kB/s | 39 kB 00:00 2024-03-29T09:05:40.471 INFO:teuthology.orchestra.run.smithi133.stdout:(120/146): libgfortran-8.5.0-20.el8.x86_64.rpm 3.8 MB/s | 645 kB 00:00 2024-03-29T09:05:40.496 INFO:teuthology.orchestra.run.smithi133.stdout:(121/146): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2024-03-29T09:05:40.638 INFO:teuthology.orchestra.run.smithi133.stdout:(122/146): libuuid-devel-2.32.1-43.el8.x86_64.r 412 kB/s | 100 kB 00:00 2024-03-29T09:05:40.979 INFO:teuthology.orchestra.run.smithi133.stdout:(123/146): xfsprogs-devel-5.0.0-12.el8.x86_64.r 150 kB/s | 73 kB 00:00 2024-03-29T09:05:41.014 INFO:teuthology.orchestra.run.smithi133.stdout:(124/146): NetworkManager-initscripts-updown-1. 383 kB/s | 144 kB 00:00 2024-03-29T09:05:41.039 INFO:teuthology.orchestra.run.smithi133.stdout:(125/146): python3-cffi-1.11.5-6.el8.x86_64.rpm 419 kB/s | 238 kB 00:00 2024-03-29T09:05:41.073 INFO:teuthology.orchestra.run.smithi133.stdout:(126/146): elfutils-libelf-devel-0.189-3.el8.x8 660 kB/s | 62 kB 00:00 2024-03-29T09:05:41.139 INFO:teuthology.orchestra.run.smithi133.stdout:(127/146): krb5-devel-1.18.2-26.el8_9.x86_64.rp 4.4 MB/s | 562 kB 00:00 2024-03-29T09:05:41.232 INFO:teuthology.orchestra.run.smithi133.stdout:(128/146): openssl-devel-1.1.1k-12.el8_9.x86_64 12 MB/s | 2.3 MB 00:00 2024-03-29T09:05:41.257 INFO:teuthology.orchestra.run.smithi133.stdout:(129/146): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 256 kB/s | 47 kB 00:00 2024-03-29T09:05:41.299 INFO:teuthology.orchestra.run.smithi133.stdout:(130/146): python3-cryptography-3.2.1-7.el8_9.x 3.4 MB/s | 559 kB 00:00 2024-03-29T09:05:41.457 INFO:teuthology.orchestra.run.smithi133.stdout:(131/146): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2024-03-29T09:05:41.574 INFO:teuthology.orchestra.run.smithi133.stdout:(132/146): elfutils-libs-0.189-3.el8.x86_64.rpm 2.5 MB/s | 303 kB 00:00 2024-03-29T09:05:41.733 INFO:teuthology.orchestra.run.smithi133.stdout:(133/146): librados2-18.2.2-1015.gd0e392d9.el8. 7.3 MB/s | 3.7 MB 00:00 2024-03-29T09:05:41.841 INFO:teuthology.orchestra.run.smithi133.stdout:(134/146): librbd1-18.2.2-1015.gd0e392d9.el8.x8 6.1 MB/s | 3.6 MB 00:00 2024-03-29T09:05:41.867 INFO:teuthology.orchestra.run.smithi133.stdout:(135/146): libmount-2.32.1-43.el8.x86_64.rpm 809 kB/s | 237 kB 00:00 2024-03-29T09:05:41.917 INFO:teuthology.orchestra.run.smithi133.stdout:(136/146): libsmartcols-2.32.1-43.el8.x86_64.rp 976 kB/s | 179 kB 00:00 2024-03-29T09:05:41.950 INFO:teuthology.orchestra.run.smithi133.stdout:(137/146): libuuid-2.32.1-43.el8.x86_64.rpm 909 kB/s | 99 kB 00:00 2024-03-29T09:05:42.039 INFO:teuthology.orchestra.run.smithi133.stdout:(138/146): util-linux-user-2.32.1-43.el8.x86_64 879 kB/s | 103 kB 00:00 2024-03-29T09:05:42.085 INFO:teuthology.orchestra.run.smithi133.stdout:(139/146): util-linux-2.32.1-43.el8.x86_64.rpm 11 MB/s | 2.5 MB 00:00 2024-03-29T09:05:42.127 INFO:teuthology.orchestra.run.smithi133.stdout:(140/146): iproute-6.2.0-5.el8_9.x86_64.rpm 4.7 MB/s | 854 kB 00:00 2024-03-29T09:05:42.194 INFO:teuthology.orchestra.run.smithi133.stdout:(141/146): libblkid-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2024-03-29T09:05:42.211 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-03-29T09:05:42.219 INFO:teuthology.orchestra.run.smithi133.stdout:(142/146): libfdisk-2.32.1-43.el8.x86_64.rpm 1.9 MB/s | 254 kB 00:00 2024-03-29T09:05:42.294 INFO:teuthology.orchestra.run.smithi133.stdout:(143/146): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.2 MB/s | 711 kB 00:00 2024-03-29T09:05:42.353 INFO:teuthology.orchestra.run.smithi133.stdout:(144/146): openssl-libs-1.1.1k-12.el8_9.x86_64. 9.3 MB/s | 1.5 MB 00:00 2024-03-29T09:05:42.386 INFO:teuthology.orchestra.run.smithi133.stdout:(145/146): xfsdump-3.1.8-7.el8_9.x86_64.rpm 1.9 MB/s | 333 kB 00:00 2024-03-29T09:05:42.470 INFO:teuthology.orchestra.run.smithi133.stdout:(146/146): xfsprogs-5.0.0-12.el8.x86_64.rpm 6.1 MB/s | 1.1 MB 00:00 2024-03-29T09:05:42.473 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-03-29T09:05:42.473 INFO:teuthology.orchestra.run.smithi133.stdout:Total 13 MB/s | 213 MB 00:16 2024-03-29T09:05:42.661 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-03-29T09:05:42.662 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-03-29T09:05:44.370 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-03-29T09:05:44.382 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-03-29T09:05:44.628 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-03-29T09:05:45.083 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-03-29T09:05:45.083 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-03-29T09:05:46.817 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-03-29T09:05:46.828 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-03-29T09:05:50.447 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-03-29T09:05:50.631 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-29T09:05:50.907 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-29T09:05:51.174 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-29T09:05:51.383 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-29T09:05:51.531 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-29T09:05:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-29T09:05:51.872 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-29T09:05:51.965 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-29T09:05:52.162 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-29T09:05:52.397 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 6/162 2024-03-29T09:05:52.614 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-29T09:05:52.754 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/162 2024-03-29T09:05:52.806 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-03-29T09:05:52.997 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-29T09:05:53.007 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-03-29T09:05:53.251 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-29T09:05:53.445 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-29T09:05:53.520 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/162 2024-03-29T09:05:53.631 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 9/162 2024-03-29T09:05:53.686 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/162 2024-03-29T09:05:53.855 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-29T09:05:53.950 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/162 2024-03-29T09:05:54.068 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 9/162 2024-03-29T09:05:54.105 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-29T09:05:54.195 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/162 2024-03-29T09:05:54.405 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/162 2024-03-29T09:05:54.422 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 10/162 2024-03-29T09:05:54.580 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-29T09:05:54.590 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 10/162 2024-03-29T09:05:54.684 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/162 2024-03-29T09:05:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 11/162 2024-03-29T09:05:54.892 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 11/162 2024-03-29T09:05:54.926 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/162 2024-03-29T09:05:55.146 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-29T09:05:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.2.2-1015.gd0e392d9.el8.x86_64 12/162 2024-03-29T09:05:55.488 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-29T09:05:55.603 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-29T09:05:55.606 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/162 2024-03-29T09:05:55.763 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 9/162 2024-03-29T09:05:55.831 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-29T09:05:56.087 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-29T09:05:56.225 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 9/162 2024-03-29T09:05:56.294 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-29T09:05:56.421 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 10/162 2024-03-29T09:05:56.570 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 10/162 2024-03-29T09:05:56.709 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-29T09:05:56.757 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 11/162 2024-03-29T09:05:56.872 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 11/162 2024-03-29T09:05:56.956 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-29T09:05:57.097 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 19/162 2024-03-29T09:05:57.206 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:18.2.2-1015.gd0e392d9.el8.x86_64 12/162 2024-03-29T09:05:57.208 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 19/162 2024-03-29T09:05:57.363 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86 20/162 2024-03-29T09:05:57.462 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-29T09:05:57.472 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86 20/162 2024-03-29T09:05:57.567 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/162 2024-03-29T09:05:57.628 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/162 2024-03-29T09:05:57.736 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpq-13.11-1.el8.x86_64 14/162 2024-03-29T09:05:57.962 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/162 2024-03-29T09:05:58.003 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/162 2024-03-29T09:05:58.184 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/162 2024-03-29T09:05:58.186 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/162 2024-03-29T09:05:58.421 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-29T09:05:58.569 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/162 2024-03-29T09:05:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/162 2024-03-29T09:05:58.815 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-29T09:05:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 18/162 2024-03-29T09:05:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 19/162 2024-03-29T09:05:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 19/162 2024-03-29T09:05:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86 20/162 2024-03-29T09:05:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86 20/162 2024-03-29T09:05:59.501 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/162 2024-03-29T09:05:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-29T09:05:59.713 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-29T09:05:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/162 2024-03-29T09:05:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-29T09:06:00.381 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/162 2024-03-29T09:06:00.604 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/162 2024-03-29T09:06:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/162 2024-03-29T09:06:00.795 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-29T09:06:01.403 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/162 2024-03-29T09:06:01.459 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/162 2024-03-29T09:06:01.637 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/162 2024-03-29T09:06:01.720 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/162 2024-03-29T09:06:01.986 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/162 2024-03-29T09:06:02.231 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-29T09:06:02.457 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/162 2024-03-29T09:06:02.469 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.2.2-1015.gd0e392d9.el8. 31/162 2024-03-29T09:06:02.617 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.2.2-1015.gd0e392d9.el 32/162 2024-03-29T09:06:02.744 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/162 2024-03-29T09:06:02.895 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/162 2024-03-29T09:06:02.946 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.2.2-1015.gd0e392d9.el8.x86_6 33/162 2024-03-29T09:06:03.087 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/162 2024-03-29T09:06:03.161 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-29T09:06:03.379 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/162 2024-03-29T09:06:03.711 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/162 2024-03-29T09:06:03.973 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/162 2024-03-29T09:06:04.152 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:18.2.2-1015.gd0e392d9.el8. 31/162 2024-03-29T09:06:04.258 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:18.2.2-1015.gd0e392d9.el 32/162 2024-03-29T09:06:04.534 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:18.2.2-1015.gd0e392d9.el8.x86_6 33/162 2024-03-29T09:06:05.193 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/162 2024-03-29T09:06:05.575 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/162 2024-03-29T09:06:05.590 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/162 2024-03-29T09:06:05.946 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/162 2024-03-29T09:06:06.202 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/162 2024-03-29T09:06:06.444 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-29T09:06:06.649 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/162 2024-03-29T09:06:07.036 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/162 2024-03-29T09:06:07.301 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/162 2024-03-29T09:06:07.522 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-29T09:06:07.749 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-29T09:06:07.969 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 41/162 2024-03-29T09:06:08.209 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.2.2-1015.gd0e392d9.el8.x86_6 42/162 2024-03-29T09:06:08.429 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-29T09:06:08.471 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/162 2024-03-29T09:06:08.534 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-29T09:06:08.777 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-29T09:06:08.851 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 38/162 2024-03-29T09:06:08.867 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-29T09:06:09.053 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-29T09:06:09.062 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/162 2024-03-29T09:06:09.232 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-29T09:06:09.275 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/162 2024-03-29T09:06:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-29T09:06:09.509 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 41/162 2024-03-29T09:06:09.687 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:18.2.2-1015.gd0e392d9.el8.x86_6 42/162 2024-03-29T09:06:09.926 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/162 2024-03-29T09:06:10.031 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/162 2024-03-29T09:06:10.087 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-29T09:06:10.210 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-29T09:06:10.323 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/162 2024-03-29T09:06:10.428 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-29T09:06:10.472 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-29T09:06:10.502 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/162 2024-03-29T09:06:10.652 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-29T09:06:10.706 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/162 2024-03-29T09:06:10.749 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 49/162 2024-03-29T09:06:10.918 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-29T09:06:11.069 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : jq-1.6-7.el8.x86_64 50/162 2024-03-29T09:06:11.348 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.1-8.el8.noarch 51/162 2024-03-29T09:06:11.492 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/162 2024-03-29T09:06:11.560 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-43.el8.x86_64 52/162 2024-03-29T09:06:11.651 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 52/162 2024-03-29T09:06:11.844 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/162 2024-03-29T09:06:11.844 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-29T09:06:11.876 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/162 2024-03-29T09:06:12.020 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/162 2024-03-29T09:06:12.057 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/162 2024-03-29T09:06:12.217 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/162 2024-03-29T09:06:12.236 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/162 2024-03-29T09:06:12.389 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-16.8-9.el8.noarch 56/162 2024-03-29T09:06:12.467 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/162 2024-03-29T09:06:12.639 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/162 2024-03-29T09:06:12.640 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.7-17.el8.x86_64 57/162 2024-03-29T09:06:12.856 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 58/162 2024-03-29T09:06:12.873 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/162 2024-03-29T09:06:13.110 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 59/162 2024-03-29T09:06:13.173 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/162 2024-03-29T09:06:13.306 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 60/162 2024-03-29T09:06:13.452 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/162 2024-03-29T09:06:13.591 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 61/162 2024-03-29T09:06:13.699 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/162 2024-03-29T09:06:13.861 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/162 2024-03-29T09:06:13.929 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 62/162 2024-03-29T09:06:14.060 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/162 2024-03-29T09:06:14.200 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/162 2024-03-29T09:06:14.465 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 64/162 2024-03-29T09:06:14.511 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/162 2024-03-29T09:06:14.720 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 65/162 2024-03-29T09:06:14.941 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 60/162 2024-03-29T09:06:14.994 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 66/162 2024-03-29T09:06:15.258 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/162 2024-03-29T09:06:15.448 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 67/162 2024-03-29T09:06:15.486 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 62/162 2024-03-29T09:06:15.708 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/162 2024-03-29T09:06:15.816 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 68/162 2024-03-29T09:06:15.974 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 64/162 2024-03-29T09:06:16.120 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/162 2024-03-29T09:06:16.261 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 65/162 2024-03-29T09:06:16.381 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 70/162 2024-03-29T09:06:16.487 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 66/162 2024-03-29T09:06:16.623 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 71/162 2024-03-29T09:06:16.703 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/162 2024-03-29T09:06:16.915 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 72/162 2024-03-29T09:06:16.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/162 2024-03-29T09:06:17.210 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 73/162 2024-03-29T09:06:17.282 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/162 2024-03-29T09:06:17.429 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 74/162 2024-03-29T09:06:17.434 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-29T09:06:17.541 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 70/162 2024-03-29T09:06:17.728 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 75/162 2024-03-29T09:06:17.862 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-7.el8.x86_64 71/162 2024-03-29T09:06:17.949 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 76/162 2024-03-29T09:06:18.107 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.1-8.el8.noarch 72/162 2024-03-29T09:06:18.228 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 77/162 2024-03-29T09:06:18.301 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-43.el8.x86_64 73/162 2024-03-29T09:06:18.418 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 73/162 2024-03-29T09:06:18.438 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-29T09:06:18.575 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-29T09:06:18.625 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-29T09:06:18.776 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/162 2024-03-29T09:06:18.933 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-29T09:06:18.964 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-29T09:06:19.166 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-29T09:06:19.219 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-16.8-9.el8.noarch 77/162 2024-03-29T09:06:19.364 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-29T09:06:19.385 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 78/162 2024-03-29T09:06:19.521 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-29T09:06:19.608 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 79/162 2024-03-29T09:06:19.797 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-29T09:06:19.882 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 80/162 2024-03-29T09:06:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-29T09:06:20.133 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 81/162 2024-03-29T09:06:20.280 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 82/162 2024-03-29T09:06:20.291 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-29T09:06:20.479 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 83/162 2024-03-29T09:06:20.487 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-29T09:06:20.682 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-29T09:06:20.713 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 84/162 2024-03-29T09:06:20.906 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-29T09:06:20.961 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 85/162 2024-03-29T09:06:21.130 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 86/162 2024-03-29T09:06:21.139 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-29T09:06:21.323 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-29T09:06:21.345 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 87/162 2024-03-29T09:06:21.498 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 88/162 2024-03-29T09:06:21.618 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-29T09:06:21.730 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 89/162 2024-03-29T09:06:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-29T09:06:21.922 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 90/162 2024-03-29T09:06:22.086 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-29T09:06:22.327 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-29T09:06:22.515 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 91/162 2024-03-29T09:06:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-2.6-1.el8.noarch 92/162 2024-03-29T09:06:23.076 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 93/162 2024-03-29T09:06:23.294 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 94/162 2024-03-29T09:06:23.517 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 95/162 2024-03-29T09:06:23.630 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-29T09:06:23.977 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-29T09:06:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-29T09:06:24.624 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-29T09:06:24.820 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 96/162 2024-03-29T09:06:25.152 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 97/162 2024-03-29T09:06:25.421 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 98/162 2024-03-29T09:06:25.662 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-29T09:06:25.838 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 99/162 2024-03-29T09:06:25.932 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-29T09:06:26.168 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 100/162 2024-03-29T09:06:26.353 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 101/162 2024-03-29T09:06:26.758 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-29T09:06:26.940 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 103/162 2024-03-29T09:06:27.038 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 103/162 2024-03-29T09:06:27.206 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 102/162 2024-03-29T09:06:27.274 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 104/162 2024-03-29T09:06:27.374 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 103/162 2024-03-29T09:06:27.470 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 103/162 2024-03-29T09:06:28.058 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 104/162 2024-03-29T09:06:32.100 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 105/162 2024-03-29T09:06:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 105/162 2024-03-29T09:06:32.857 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 105/162 2024-03-29T09:06:33.070 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 105/162 2024-03-29T09:06:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 106/162 2024-03-29T09:06:33.553 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 106/162 2024-03-29T09:06:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 105/162 2024-03-29T09:06:33.772 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 107/162 2024-03-29T09:06:34.458 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 105/162 2024-03-29T09:06:34.720 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 106/162 2024-03-29T09:06:34.866 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 106/162 2024-03-29T09:06:35.049 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 107/162 2024-03-29T09:06:54.789 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 107/162 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /run 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /dev 2024-03-29T09:06:54.790 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:06:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 108/162 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 107/162 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /run 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-03-29T09:06:56.109 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-03-29T09:06:56.110 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-03-29T09:06:56.110 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2024-03-29T09:06:56.110 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:06:56.469 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 108/162 2024-03-29T09:06:56.967 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 108/162 2024-03-29T09:06:57.242 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 109/162 2024-03-29T09:06:57.506 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 109/162 2024-03-29T09:06:57.683 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 110/162 2024-03-29T09:06:58.225 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 110/162 2024-03-29T09:06:58.231 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 108/162 2024-03-29T09:06:58.425 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 111/162 2024-03-29T09:06:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 109/162 2024-03-29T09:06:58.466 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 111/162 2024-03-29T09:06:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 109/162 2024-03-29T09:06:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 110/162 2024-03-29T09:06:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 110/162 2024-03-29T09:06:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 111/162 2024-03-29T09:06:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 111/162 2024-03-29T09:07:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 112/162 2024-03-29T09:07:01.455 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 112/162 2024-03-29T09:07:01.587 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1015.gd0e392d9.e 113/162 2024-03-29T09:07:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1015.gd0e392d9. 114/162 2024-03-29T09:07:02.040 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noa 115/162 2024-03-29T09:07:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noa 115/162 2024-03-29T09:07:02.402 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 112/162 2024-03-29T09:07:02.599 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.n 116/162 2024-03-29T09:07:02.618 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 112/162 2024-03-29T09:07:02.734 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:18.2.2-1015.gd0e392d9.e 113/162 2024-03-29T09:07:02.900 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:18.2.2-1015.gd0e392d9. 114/162 2024-03-29T09:07:03.118 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noa 115/162 2024-03-29T09:07:03.393 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noa 115/162 2024-03-29T09:07:03.639 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.n 116/162 2024-03-29T09:07:03.681 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.n 116/162 2024-03-29T09:07:03.865 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e 117/162 2024-03-29T09:07:03.994 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e 117/162 2024-03-29T09:07:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1015.gd0e392d9.el 118/162 2024-03-29T09:07:04.682 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 119/162 2024-03-29T09:07:04.720 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.n 116/162 2024-03-29T09:07:04.750 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 119/162 2024-03-29T09:07:04.859 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e 117/162 2024-03-29T09:07:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 120/162 2024-03-29T09:07:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 120/162 2024-03-29T09:07:04.992 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e 117/162 2024-03-29T09:07:05.377 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:18.2.2-1015.gd0e392d9.el 118/162 2024-03-29T09:07:05.539 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 119/162 2024-03-29T09:07:05.610 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 119/162 2024-03-29T09:07:05.800 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 120/162 2024-03-29T09:07:05.830 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 120/162 2024-03-29T09:07:06.365 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.2.2-1015.gd0e392d9.el8.x86_64 121/162 2024-03-29T09:07:06.565 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 122/162 2024-03-29T09:07:06.626 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 122/162 2024-03-29T09:07:06.789 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1015.gd0e39 123/162 2024-03-29T09:07:06.860 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1015.gd0e39 123/162 2024-03-29T09:07:07.039 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 124/162 2024-03-29T09:07:07.162 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:18.2.2-1015.gd0e392d9.el8.x86_64 121/162 2024-03-29T09:07:07.331 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 122/162 2024-03-29T09:07:07.393 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 122/162 2024-03-29T09:07:07.438 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 124/162 2024-03-29T09:07:07.551 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:18.2.2-1015.gd0e39 123/162 2024-03-29T09:07:07.622 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.2-1015.gd0e39 123/162 2024-03-29T09:07:07.673 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 125/162 2024-03-29T09:07:07.809 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 124/162 2024-03-29T09:07:08.210 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 124/162 2024-03-29T09:07:08.383 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 125/162 2024-03-29T09:07:12.490 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 125/162 2024-03-29T09:07:13.201 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 125/162 2024-03-29T09:07:13.718 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:18.2.2-1015.gd0e392d9.el8.x86_64 126/162 2024-03-29T09:07:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.2.2-1015.gd0e392d9.el8.x86_64 126/162 2024-03-29T09:07:14.082 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 127/162 2024-03-29T09:07:14.093 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 127/162 2024-03-29T09:07:14.282 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-25.el8.x86_64 128/162 2024-03-29T09:07:14.332 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 128/162 2024-03-29T09:07:14.753 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 129/162 2024-03-29T09:07:15.105 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 130/162 2024-03-29T09:07:15.302 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 128/162 2024-03-29T09:07:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-25.el8.x86_64 131/162 2024-03-29T09:07:15.367 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 131/162 2024-03-29T09:07:15.577 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-29T09:07:15.638 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 129/162 2024-03-29T09:07:15.818 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-29T09:07:15.931 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 130/162 2024-03-29T09:07:16.022 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.2.2-1015.gd0e392d9.el8.x86_ 134/162 2024-03-29T09:07:16.123 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 131/162 2024-03-29T09:07:16.354 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 132/162 2024-03-29T09:07:16.537 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 133/162 2024-03-29T09:07:16.742 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:18.2.2-1015.gd0e392d9.el8.x86_ 134/162 2024-03-29T09:07:19.379 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-29T09:07:19.712 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-29T09:07:19.799 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-29T09:07:20.177 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-29T09:07:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-29T09:07:20.516 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-29T09:07:20.598 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.23-4.module+el8.9.0+21317+d 135/162 2024-03-29T09:07:20.657 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-29T09:07:20.891 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-29T09:07:20.989 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 136/162 2024-03-29T09:07:21.121 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:18.2.2-1015.gd0e392d9.el8.noarch 139/162 2024-03-29T09:07:21.335 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 137/162 2024-03-29T09:07:21.346 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 140/162 2024-03-29T09:07:21.603 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 138/162 2024-03-29T09:07:21.695 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 141/162 2024-03-29T09:07:21.774 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:18.2.2-1015.gd0e392d9.el8.noarch 139/162 2024-03-29T09:07:22.024 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 140/162 2024-03-29T09:07:22.074 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-29T09:07:22.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.1-9.el8.x86_64 143/162 2024-03-29T09:07:22.361 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 141/162 2024-03-29T09:07:22.411 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 143/162 2024-03-29T09:07:22.619 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.0.4-10.el8.x86_64 144/162 2024-03-29T09:07:22.649 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 144/162 2024-03-29T09:07:22.683 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 142/162 2024-03-29T09:07:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-29T09:07:22.869 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-29T09:07:22.872 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 143/162 2024-03-29T09:07:22.894 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 145/162 2024-03-29T09:07:23.030 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 146/162 2024-03-29T09:07:23.066 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 144/162 2024-03-29T09:07:23.216 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-29T09:07:23.280 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.1-9.el8.x86_64 145/162 2024-03-29T09:07:23.390 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 145/162 2024-03-29T09:07:23.429 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-29T09:07:23.597 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.0.4-10.el8.x86_64 146/162 2024-03-29T09:07:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-29T09:07:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-29T09:07:23.627 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 146/162 2024-03-29T09:07:23.790 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 147/162 2024-03-29T09:07:23.925 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-29T09:07:23.925 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-29T09:07:23.984 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 148/162 2024-03-29T09:07:24.135 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 149/162 2024-03-29T09:07:24.135 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-29T09:07:24.227 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-29T09:07:24.228 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-29T09:07:24.419 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 150/162 2024-03-29T09:07:24.419 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-29T09:07:24.503 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-29T09:07:24.662 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 151/162 2024-03-29T09:07:24.662 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-29T09:07:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-29T09:07:24.930 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 152/162 2024-03-29T09:07:24.945 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-29T09:07:25.174 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 153/162 2024-03-29T09:07:25.204 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-29T09:07:25.204 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-29T09:07:25.347 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 154/162 2024-03-29T09:07:25.455 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-29T09:07:25.456 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-29T09:07:25.574 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 155/162 2024-03-29T09:07:25.574 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-29T09:07:25.772 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-29T09:07:25.772 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-29T09:07:25.775 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 156/162 2024-03-29T09:07:25.775 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 157/162 2024-03-29T09:07:26.042 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 157/162 2024-03-29T09:07:26.042 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-29T09:07:26.055 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-29T09:07:26.055 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-29T09:07:26.292 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 158/162 2024-03-29T09:07:26.292 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-29T09:07:26.321 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-29T09:07:26.322 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-29T09:07:26.508 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 159/162 2024-03-29T09:07:26.508 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-29T09:07:26.605 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-29T09:07:26.750 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 160/162 2024-03-29T09:07:26.763 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-29T09:07:26.763 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-29T09:07:26.899 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 161/162 2024-03-29T09:07:26.900 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-29T09:07:27.404 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-29T09:07:27.404 INFO:teuthology.orchestra.run.smithi099.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-29T09:07:27.404 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:27.484 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 162/162 2024-03-29T09:07:27.484 INFO:teuthology.orchestra.run.smithi133.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-29T09:07:27.484 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.2.2-1015.gd0e392d9.el8.x86_64 1/162 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 2/162 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 3/162 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 4/162 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1015.gd0e39 5/162 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 6/162 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 7/162 2024-03-29T09:07:28.074 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 8/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 9/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 10/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 11/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.2.2-1015.gd0e392d9.el8.x86_64 12/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 13/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.2.2-1015.gd0e392d9.el8.x86_ 14/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 15/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 16/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.2.2-1015.gd0e392d9.el8.x86_6 17/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86 18/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 19/162 2024-03-29T09:07:28.075 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1015.gd0e392d9.el 20/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.2.2-1015.gd0e392d9.el8. 21/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.2.2-1015.gd0e392d9.el8.x86_6 22/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.2.2-1015.gd0e392d9.el8.x86_64 23/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 24/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 25/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 26/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 27/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 28/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1015.gd0e392d9. 29/162 2024-03-29T09:07:28.076 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noa 30/162 2024-03-29T09:07:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.n 31/162 2024-03-29T09:07:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e 32/162 2024-03-29T09:07:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1015.gd0e392d9.el 33/162 2024-03-29T09:07:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 34/162 2024-03-29T09:07:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1015.gd0e392d9.e 35/162 2024-03-29T09:07:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 36/162 2024-03-29T09:07:28.089 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 37/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:18.2.2-1015.gd0e392d9.el8.noarch 38/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-29T09:07:28.090 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-29T09:07:28.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/162 2024-03-29T09:07:28.092 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/162 2024-03-29T09:07:28.093 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 92/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 93/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 94/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 95/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 96/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 97/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 98/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 99/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 100/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 101/162 2024-03-29T09:07:28.094 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 103/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 104/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 105/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.0.4-10.el8.x86_64 106/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.1-9.el8.x86_64 107/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 109/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 110/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/162 2024-03-29T09:07:28.095 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : jq-1.6-7.el8.x86_64 124/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 126/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 127/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 128/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 129/162 2024-03-29T09:07:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 130/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 131/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 133/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-29T09:07:28.097 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-29T09:07:28.098 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-29T09:07:28.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-29T09:07:28.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-29T09:07:28.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-29T09:07:28.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-29T09:07:28.099 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-29T09:07:28.153 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:18.2.2-1015.gd0e392d9.el8.x86_64 1/162 2024-03-29T09:07:28.153 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 2/162 2024-03-29T09:07:28.153 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 3/162 2024-03-29T09:07:28.153 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 4/162 2024-03-29T09:07:28.153 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:18.2.2-1015.gd0e39 5/162 2024-03-29T09:07:28.153 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 6/162 2024-03-29T09:07:28.153 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 7/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 8/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 9/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 10/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 11/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:18.2.2-1015.gd0e392d9.el8.x86_64 12/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 13/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:18.2.2-1015.gd0e392d9.el8.x86_ 14/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 15/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 16/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:18.2.2-1015.gd0e392d9.el8.x86_6 17/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86 18/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 19/162 2024-03-29T09:07:28.154 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:18.2.2-1015.gd0e392d9.el 20/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:18.2.2-1015.gd0e392d9.el8. 21/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:18.2.2-1015.gd0e392d9.el8.x86_6 22/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:18.2.2-1015.gd0e392d9.el8.x86_64 23/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 24/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 25/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 26/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 27/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 28/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:18.2.2-1015.gd0e392d9. 29/162 2024-03-29T09:07:28.155 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noa 30/162 2024-03-29T09:07:28.156 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.n 31/162 2024-03-29T09:07:28.168 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e 32/162 2024-03-29T09:07:28.168 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:18.2.2-1015.gd0e392d9.el 33/162 2024-03-29T09:07:28.168 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 34/162 2024-03-29T09:07:28.168 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:18.2.2-1015.gd0e392d9.e 35/162 2024-03-29T09:07:28.168 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 36/162 2024-03-29T09:07:28.168 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 37/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:18.2.2-1015.gd0e392d9.el8.noarch 38/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/162 2024-03-29T09:07:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/162 2024-03-29T09:07:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-16.8-9.el8.noarch 72/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 73/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 74/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 75/162 2024-03-29T09:07:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 76/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 77/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 78/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.1-9.el8.x86_64 80/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 81/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 82/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 83/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 87/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 89/162 2024-03-29T09:07:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 92/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 93/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 94/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.1-8.el8.noarch 95/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpq-13.11-1.el8.x86_64 96/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-7.el8.x86_64 97/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 98/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 99/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 100/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a 101/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-10.23-4.module+el8.9.0+21317+ 102/162 2024-03-29T09:07:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-10.23-4.module+el8.9.0+21317+d 103/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 104/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 106/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 107/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 108/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 109/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 110/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 111/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 113/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 114/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 115/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 116/162 2024-03-29T09:07:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 117/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 118/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 124/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 125/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 126/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 127/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 128/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 129/162 2024-03-29T09:07:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 130/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 131/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 132/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 133/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 134/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 135/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 136/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 137/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 138/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 139/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 140/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 141/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 142/162 2024-03-29T09:07:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 143/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 144/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 145/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 146/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 147/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 148/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 149/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 150/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 151/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 152/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 153/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 154/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 155/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 156/162 2024-03-29T09:07:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 157/162 2024-03-29T09:07:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 158/162 2024-03-29T09:07:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 159/162 2024-03-29T09:07:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 160/162 2024-03-29T09:07:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 161/162 2024-03-29T09:07:43.751 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-29T09:07:43.751 INFO:teuthology.orchestra.run.smithi099.stdout:Installed products updated. 2024-03-29T09:07:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:44.401 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-03-29T09:07:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-29T09:07:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-29T09:07:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-29T09:07:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.401 INFO:teuthology.orchestra.run.smithi099.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-03-29T09:07:44.402 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.1-8.el8.noarch 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.0.4-10.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.403 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.404 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.1-9.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: jq-1.6-7.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-29T09:07:44.405 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libpq-13.11-1.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-29T09:07:44.406 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-29T09:07:44.407 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-29T09:07:44.408 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-29T09:07:44.409 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-2.6-1.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-29T09:07:44.410 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-29T09:07:44.411 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:44.412 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-03-29T09:07:44.757 DEBUG:teuthology.parallel:result is None 2024-03-29T09:07:47.776 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 162/162 2024-03-29T09:07:47.777 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2024-03-29T09:07:48.800 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:48.800 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-03-29T09:07:48.800 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-03-29T09:07:48.800 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-03-29T09:07:48.800 INFO:teuthology.orchestra.run.smithi133.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-03-29T09:07:48.800 INFO:teuthology.orchestra.run.smithi133.stdout: libblkid-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.800 INFO:teuthology.orchestra.run.smithi133.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: libmount-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-03-29T09:07:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.1-8.el8.noarch 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.0.4-10.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.802 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:18.2.2-1015.gd0e392d9.el8.noarch 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-03-29T09:07:48.803 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.1-9.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.7-17.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.6-7.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.2-3.el8.x86_64 2024-03-29T09:07:48.804 INFO:teuthology.orchestra.run.smithi133.stdout: libpq-13.11-1.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-25.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.3.1-3.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-03-29T09:07:48.805 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.15-6.el8.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-10.23-4.module+el8.9.0+21317+d2c6bf8a.x86_64 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-03-29T09:07:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.5.1-7.el8.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-6.2-6.el8.noarch 2024-03-29T09:07:48.807 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.0.6-14.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-03-29T09:07:48.808 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-16.8-9.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-2.6-1.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.14-14.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2017.2-11.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-03-29T09:07:48.809 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.4.1-12.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-03-29T09:07:48.810 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:18.2.2-1015.gd0e392d9.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-1.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.13.0-2.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-43.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:48.811 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-03-29T09:07:49.158 DEBUG:teuthology.parallel:result is None 2024-03-29T09:07:49.159 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T09:07:49.377 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T09:07:49.436 INFO:teuthology.orchestra.run.smithi099.stdout:18.2.2-1015.gd0e392d9.el8 2024-03-29T09:07:49.436 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1015.gd0e392d9.el8 2024-03-29T09:07:49.436 INFO:teuthology.task.install:The correct ceph version 18.2.2-1015.gd0e392d9 is installed. 2024-03-29T09:07:49.438 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T09:07:49.573 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-29T09:07:49.626 INFO:teuthology.orchestra.run.smithi133.stdout:18.2.2-1015.gd0e392d9.el8 2024-03-29T09:07:49.626 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1015.gd0e392d9.el8 2024-03-29T09:07:49.626 INFO:teuthology.task.install:The correct ceph version 18.2.2-1015.gd0e392d9 is installed. 2024-03-29T09:07:49.628 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T09:07:49.628 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:49.629 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T09:07:49.725 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:49.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T09:07:49.812 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T09:07:49.812 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:49.812 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T09:07:49.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T09:07:49.994 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:49.994 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T09:07:50.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T09:07:50.166 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T09:07:50.166 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:50.167 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T09:07:50.239 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T09:07:50.344 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:50.344 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T09:07:50.416 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T09:07:50.524 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T09:07:50.524 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:50.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T09:07:50.589 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T09:07:50.696 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:50.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T09:07:50.764 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T09:07:50.873 INFO:teuthology.run_tasks:Running task ceph... 2024-03-29T09:07:51.248 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-29T09:07:51.248 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2024-03-29T09:07:51.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-03-29T09:07:51.320 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-29T09:07:51.320 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T09:07:51.325 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T09:07:51.389 INFO:tasks.ceph:Creating extra log directories... 2024-03-29T09:07:51.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T09:07:51.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T09:07:51.478 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-29T09:07:51.478 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-29T09:07:51.478 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-29T09:07:51.478 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:51.478 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T09:07:51.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T09:07:51.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T09:07:51.945 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:51.946 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T09:07:51.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T09:07:52.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-29T09:07:52.260 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-29T09:07:52.260 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-29T09:07:52.261 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169', 'branch': 'wip-vshankar-testing-20240326.105515-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7628169', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240326.105515-reef'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240326.105515-reef', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7327, 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'sleep_before_teardown': 0, 'subset': '990/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240326.105515-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHjuiw4ET3PmkH1wWnugZbpBQ11jBn6EVX0hktynHK1DwKhuOOn59FfDYFbktPv+hJf2YQy+VF9dgCgM4ZmInxo=', 'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAdC+qenmuQZ2Ai2w2X9Q44TgT72N3dd0UgBMDYJpKaq1BCq99ERVhZj4/lvVfPK/WygeKTtsZP/EXl4Fppfs0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-28_11:21:59', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-29T09:07:52.261 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T09:07:52.265 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T09:07:52.305 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T09:07:52.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T09:07:52.411 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:52.412 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T09:07:52.453 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T09:07:52.453 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-03-29T09:07:52.534 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T09:07:52.534 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:52.534 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-03-29T09:07:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-29 09:07:27.527076124 +0000 2024-03-29T09:07:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-29 09:03:47.571969425 +0000 2024-03-29T09:07:52.535 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-29 09:03:47.571969425 +0000 2024-03-29T09:07:52.535 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-29 09:03:47.544970039 +0000 2024-03-29T09:07:52.535 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T09:07:52.636 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-29T09:07:52.636 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-29T09:07:52.636 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000285268 s, 1.8 MB/s 2024-03-29T09:07:52.638 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T09:07:52.682 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-29 09:07:27.527076124 +0000 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-29 09:03:48.047958592 +0000 2024-03-29T09:07:52.764 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-29 09:03:48.047958592 +0000 2024-03-29T09:07:52.765 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-29 09:03:48.021959184 +0000 2024-03-29T09:07:52.765 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T09:07:52.865 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-29T09:07:52.865 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-29T09:07:52.866 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000292186 s, 1.8 MB/s 2024-03-29T09:07:52.868 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T09:07:52.911 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-03-29T09:07:52.992 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T09:07:52.992 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:52.993 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-03-29T09:07:52.993 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:52.993 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:52.993 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-29 09:07:27.527076124 +0000 2024-03-29T09:07:52.993 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-29 09:03:48.507948123 +0000 2024-03-29T09:07:52.993 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-29 09:03:48.507948123 +0000 2024-03-29T09:07:52.993 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-29 09:03:48.486948601 +0000 2024-03-29T09:07:52.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T09:07:53.094 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-29T09:07:53.094 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-29T09:07:53.094 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000270014 s, 1.9 MB/s 2024-03-29T09:07:53.096 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T09:07:53.139 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-03-29T09:07:53.220 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-03-29 09:07:27.528076101 +0000 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-03-29 09:03:48.978937404 +0000 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-03-29 09:03:48.978937404 +0000 2024-03-29T09:07:53.221 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-03-29 09:03:48.954937950 +0000 2024-03-29T09:07:53.221 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T09:07:53.322 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-03-29T09:07:53.322 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-03-29T09:07:53.322 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000296986 s, 1.7 MB/s 2024-03-29T09:07:53.325 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T09:07:53.367 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T09:07:53.367 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:53.367 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T09:07:53.413 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T09:07:53.414 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-03-29T09:07:53.499 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T09:07:53.499 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:53.499 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 448 Links: 1 2024-03-29T09:07:53.499 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:53.499 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:53.500 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-29 09:07:27.607293474 +0000 2024-03-29T09:07:53.500 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-29 09:03:47.561549413 +0000 2024-03-29T09:07:53.500 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-29 09:03:47.561549413 +0000 2024-03-29T09:07:53.500 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-29 09:03:47.535549878 +0000 2024-03-29T09:07:53.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T09:07:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-29T09:07:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-29T09:07:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000537501 s, 953 kB/s 2024-03-29T09:07:53.608 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T09:07:53.652 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 454 Links: 1 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-29 09:07:27.607293474 +0000 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-29 09:03:48.035540945 +0000 2024-03-29T09:07:53.734 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-29 09:03:48.035540945 +0000 2024-03-29T09:07:53.735 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-29 09:03:48.010541392 +0000 2024-03-29T09:07:53.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T09:07:53.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-29T09:07:53.835 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-29T09:07:53.835 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000288046 s, 1.8 MB/s 2024-03-29T09:07:53.838 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T09:07:53.880 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 460 Links: 1 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-29 09:07:27.607293474 +0000 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-29 09:03:48.518532317 +0000 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-29 09:03:48.518532317 +0000 2024-03-29T09:07:53.962 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-29 09:03:48.497532692 +0000 2024-03-29T09:07:53.963 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T09:07:54.064 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-29T09:07:54.064 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-29T09:07:54.064 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000283299 s, 1.8 MB/s 2024-03-29T09:07:54.067 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T09:07:54.109 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-03-29T09:07:54.191 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T09:07:54.191 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T09:07:54.191 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 466 Links: 1 2024-03-29T09:07:54.191 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T09:07:54.192 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-29T09:07:54.192 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-29 09:07:27.608293456 +0000 2024-03-29T09:07:54.192 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-29 09:03:48.991523867 +0000 2024-03-29T09:07:54.192 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-29 09:03:48.991523867 +0000 2024-03-29T09:07:54.192 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-29 09:03:48.967524296 +0000 2024-03-29T09:07:54.192 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T09:07:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-29T09:07:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-29T09:07:54.293 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000310694 s, 1.6 MB/s 2024-03-29T09:07:54.296 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T09:07:54.339 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T09:07:54.339 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T09:07:54.339 INFO:tasks.ceph:Generating config... 2024-03-29T09:07:54.341 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-29T09:07:54.342 INFO:tasks.ceph:[client] debug client = 20 2024-03-29T09:07:54.342 INFO:tasks.ceph:[client] debug ms = 1 2024-03-29T09:07:54.342 INFO:tasks.ceph:[client] pid file = 2024-03-29T09:07:54.342 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-29T09:07:54.342 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-29T09:07:54.342 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-29T09:07:54.342 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-29T09:07:54.343 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-29T09:07:54.344 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-29T09:07:54.344 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-29T09:07:54.344 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-29T09:07:54.344 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-29T09:07:54.344 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-29T09:07:54.344 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-29T09:07:54.345 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-29T09:07:54.345 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-29T09:07:54.345 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-29T09:07:54.345 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-29T09:07:54.345 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-29T09:07:54.345 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-29T09:07:54.346 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-29T09:07:54.346 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-29T09:07:54.346 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-29T09:07:54.346 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-29T09:07:54.346 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-29T09:07:54.347 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-29T09:07:54.347 INFO:tasks.ceph:Setting up mon.a... 2024-03-29T09:07:54.347 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-29T09:07:54.454 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.keyring 2024-03-29T09:07:54.461 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-29T09:07:54.556 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T09:07:54.620 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.99'), ('mon.b', '172.21.15.133'), ('mon.c', '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]')] 2024-03-29T09:07:54.620 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.99,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-29T09:07:54.621 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:54.621 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T09:07:54.662 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.99 --add b 172.21.15.133 --addv c '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:07:54.784 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: generated fsid 83e3f490-14b9-4a33-8e2d-bbf792b299cc 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:setting min_mon_release = pacific 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:epoch 0 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:fsid 83e3f490-14b9-4a33-8e2d-bbf792b299cc 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:last_changed 2024-03-29T09:07:54.785944+0000 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:created 2024-03-29T09:07:54.785944+0000 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:min_mon_release 16 (pacific) 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:election_strategy: 1 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:0: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.a 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:2: [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] mon.c 2024-03-29T09:07:54.785 INFO:teuthology.orchestra.run.smithi099.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-29T09:07:54.787 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T09:07:54.828 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 83e3f490-14b9-4a33-8e2d-bbf792b299cc... 2024-03-29T09:07:54.828 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T09:07:54.872 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T09:07:54.979 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2024-03-29T09:07:54.980 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.981 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2024-03-29T09:07:54.982 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.983 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2024-03-29T09:07:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2024-03-29T09:07:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2024-03-29T09:07:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2024-03-29T09:07:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.99,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-03-29T09:07:54.984 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = 83e3f490-14b9-4a33-8e2d-bbf792b299cc 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2024-03-29T09:07:54.985 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.986 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 20 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 20 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 10 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2024-03-29T09:07:54.987 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-03-29T09:07:54.988 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.989 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: mon op complaint time = 120 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: client mount timeout = 600 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: debug client = 20 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-03-29T09:07:54.990 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = "" 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout:[mds] 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds = 20 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: debug mds balancer = 20 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug frag = True 2024-03-29T09:07:54.991 INFO:teuthology.orchestra.run.smithi099.stdout: mds debug scatterstat = True 2024-03-29T09:07:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: mds op complaint time = 180 2024-03-29T09:07:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: mds verify scatter = True 2024-03-29T09:07:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: osd op complaint time = 180 2024-03-29T09:07:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: rados mon op timeout = 900 2024-03-29T09:07:54.992 INFO:teuthology.orchestra.run.smithi099.stdout: rados osd op timeout = 900 2024-03-29T09:07:54.992 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-03-29T09:07:54.992 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-03-29T09:07:54.993 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-03-29T09:07:54.994 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.995 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.99,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 83e3f490-14b9-4a33-8e2d-bbf792b299cc 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.996 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-03-29T09:07:54.997 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 20 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 20 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 10 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T09:07:54.998 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:54.999 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T09:07:55.000 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-03-29T09:07:55.001 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-03-29T09:07:55.002 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-03-29T09:07:55.003 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-03-29T09:07:55.016 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-29T09:07:55.016 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-29T09:07:55.123 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-29T09:07:55.123 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:55.123 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-29T09:07:55.163 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:55.163 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-29T09:07:55.244 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:07:55.245 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:55.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T09:07:55.245 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T09:07:55.373 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:07:55.373 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:07:55.454 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:07:55.454 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:55.454 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T09:07:55.454 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T09:07:55.544 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:07:55.544 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:07:55.625 INFO:tasks.ceph:Setting up mon nodes... 2024-03-29T09:07:55.625 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-29T09:07:55.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T09:07:55.741 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T09:07:55.748 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T09:07:55.874 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T09:07:55.881 INFO:tasks.ceph:Setting up mds nodes... 2024-03-29T09:07:55.881 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T09:07:55.997 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T09:07:56.003 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-29T09:07:56.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T09:07:56.183 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T09:07:56.188 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-29T09:07:56.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T09:07:56.369 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T09:07:56.375 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-29T09:07:56.439 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T09:07:56.555 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T09:07:56.560 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-29T09:07:56.625 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-29T09:07:56.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T09:07:56.718 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-29T09:07:56.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-29T09:07:56.839 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-29T09:07:56.869 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T09:07:56.961 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-29T09:07:56.991 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-29T09:07:57.082 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-29T09:07:57.113 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-29T09:07:57.113 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T09:07:57.114 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-29T09:07:57.179 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T09:07:57.179 INFO:tasks.ceph:role: osd.0 2024-03-29T09:07:57.179 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:07:57.179 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T09:07:57.251 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:07:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:07:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:07:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:07:57.251 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:07:57.251 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:07:57.251 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:07:57.252 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:07:57.252 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:07:57.252 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:07:57.682 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-03-29T09:07:57.686 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-03-29T09:07:57.686 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-29T09:07:57.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-29T09:07:57.957 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-29T09:07:58.022 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T09:07:58.022 INFO:tasks.ceph:role: osd.1 2024-03-29T09:07:58.022 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:07:58.022 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T09:07:58.093 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:07:58.094 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:07:58.507 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-03-29T09:07:58.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-03-29T09:07:58.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-29T09:07:58.650 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-29T09:07:58.720 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-29T09:07:58.784 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T09:07:58.785 INFO:tasks.ceph:role: osd.2 2024-03-29T09:07:58.785 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:07:58.785 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T09:07:58.857 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:07:58.857 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:07:58.857 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:07:58.857 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:07:58.857 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:07:58.857 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:07:58.857 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:07:58.858 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:07:58.858 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:07:58.858 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:07:59.315 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-03-29T09:07:59.318 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-03-29T09:07:59.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-29T09:07:59.509 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-29T09:07:59.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-29T09:07:59.648 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T09:07:59.649 INFO:tasks.ceph:role: osd.3 2024-03-29T09:07:59.649 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:07:59.649 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T09:07:59.720 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:07:59.721 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:08:00.153 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-03-29T09:08:00.156 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-03-29T09:08:00.157 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-29T09:08:00.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-29T09:08:00.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:00.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:00.486+0000 7f2338db6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-29T09:08:00.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:00.487+0000 7f2338db6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-29T09:08:00.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:00.487+0000 7f2338db6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T09:08:00.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:00.487+0000 7f2338db6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T09:08:00.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:00.487+0000 7f2338db6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T09:08:00.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:00.489+0000 7f2338db6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-29T09:08:03.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T09:08:03.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:03.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:03.543+0000 7f0db1bd5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-29T09:08:03.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:03.543+0000 7f0db1bd5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-29T09:08:03.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:03.543+0000 7f0db1bd5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T09:08:03.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:03.543+0000 7f0db1bd5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T09:08:03.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:03.543+0000 7f0db1bd5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T09:08:03.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:03.545+0000 7f0db1bd5640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-29T09:08:06.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T09:08:06.492 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:06.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:06.595+0000 7fa197394640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-29T09:08:06.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:06.596+0000 7fa197394640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-29T09:08:06.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:06.596+0000 7fa197394640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T09:08:06.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:06.596+0000 7fa197394640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T09:08:06.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:06.596+0000 7fa197394640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T09:08:06.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:06.597+0000 7fa197394640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-29T09:08:09.481 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T09:08:09.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:09.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:09.652+0000 7f47f8074640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-29T09:08:09.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:09.653+0000 7f47f8074640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-29T09:08:09.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:09.653+0000 7f47f8074640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T09:08:09.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:09.653+0000 7f47f8074640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T09:08:09.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:09.653+0000 7f47f8074640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T09:08:09.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:09.655+0000 7f47f8074640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-29T09:08:12.531 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T09:08:12.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-29T09:08:12.669 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T09:08:12.669 INFO:tasks.ceph:role: osd.4 2024-03-29T09:08:12.669 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:08:12.670 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T09:08:12.742 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:08:12.742 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:08:12.742 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:08:12.742 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:08:12.742 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:08:12.742 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:08:12.743 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:08:12.743 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:08:12.743 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:08:12.743 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:08:13.186 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-29T09:08:13.192 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-29T09:08:13.192 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-29T09:08:13.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-29T09:08:13.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-29T09:08:13.548 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T09:08:13.548 INFO:tasks.ceph:role: osd.5 2024-03-29T09:08:13.548 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:08:13.548 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:08:13.620 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:08:14.030 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-29T09:08:14.034 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-29T09:08:14.034 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-29T09:08:14.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-29T09:08:14.241 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-29T09:08:14.306 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T09:08:14.306 INFO:tasks.ceph:role: osd.6 2024-03-29T09:08:14.306 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:08:14.307 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T09:08:14.379 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:08:14.379 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:08:14.379 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:08:14.379 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:08:14.379 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:08:14.380 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:08:14.380 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:08:14.380 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:08:14.380 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:08:14.380 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:08:14.824 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-29T09:08:14.828 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-29T09:08:14.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-29T09:08:15.015 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-29T09:08:15.086 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-29T09:08:15.151 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T09:08:15.151 INFO:tasks.ceph:role: osd.7 2024-03-29T09:08:15.151 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:08:15.151 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T09:08:15.222 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T09:08:15.222 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T09:08:15.222 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T09:08:15.222 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T09:08:15.223 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T09:08:15.223 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-29T09:08:15.223 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T09:08:15.223 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T09:08:15.223 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T09:08:15.223 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T09:08:15.650 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-29T09:08:15.654 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-29T09:08:15.654 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-29T09:08:15.815 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-29T09:08:15.886 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:15.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:15.999+0000 7f53d4c25640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-29T09:08:15.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:15.999+0000 7f53d4c25640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-29T09:08:15.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:15.999+0000 7f53d4c25640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T09:08:15.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:15.999+0000 7f53d4c25640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T09:08:15.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:15.999+0000 7f53d4c25640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T09:08:16.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:16.001+0000 7f53d4c25640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-29T09:08:18.886 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T09:08:18.953 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:19.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:19.053+0000 7f1fc07cd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-29T09:08:19.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:19.054+0000 7f1fc07cd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-29T09:08:19.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:19.054+0000 7f1fc07cd640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T09:08:19.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:19.054+0000 7f1fc07cd640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T09:08:19.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:19.054+0000 7f1fc07cd640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T09:08:19.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:19.056+0000 7f1fc07cd640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-29T09:08:21.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T09:08:22.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:22.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:22.104+0000 7fad191df640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-29T09:08:22.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:22.105+0000 7fad191df640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-29T09:08:22.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:22.105+0000 7fad191df640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T09:08:22.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:22.105+0000 7fad191df640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T09:08:22.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:22.105+0000 7fad191df640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T09:08:22.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:22.106+0000 7fad191df640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-29T09:08:22.262 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:08:22.267 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:08:24.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-29T09:08:25.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:25.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:25.158+0000 7ff61a33a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-29T09:08:25.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:25.158+0000 7ff61a33a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-29T09:08:25.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:25.158+0000 7ff61a33a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T09:08:25.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:25.158+0000 7ff61a33a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T09:08:25.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:25.158+0000 7ff61a33a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T09:08:25.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:25.160+0000 7ff61a33a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-29T09:08:28.042 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-29T09:08:28.106 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-29T09:08:28.107 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:28.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-29T09:08:28.178 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:28.178 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-29T09:08:28.285 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:28.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-29T09:08:28.388 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:28.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-29T09:08:28.493 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:28.493 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-29T09:08:28.596 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:28.596 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-29T09:08:28.700 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:28.700 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-29T09:08:28.804 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:28.805 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-29T09:08:28.870 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:28.871 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-29T09:08:28.975 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:28.975 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-29T09:08:29.079 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:29.079 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-29T09:08:29.183 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:29.183 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-29T09:08:29.287 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:29.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-29T09:08:29.391 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:29.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-29T09:08:29.495 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:29.495 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-29T09:08:29.538 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:29.538 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-29T09:08:29.619 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:29.619 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-29T09:08:29.660 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:29.660 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-29T09:08:29.743 INFO:tasks.ceph:Adding keys to all mons... 2024-03-29T09:08:29.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T09:08:29.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T09:08:29.804 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2024-03-29T09:08:29.804 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDrhAZmLEdRLBAAnVR8ae1uZ/ezeRT77pc7dw== 2024-03-29T09:08:29.804 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.a] 2024-03-29T09:08:29.804 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDrhAZmrFiLOxAAKlHfn8PZHThkmZP7F+CTSA== 2024-03-29T09:08:29.804 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.c] 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDshAZmfXkACxAAcCfih1bN4d00TqYlKIWjLg== 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDwhAZmLMALHRAARJg5Ykd30R69SyDTaCBIFw== 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDzhAZmuL5pIBAA6jXsRZ/S6bp2Clf6SAQamg== 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD2hAZmTOqEIxAAbl19sfN+WekZStEZ7F3WFQ== 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD5hAZmjSHvJhAAWVvxpRqaTu+An+sB6KW0QA== 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDrhAZm/LczNBAAakX4jUFzQMJdJNA14crh/A== 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.b] 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDshAZmQJsgFhAAXgodL3u1/qBUxO/ZydPMhQ== 2024-03-29T09:08:29.805 INFO:teuthology.orchestra.run.smithi099.stdout:[mds.d] 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDshAZmKnAzIRAA2psjQhAebm0ZQ+d833RbsA== 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQD/hAZm59SYOxAA6ZlyViyEh4T2LzLugojRlw== 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQADhQZmX0pCAxAAJ3tggpnioWiMCdGoDitpwg== 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQAGhQZmA85NBhAAeK3A7HRf5p03ImHioYPkxw== 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQAJhQZmt9t7CRAAikERd7yLpKlPADqOxYTe4Q== 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDshAZm8QvmKhAAtjcMwjNBLPSYOuxJAXZEnw== 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout:[client.2] 2024-03-29T09:08:29.806 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDshAZmvXccMhAALKLyc3Cfc3yjx3671P2yNw== 2024-03-29T09:08:29.807 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2024-03-29T09:08:29.807 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDshAZmKIVfORAAYXf8+I1h3ctIq9tQRkNJMQ== 2024-03-29T09:08:29.807 INFO:teuthology.orchestra.run.smithi099.stdout:[client.3] 2024-03-29T09:08:29.807 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQDthAZmlzsGBRAA3lYwuTrdEqSvk/f3Zx4NSA== 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.y] 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDrhAZmLEdRLBAAnVR8ae1uZ/ezeRT77pc7dw== 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDrhAZmrFiLOxAAKlHfn8PZHThkmZP7F+CTSA== 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDshAZmfXkACxAAcCfih1bN4d00TqYlKIWjLg== 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDwhAZmLMALHRAARJg5Ykd30R69SyDTaCBIFw== 2024-03-29T09:08:29.844 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDzhAZmuL5pIBAA6jXsRZ/S6bp2Clf6SAQamg== 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQD2hAZmTOqEIxAAbl19sfN+WekZStEZ7F3WFQ== 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQD5hAZmjSHvJhAAWVvxpRqaTu+An+sB6KW0QA== 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDrhAZm/LczNBAAakX4jUFzQMJdJNA14crh/A== 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDshAZmQJsgFhAAXgodL3u1/qBUxO/ZydPMhQ== 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDshAZmKnAzIRAA2psjQhAebm0ZQ+d833RbsA== 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.4] 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQD/hAZm59SYOxAA6ZlyViyEh4T2LzLugojRlw== 2024-03-29T09:08:29.845 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.5] 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQADhQZmX0pCAxAAJ3tggpnioWiMCdGoDitpwg== 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.6] 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAGhQZmA85NBhAAeK3A7HRf5p03ImHioYPkxw== 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.7] 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAJhQZmt9t7CRAAikERd7yLpKlPADqOxYTe4Q== 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDshAZm8QvmKhAAtjcMwjNBLPSYOuxJAXZEnw== 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout:[client.2] 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDshAZmvXccMhAALKLyc3Cfc3yjx3671P2yNw== 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDshAZmKIVfORAAYXf8+I1h3ctIq9tQRkNJMQ== 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout:[client.3] 2024-03-29T09:08:29.846 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDthAZmlzsGBRAA3lYwuTrdEqSvk/f3Zx4NSA== 2024-03-29T09:08:29.847 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T09:08:29.852 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T09:08:29.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:29.995 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:30.101 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:30.106 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:30.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.216 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.319 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.323 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.534 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.538 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T09:08:30.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T09:08:30.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:30.752 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:30.860 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:30.864 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T09:08:30.975 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:30.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:31.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:31.087 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:31.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:31.194 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:31.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:31.301 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T09:08:31.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.515 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.519 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.632 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T09:08:31.847 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-29T09:08:31.847 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-29T09:08:31.912 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T09:08:32.030 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-29T09:08:32.095 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-29T09:08:32.161 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T09:08:32.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-29T09:08:32.344 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-29T09:08:32.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T09:08:32.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-29T09:08:32.592 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:32.596 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T09:08:32.636 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-29T09:08:32.637 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-29T09:08:32.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-29T09:08:32.678 INFO:tasks.ceph.mon.a:Started 2024-03-29T09:08:32.678 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-29T09:08:32.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-29T09:08:32.682 INFO:tasks.ceph.mon.b:Started 2024-03-29T09:08:32.682 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-29T09:08:32.683 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-29T09:08:32.686 INFO:tasks.ceph.mon.c:Started 2024-03-29T09:08:32.686 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-29T09:08:32.687 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-29T09:08:32.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-29T09:08:32.691 INFO:tasks.ceph.mgr.y:Started 2024-03-29T09:08:32.691 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-29T09:08:32.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-29T09:08:32.694 INFO:tasks.ceph.mgr.x:Started 2024-03-29T09:08:32.695 INFO:tasks.ceph:Setting crush tunables to default 2024-03-29T09:08:32.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd crush tunables default 2024-03-29T09:08:32.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.885+0000 7f29f2000700 1 Processor -- start 2024-03-29T09:08:32.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.886+0000 7f29f2000700 1 -- start start 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f2000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f2000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f2000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec0d3e60 con 0x7f29ec0b3de0 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f2000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec0d0b40 con 0x7f29ec0d3960 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f2000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0ce7c0 con 0x7f29ec160790 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29ebe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f0c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f0c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:59376/0 (socket says 172.21.15.99:59376) 2024-03-29T09:08:32.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29ebe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:56102/0 (socket says 172.21.15.99:56102) 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f1600700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f29ec0b3de0 0x7f29ec160f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:44762/0 (socket says 172.21.15.99:44762) 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 learned_addr learned my addr 172.21.15.99:0/3591467591 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 msgr2=0x7f29ec0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 msgr2=0x7f29ec0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.887+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.888+0000 7f29ebe00700 1 -- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 msgr2=0x7f29ec0cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.888+0000 7f29ebe00700 1 -- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 msgr2=0x7f29ec0cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.888+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T09:08:32.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:32.888+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T09:08:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.088+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.088+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 msgr2=0x7f29ec0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T09:08:33.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.088+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 msgr2=0x7f29ec0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T09:08:33.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.088+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T09:08:33.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.088+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T09:08:33.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.089+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.089+0000 7f29ebe00700 1 -- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 msgr2=0x7f29ec0cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T09:08:33.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.089+0000 7f29ebe00700 1 -- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 msgr2=0x7f29ec0cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T09:08:33.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.089+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T09:08:33.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.089+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T09:08:33.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.097+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386246739 0 0) 0x7f29ec0d3e60 con 0x7f29ec0b3de0 2024-03-29T09:08:33.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.097+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d40035b0 con 0x7f29ec0b3de0 2024-03-29T09:08:33.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3540657108 0 0) 0x7f29d40035b0 con 0x7f29ec0b3de0 2024-03-29T09:08:33.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 msgr2=0x7f29ec0cf840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T09:08:33.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29ec0cf840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 msgr2=0x7f29ec0cfd80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T09:08:33.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 --2- 172.21.15.99:0/3591467591 >> v2:172.21.15.133:3300/0 conn(0x7f29ec0d3960 0x7f29ec0cfd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0ceaa0 con 0x7f29ec0b3de0 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 <== mon.1 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f29e40019f0 con 0x7f29ec0b3de0 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 >> v1:172.21.15.133:6789/0 conn(0x7f29ec0b3de0 legacy=0x7f29ec160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d4004d90 0x7f29d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0d0b40 con 0x7f29d4004d90 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec0ce7c0 con 0x7f29d4003a00 2024-03-29T09:08:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.104+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d400bba0 con 0x7f29ec160790 2024-03-29T09:08:33.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 msgr2=0x7f29d4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:33.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29d4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d4004d90 msgr2=0x7f29d4005210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T09:08:33.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f1600700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d4004d90 0x7f29d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d4004d90 0x7f29d4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f0c00700 1 -- 172.21.15.99:0/3591467591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40035b0 con 0x7f29d4003a00 2024-03-29T09:08:33.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29f1600700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29d4004d90 0x7f29d4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:08:33.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.105+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec160790 0x7f29d4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:33.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.106+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29d4003e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009940 tx=0x7f29d4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd922f235a643e87 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:33.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.109+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0019d90 con 0x7f29d4003a00 2024-03-29T09:08:33.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.109+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e001d070 con 0x7f29d4003a00 2024-03-29T09:08:33.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.109+0000 7f29eb400700 1 -- 172.21.15.99:0/3591467591 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0022670 con 0x7f29d4003a00 2024-03-29T09:08:33.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.110+0000 7f29f2000700 1 -- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 msgr2=0x7f29d4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:33.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.110+0000 7f29f2000700 1 --2- 172.21.15.99:0/3591467591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29d4003e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009940 tx=0x7f29d4005880 comp rx=0 tx=0).stop 2024-03-29T09:08:33.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.110+0000 7f29f1600700 1 -- 172.21.15.99:0/3591467591 reap_dead start 2024-03-29T09:08:33.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.110+0000 7f29f2000700 1 -- 172.21.15.99:0/3591467591 shutdown_connections 2024-03-29T09:08:33.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.110+0000 7f29f2000700 1 -- 172.21.15.99:0/3591467591 >> 172.21.15.99:0/3591467591 conn(0x7f29ec09c340 msgr2=0x7f29ec0d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:33.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.110+0000 7f29f2000700 1 -- 172.21.15.99:0/3591467591 shutdown_connections 2024-03-29T09:08:33.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.110+0000 7f29f2000700 1 -- 172.21.15.99:0/3591467591 wait complete. 2024-03-29T09:08:33.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.113+0000 7f29f2000700 1 Processor -- start 2024-03-29T09:08:33.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.113+0000 7f29f2000700 1 -- start start 2024-03-29T09:08:33.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.114+0000 7f29f2000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29ec14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:33.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.114+0000 7f29f2000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15ca00 0x7f29ec15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:33.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.114+0000 7f29f2000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec15d3c0 0x7f29ec153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:33.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.114+0000 7f29f2000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec152be0 con 0x7f29ec15ca00 2024-03-29T09:08:33.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.114+0000 7f29f2000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec15c0d0 con 0x7f29d4003a00 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.114+0000 7f29f2000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec15d850 con 0x7f29ec15d3c0 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15ca00 0x7f29ec15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15ca00 0x7f29ec15ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43500/0 (socket says 172.21.15.99:43500) 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 -- 172.21.15.99:0/3255166548 learned_addr learned my addr 172.21.15.99:0/3255166548 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29f1600700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec15d3c0 0x7f29ec153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29ec14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 -- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec15d3c0 msgr2=0x7f29ec153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec15d3c0 0x7f29ec153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 -- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 msgr2=0x7f29ec14dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29ec14dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 -- 172.21.15.99:0/3255166548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec0b5f40 con 0x7f29ec15ca00 2024-03-29T09:08:33.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29f0c00700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29ec14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29ebe00700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15ca00 0x7f29ec15ce80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f29d800e860 tx=0x7f29d800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29e8c00700 1 -- 172.21.15.99:0/3255166548 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80155f0 con 0x7f29ec15ca00 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29e8c00700 1 -- 172.21.15.99:0/3255166548 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d800c040 con 0x7f29ec15ca00 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec15bdf0 con 0x7f29ec15ca00 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.116+0000 7f29e8c00700 1 -- 172.21.15.99:0/3255166548 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8015970 con 0x7f29ec15ca00 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.117+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec152800 con 0x7f29ec15ca00 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.117+0000 7f29e8c00700 1 -- 172.21.15.99:0/3255166548 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f29d8005a20 con 0x7f29ec15ca00 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.117+0000 7f29e8c00700 1 -- 172.21.15.99:0/3255166548 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f29d8006940 con 0x7f29ec15ca00 2024-03-29T09:08:33.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.118+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec152380 con 0x7f29ec15ca00 2024-03-29T09:08:33.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.123+0000 7f29e8c00700 1 -- 172.21.15.99:0/3255166548 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f29d803eb50 con 0x7f29ec15ca00 2024-03-29T09:08:33.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.201+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f29ec152d20 con 0x7f29ec15ca00 2024-03-29T09:08:33.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.204+0000 7f29e8c00700 1 -- 172.21.15.99:0/3255166548 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f29ec152d20 con 0x7f29ec15ca00 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15ca00 msgr2=0x7f29ec15ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15ca00 0x7f29ec15ce80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f29d800e860 tx=0x7f29d800eb70 comp rx=0 tx=0).stop 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 shutdown_connections 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f29ec15d3c0 0x7f29ec153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29d4003a00 0x7f29ec14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 --2- 172.21.15.99:0/3255166548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f29ec15ca00 0x7f29ec15ce80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 >> 172.21.15.99:0/3255166548 conn(0x7f29ec09c340 msgr2=0x7f29ec0b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 shutdown_connections 2024-03-29T09:08:33.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:33.205+0000 7f29f2000700 1 -- 172.21.15.99:0/3255166548 wait complete. 2024-03-29T09:08:33.206 INFO:teuthology.orchestra.run.smithi099.stderr:adjusted tunables profile to default 2024-03-29T09:08:33.223 INFO:tasks.ceph:check_enable_crimson: False 2024-03-29T09:08:33.223 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-29T09:08:33.223 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:33.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-29T09:08:33.290 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:33.290 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-29T09:08:33.394 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:33.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-29T09:08:33.499 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:08:33.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-29T09:08:33.608 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:33.608 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-29T09:08:33.612 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:33.613+0000 7fd4e00ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T09:08:33.654 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:33.654+0000 7f7c7e72d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T09:08:33.672 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:33.673 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-29T09:08:33.719 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:33.718+0000 7fd4e00ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T09:08:33.751 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:33.751+0000 7f7c7e72d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T09:08:33.782 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:33.782 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-29T09:08:33.818 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:33.819+0000 7fd4e00ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T09:08:33.840 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:33.840+0000 7f7c7e72d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T09:08:33.890 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:08:33.890 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-29T09:08:33.922 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:33.922+0000 7fd4e00ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T09:08:33.928 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:33.928+0000 7f7c7e72d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T09:08:33.991 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new f0f3c53f-e223-47cf-9087-542a9d6e72c6 0 2024-03-29T09:08:34.137 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:34.137+0000 7f7c7e72d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T09:08:34.169 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:34.170+0000 7fd4e00ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T09:08:34.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.237+0000 7effcb200700 1 Processor -- start 2024-03-29T09:08:34.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.237+0000 7effcb200700 1 -- start start 2024-03-29T09:08:34.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.238+0000 7effcb200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 0x7effc40cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.238+0000 7effcb200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc406bf50 con 0x7effc415d780 2024-03-29T09:08:34.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.238+0000 7effcb200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc409a350 con 0x7effc4099a40 2024-03-29T09:08:34.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.238+0000 7effcb200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc406c1f0 con 0x7effc4099ec0 2024-03-29T09:08:34.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.238+0000 7effc3400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7effc4099a40 0x7effc40cb5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54218/0 (socket says 172.21.15.133:54218) 2024-03-29T09:08:34.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.238+0000 7effc3400700 1 -- 172.21.15.133:0/495838052 learned_addr learned my addr 172.21.15.133:0/495838052 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056596000 0 0) 0x7effc409a350 con 0x7effc4099a40 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa80035b0 con 0x7effc4099a40 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9559542 0 0) 0x7effc406bf50 con 0x7effc415d780 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc409a350 con 0x7effc415d780 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc9200700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 0x7effc40cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 238861431 0 0) 0x7effa80035b0 con 0x7effc4099a40 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 msgr2=0x7effc40cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 0x7effc40cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 >> v1:172.21.15.99:6789/0 conn(0x7effc415d780 legacy=0x7effc40cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc406c4d0 con 0x7effc4099a40 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 <== mon.1 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7effb00016b0 con 0x7effc4099a40 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 >> v1:172.21.15.133:6789/0 conn(0x7effc4099a40 legacy=0x7effc40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa8004bf0 0x7effa8004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 0x7effa80037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.239+0000 7effc2a00700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effa8003d30 0x7effa80041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb4001230 con 0x7effa8004bf0 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc406c1f0 con 0x7effa8003d30 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa800bac0 con 0x7effc4099ec0 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3400700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 0x7effa80037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3e00700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa8004bf0 0x7effa8004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc9200700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effa8003d30 0x7effa80041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3400700 1 -- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effa8003d30 msgr2=0x7effa80041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:34.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3400700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effa8003d30 0x7effa80041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3400700 1 -- 172.21.15.133:0/495838052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa8004bf0 msgr2=0x7effa8004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3400700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effa8004bf0 0x7effa8004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3400700 1 -- 172.21.15.133:0/495838052 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80035b0 con 0x7effc4099ec0 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.240+0000 7effc3400700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 0x7effa80037f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effb000c240 tx=0x7effa8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ea448460c9b016 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb000cd80 con 0x7effc4099ec0 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb0007b10 con 0x7effc4099ec0 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effc2a00700 1 -- 172.21.15.133:0/495838052 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb0005ba0 con 0x7effc4099ec0 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effcb200700 1 -- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 msgr2=0x7effa80037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effcb200700 1 --2- 172.21.15.133:0/495838052 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc4099ec0 0x7effa80037f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7effb000c240 tx=0x7effa8005880 comp rx=0 tx=0).stop 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effc9200700 1 -- 172.21.15.133:0/495838052 reap_dead start 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effcb200700 1 -- 172.21.15.133:0/495838052 shutdown_connections 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effcb200700 1 -- 172.21.15.133:0/495838052 >> 172.21.15.133:0/495838052 conn(0x7effc40b2b20 msgr2=0x7effc40b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effcb200700 1 -- 172.21.15.133:0/495838052 shutdown_connections 2024-03-29T09:08:34.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.241+0000 7effcb200700 1 -- 172.21.15.133:0/495838052 wait complete. 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.244+0000 7effcb200700 1 Processor -- start 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.244+0000 7effcb200700 1 -- start start 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effcb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc4099ec0 0x7effc414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effcb200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc40cda10 0x7effc40cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effcb200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effc40b59e0 0x7effc40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effcb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc40ce3d0 con 0x7effc4099ec0 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effcb200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc414e7f0 con 0x7effc40b59e0 2024-03-29T09:08:34.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effcb200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc40c5b40 con 0x7effc40cda10 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effc9200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effc40b59e0 0x7effc40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effc3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc40cda10 0x7effc40cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effc3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc4099ec0 0x7effc414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effc3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc4099ec0 0x7effc414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.133:56744/0 (socket says 172.21.15.133:56744) 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.245+0000 7effc3e00700 1 -- 172.21.15.133:0/1511694931 learned_addr learned my addr 172.21.15.133:0/1511694931 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc3e00700 1 -- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc40cda10 msgr2=0x7effc40cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc3e00700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc40cda10 0x7effc40cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc3e00700 1 -- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effc40b59e0 msgr2=0x7effc40b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc3e00700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effc40b59e0 0x7effc40b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc3e00700 1 -- 172.21.15.133:0/1511694931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc40c5de0 con 0x7effc4099ec0 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc9200700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effc40b59e0 0x7effc40b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc3400700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc40cda10 0x7effc40cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effc3e00700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc4099ec0 0x7effc414e2b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effb400fb30 tx=0x7effb400fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:34.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effbbe00700 1 -- 172.21.15.133:0/1511694931 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4001670 con 0x7effc4099ec0 2024-03-29T09:08:34.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effbbe00700 1 -- 172.21.15.133:0/1511694931 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb4001d30 con 0x7effc4099ec0 2024-03-29T09:08:34.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc40c60c0 con 0x7effc4099ec0 2024-03-29T09:08:34.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.246+0000 7effbbe00700 1 -- 172.21.15.133:0/1511694931 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4005520 con 0x7effc4099ec0 2024-03-29T09:08:34.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.247+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc40c6680 con 0x7effc4099ec0 2024-03-29T09:08:34.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.247+0000 7effbbe00700 1 -- 172.21.15.133:0/1511694931 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7effb4003980 con 0x7effc4099ec0 2024-03-29T09:08:34.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.247+0000 7effbbe00700 1 -- 172.21.15.133:0/1511694931 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7effb400de00 con 0x7effc4099ec0 2024-03-29T09:08:34.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.248+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc40678b0 con 0x7effc4099ec0 2024-03-29T09:08:34.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.252+0000 7effbbe00700 1 -- 172.21.15.133:0/1511694931 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7effb400d5c0 con 0x7effc4099ec0 2024-03-29T09:08:34.341 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:34.341+0000 7f7c7e72d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T09:08:34.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.360+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0f3c53f-e223-47cf-9087-542a9d6e72c6", "id": 0} v 0) v1 -- 0x7effc4072d00 con 0x7effc4099ec0 2024-03-29T09:08:34.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.367+0000 7effbbe00700 1 -- 172.21.15.133:0/1511694931 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0f3c53f-e223-47cf-9087-542a9d6e72c6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7effb400d760 con 0x7effc4099ec0 2024-03-29T09:08:34.367 INFO:teuthology.orchestra.run.smithi133.stdout:0 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc4099ec0 msgr2=0x7effc414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc4099ec0 0x7effc414e2b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7effb400fb30 tx=0x7effb400fe40 comp rx=0 tx=0).stop 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 shutdown_connections 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effc40b59e0 0x7effc40b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effc4099ec0 0x7effc414e2b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 --2- 172.21.15.133:0/1511694931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effc40cda10 0x7effc40cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 >> 172.21.15.133:0/1511694931 conn(0x7effc40b2b20 msgr2=0x7effc415b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 shutdown_connections 2024-03-29T09:08:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.369+0000 7effcb200700 1 -- 172.21.15.133:0/1511694931 wait complete. 2024-03-29T09:08:34.386 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 0e36e77e-2573-41e5-ae41-fe615e4076dd 1 2024-03-29T09:08:34.403 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:34.404+0000 7fd4e00ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T09:08:34.474 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:34.474+0000 7f7c7e72d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T09:08:34.548 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:34.548+0000 7fd4e00ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T09:08:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.577+0000 7fc326600700 1 Processor -- start 2024-03-29T09:08:34.576 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.577+0000 7fc326600700 1 -- start start 2024-03-29T09:08:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc326600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32015d2d0 0x7fc3200d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc326600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc3200d1060 0x7fc3200d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc326600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3200cdeb0 con 0x7fc3200d0680 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc326600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3200ce190 con 0x7fc3200d1060 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc326600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200ce470 con 0x7fc32015d2d0 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc325200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32015d2d0 0x7fc3200d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc31fe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc3200d1060 0x7fc3200d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc325200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32015d2d0 0x7fc3200d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:45826/0 (socket says 172.21.15.133:45826) 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc31fe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc3200d1060 0x7fc3200d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39280/0 (socket says 172.21.15.133:39280) 2024-03-29T09:08:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.578+0000 7fc325200700 1 -- 172.21.15.133:0/888236538 learned_addr learned my addr 172.21.15.133:0/888236538 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31fe00700 1 -- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32015d2d0 msgr2=0x7fc3200d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31fe00700 1 --2- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32015d2d0 0x7fc3200d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31fe00700 1 -- 172.21.15.133:0/888236538 >> v1:172.21.15.99:6789/0 conn(0x7fc3200d0680 legacy=0x7fc320160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31fe00700 1 -- 172.21.15.133:0/888236538 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200ce750 con 0x7fc3200d1060 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc325200700 1 --2- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32015d2d0 0x7fc3200d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31fe00700 1 --2- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d1060 0x7fc3200d1510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009840 tx=0x7fc30c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdba10b40e58017b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31f400700 1 -- 172.21.15.133:0/888236538 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c011830 con 0x7fc3200d1060 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31f400700 1 -- 172.21.15.133:0/888236538 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c011990 con 0x7fc3200d1060 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.579+0000 7fc31f400700 1 -- 172.21.15.133:0/888236538 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c011d20 con 0x7fc3200d1060 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 -- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d1060 msgr2=0x7fc3200d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 --2- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d1060 0x7fc3200d1510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009840 tx=0x7fc30c009b50 comp rx=0 tx=0).stop 2024-03-29T09:08:34.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 -- 172.21.15.133:0/888236538 shutdown_connections 2024-03-29T09:08:34.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 --2- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32015d2d0 0x7fc3200d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 --2- 172.21.15.133:0/888236538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d1060 0x7fc3200d1510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 -- 172.21.15.133:0/888236538 >> 172.21.15.133:0/888236538 conn(0x7fc32009c320 msgr2=0x7fc3200d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:34.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 -- 172.21.15.133:0/888236538 shutdown_connections 2024-03-29T09:08:34.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.580+0000 7fc326600700 1 -- 172.21.15.133:0/888236538 wait complete. 2024-03-29T09:08:34.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.583+0000 7fc326600700 1 Processor -- start 2024-03-29T09:08:34.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.583+0000 7fc326600700 1 -- start start 2024-03-29T09:08:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.584+0000 7fc326600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d0680 0x7fc3200c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.584+0000 7fc326600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3200d1060 0x7fc32006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.584+0000 7fc326600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc32015d2d0 0x7fc32014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.584+0000 7fc326600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200c9190 con 0x7fc32015d2d0 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.584+0000 7fc326600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200c8510 con 0x7fc3200d0680 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.584+0000 7fc326600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200c87f0 con 0x7fc3200d1060 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d0680 0x7fc3200c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc31fe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3200d1060 0x7fc32006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc31fe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3200d1060 0x7fc32006c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:45830/0 (socket says 172.21.15.133:45830) 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d0680 0x7fc3200c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39294/0 (socket says 172.21.15.133:39294) 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc31fe00700 1 -- 172.21.15.133:0/3279853302 learned_addr learned my addr 172.21.15.133:0/3279853302 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325c00700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc32015d2d0 0x7fc32014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 -- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3200d1060 msgr2=0x7fc32006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3200d1060 0x7fc32006c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 -- 172.21.15.133:0/3279853302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc32015d2d0 msgr2=0x7fc32014bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc32015d2d0 0x7fc32014bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 -- 172.21.15.133:0/3279853302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc314009550 con 0x7fc3200d0680 2024-03-29T09:08:34.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc31fe00700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3200d1060 0x7fc32006c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.585+0000 7fc325200700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d0680 0x7fc3200c7c50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e980 tx=0x7fc31400ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.586+0000 7fc31cc00700 1 -- 172.21.15.133:0/3279853302 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314004030 con 0x7fc3200d0680 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.586+0000 7fc31cc00700 1 -- 172.21.15.133:0/3279853302 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc314004190 con 0x7fc3200d0680 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.586+0000 7fc31cc00700 1 -- 172.21.15.133:0/3279853302 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314004520 con 0x7fc3200d0680 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.586+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c0094a0 con 0x7fc3200d0680 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.586+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3200c8db0 con 0x7fc3200d0680 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.586+0000 7fc31cc00700 1 -- 172.21.15.133:0/3279853302 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc314005d20 con 0x7fc3200d0680 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.586+0000 7fc31cc00700 1 -- 172.21.15.133:0/3279853302 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc314013ba0 con 0x7fc3200d0680 2024-03-29T09:08:34.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.587+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3200c9400 con 0x7fc3200d0680 2024-03-29T09:08:34.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.590+0000 7fc31cc00700 1 -- 172.21.15.133:0/3279853302 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fc314020060 con 0x7fc3200d0680 2024-03-29T09:08:34.590 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:34.590+0000 7f7c7e72d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T09:08:34.674 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:34.674+0000 7fd4e00ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T09:08:34.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.676+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0e36e77e-2573-41e5-ae41-fe615e4076dd", "id": 1} v 0) v1 -- 0x7fc320067730 con 0x7fc3200d0680 2024-03-29T09:08:34.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.684+0000 7fc31cc00700 1 -- 172.21.15.133:0/3279853302 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e36e77e-2573-41e5-ae41-fe615e4076dd", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc314013e30 con 0x7fc3200d0680 2024-03-29T09:08:34.683 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-03-29T09:08:34.684 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:34.683+0000 7f7c7e72d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d0680 msgr2=0x7fc3200c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d0680 0x7fc3200c7c50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc31400e980 tx=0x7fc31400ec90 comp rx=0 tx=0).stop 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 shutdown_connections 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc32015d2d0 0x7fc32014bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3200d0680 0x7fc3200c7c50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 --2- 172.21.15.133:0/3279853302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc3200d1060 0x7fc32006c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 >> 172.21.15.133:0/3279853302 conn(0x7fc32009c320 msgr2=0x7fc3200b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 shutdown_connections 2024-03-29T09:08:34.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.685+0000 7fc326600700 1 -- 172.21.15.133:0/3279853302 wait complete. 2024-03-29T09:08:34.704 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 09245253-441b-44d5-ae4c-116226d0ecdf 2 2024-03-29T09:08:34.773 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:34.774+0000 7fd4e00ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T09:08:34.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.933+0000 7f595da00700 1 Processor -- start 2024-03-29T09:08:34.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.934+0000 7f595da00700 1 -- start start 2024-03-29T09:08:34.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f595da00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5958160770 0x7f59580b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f595da00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f59580d0d60 0x7f59580d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f595da00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580d3810 con 0x7f59580d3380 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f595da00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580d1520 con 0x7f5958160770 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f595da00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580d1800 con 0x7f59580d0d60 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f594fe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f59580d0d60 0x7f59580d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f5957400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f59580d3380 0x7f59580cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54220/0 (socket says 172.21.15.133:54220) 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f5957400700 1 -- 172.21.15.133:0/1631953472 learned_addr learned my addr 172.21.15.133:0/1631953472 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f5956a00700 1 --2- 172.21.15.133:0/1631953472 >> v2:172.21.15.99:3300/0 conn(0x7f5958160770 0x7f59580b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f594fe00700 1 -- 172.21.15.133:0/1631953472 >> v2:172.21.15.99:3300/0 conn(0x7f5958160770 msgr2=0x7f59580b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f594fe00700 1 --2- 172.21.15.133:0/1631953472 >> v2:172.21.15.99:3300/0 conn(0x7f5958160770 0x7f59580b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f594fe00700 1 -- 172.21.15.133:0/1631953472 >> v1:172.21.15.133:6789/0 conn(0x7f59580d3380 legacy=0x7f59580cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f594fe00700 1 -- 172.21.15.133:0/1631953472 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580d1ae0 con 0x7f59580d0d60 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.935+0000 7f5956000700 1 -- 172.21.15.133:0/1631953472 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156398330 0 0) 0x7f59580d3810 con 0x7f59580d3380 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.936+0000 7f594fe00700 1 --2- 172.21.15.133:0/1631953472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580d2e40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59580b5cf0 tx=0x7f5940009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5e5797b81b3d7d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.936+0000 7f5956000700 1 -- 172.21.15.133:0/1631953472 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940011830 con 0x7f59580d0d60 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.936+0000 7f5956000700 1 -- 172.21.15.133:0/1631953472 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940011990 con 0x7f59580d0d60 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.936+0000 7f5956000700 1 -- 172.21.15.133:0/1631953472 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940011d20 con 0x7f59580d0d60 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.936+0000 7f595da00700 1 -- 172.21.15.133:0/1631953472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 msgr2=0x7f59580d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.936+0000 7f595da00700 1 --2- 172.21.15.133:0/1631953472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580d2e40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59580b5cf0 tx=0x7f5940009b20 comp rx=0 tx=0).stop 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.937+0000 7f595da00700 1 -- 172.21.15.133:0/1631953472 shutdown_connections 2024-03-29T09:08:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.937+0000 7f595da00700 1 --2- 172.21.15.133:0/1631953472 >> v2:172.21.15.99:3300/0 conn(0x7f5958160770 0x7f59580b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.937+0000 7f595da00700 1 --2- 172.21.15.133:0/1631953472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580d2e40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.937+0000 7f595da00700 1 -- 172.21.15.133:0/1631953472 >> 172.21.15.133:0/1631953472 conn(0x7f595808afd0 msgr2=0x7f595809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.937+0000 7f595da00700 1 -- 172.21.15.133:0/1631953472 shutdown_connections 2024-03-29T09:08:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.937+0000 7f595da00700 1 -- 172.21.15.133:0/1631953472 wait complete. 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.938+0000 7f595da00700 1 Processor -- start 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.939+0000 7f595da00700 1 -- start start 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.939+0000 7f595da00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.939+0000 7f595da00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59580d3380 0x7f59580cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f595da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5958160770 0x7f59581f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f595da00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580ccc30 con 0x7f5958160770 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f595da00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580b6520 con 0x7f59580d0d60 2024-03-29T09:08:34.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f595da00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59580b6800 con 0x7f59580d3380 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f594fe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59580d3380 0x7f59580cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39308/0 (socket says 172.21.15.133:39308) 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 -- 172.21.15.133:0/1088027951 learned_addr learned my addr 172.21.15.133:0/1088027951 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f594fe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59580d3380 0x7f59580cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:45844/0 (socket says 172.21.15.133:45844) 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5957400700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5958160770 0x7f59581f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 -- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59580d3380 msgr2=0x7f59580cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59580d3380 0x7f59580cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 -- 172.21.15.133:0/1088027951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5958160770 msgr2=0x7f59581f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5958160770 0x7f59581f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f5956a00700 1 -- 172.21.15.133:0/1088027951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400094a0 con 0x7f59580d0d60 2024-03-29T09:08:34.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.940+0000 7f594fe00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59580d3380 0x7f59580cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f5956a00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580b5e40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f594800e870 tx=0x7f594800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f594ea00700 1 -- 172.21.15.133:0/1088027951 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948009590 con 0x7f59580d0d60 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f594ea00700 1 -- 172.21.15.133:0/1088027951 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948013680 con 0x7f59580d0d60 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f594ea00700 1 -- 172.21.15.133:0/1088027951 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480139f0 con 0x7f59580d0d60 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59580b6ae0 con 0x7f59580d0d60 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59580b70a0 con 0x7f59580d0d60 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f594ea00700 1 -- 172.21.15.133:0/1088027951 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f594801c450 con 0x7f59580d0d60 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.941+0000 7f594ea00700 1 -- 172.21.15.133:0/1088027951 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f594801cb20 con 0x7f59580d0d60 2024-03-29T09:08:34.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.942+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59580678b0 con 0x7f59580d0d60 2024-03-29T09:08:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:34.945+0000 7f594ea00700 1 -- 172.21.15.133:0/1088027951 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f5948021060 con 0x7f59580d0d60 2024-03-29T09:08:34.960 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:34.960+0000 7f7c7e72d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T09:08:35.047 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:35.047+0000 7f7c7e72d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T09:08:35.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.067+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "09245253-441b-44d5-ae4c-116226d0ecdf", "id": 2} v 0) v1 -- 0x7f59580d0660 con 0x7f59580d0d60 2024-03-29T09:08:35.072 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:35.072+0000 7fd4e00ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T09:08:35.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.075+0000 7f594ea00700 1 -- 172.21.15.133:0/1088027951 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "09245253-441b-44d5-ae4c-116226d0ecdf", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f594802a020 con 0x7f59580d0d60 2024-03-29T09:08:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:2 2024-03-29T09:08:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 msgr2=0x7f59580b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580b5e40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f594800e870 tx=0x7f594800eb80 comp rx=0 tx=0).stop 2024-03-29T09:08:35.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 shutdown_connections 2024-03-29T09:08:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5958160770 0x7f59581f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59580d0d60 0x7f59580b5e40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 --2- 172.21.15.133:0/1088027951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59580d3380 0x7f59580cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 >> 172.21.15.133:0/1088027951 conn(0x7f595808afd0 msgr2=0x7f59580b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 shutdown_connections 2024-03-29T09:08:35.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.076+0000 7f595da00700 1 -- 172.21.15.133:0/1088027951 wait complete. 2024-03-29T09:08:35.094 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 3d441769-67e2-4cf9-8501-9a9dee172963 3 2024-03-29T09:08:35.171 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:35.172+0000 7fd4e00ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T09:08:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.321+0000 7f1bf3400700 1 Processor -- start 2024-03-29T09:08:35.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.321+0000 7f1bf3400700 1 -- start start 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec160770 0x7f1bec0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf3400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1bec0d0d60 0x7f1bec0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf3400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1bec0d3380 0x7f1bec0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf3400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0d3810 con 0x7f1bec0d0d60 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf3400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0d1520 con 0x7f1bec0d3380 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf3400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0d1800 con 0x7f1bec160770 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf1400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1bec0d3380 0x7f1bec0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf0a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec160770 0x7f1bec0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf1400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1bec0d3380 0x7f1bec0cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39310/0 (socket says 172.21.15.133:39310) 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf0a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec160770 0x7f1bec0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:45860/0 (socket says 172.21.15.133:45860) 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bf1400700 1 -- 172.21.15.133:0/2532788739 learned_addr learned my addr 172.21.15.133:0/2532788739 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:35.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.322+0000 7f1bebe00700 1 --2- 172.21.15.133:0/2532788739 >> v2:172.21.15.99:3300/0 conn(0x7f1bec0d0d60 0x7f1bec0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1bf1400700 1 -- 172.21.15.133:0/2532788739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec160770 msgr2=0x7f1bec0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1bf1400700 1 --2- 172.21.15.133:0/2532788739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec160770 0x7f1bec0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1bf1400700 1 -- 172.21.15.133:0/2532788739 >> v2:172.21.15.99:3300/0 conn(0x7f1bec0d0d60 msgr2=0x7f1bec0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1bf1400700 1 --2- 172.21.15.133:0/2532788739 >> v2:172.21.15.99:3300/0 conn(0x7f1bec0d0d60 0x7f1bec0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1bf1400700 1 -- 172.21.15.133:0/2532788739 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec0d1ae0 con 0x7f1bec0d3380 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1bebe00700 1 --2- 172.21.15.133:0/2532788739 >> v2:172.21.15.99:3300/0 conn(0x7f1bec0d0d60 0x7f1bec0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1bf1400700 1 --2- 172.21.15.133:0/2532788739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec0cfa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0cd7e0 tx=0x7f1be400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8469447624b43f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1beb400700 1 -- 172.21.15.133:0/2532788739 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400e070 con 0x7f1bec0d3380 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1beb400700 1 -- 172.21.15.133:0/2532788739 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be400b910 con 0x7f1bec0d3380 2024-03-29T09:08:35.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.323+0000 7f1beb400700 1 -- 172.21.15.133:0/2532788739 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be400bcc0 con 0x7f1bec0d3380 2024-03-29T09:08:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 -- 172.21.15.133:0/2532788739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 msgr2=0x7f1bec0cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 --2- 172.21.15.133:0/2532788739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec0cfa80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1bec0cd7e0 tx=0x7f1be400b330 comp rx=0 tx=0).stop 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 -- 172.21.15.133:0/2532788739 shutdown_connections 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 --2- 172.21.15.133:0/2532788739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec160770 0x7f1bec0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 --2- 172.21.15.133:0/2532788739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec0cfa80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 --2- 172.21.15.133:0/2532788739 >> v2:172.21.15.99:3300/0 conn(0x7f1bec0d0d60 0x7f1bec0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 -- 172.21.15.133:0/2532788739 >> 172.21.15.133:0/2532788739 conn(0x7f1bec08afd0 msgr2=0x7f1bec09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 -- 172.21.15.133:0/2532788739 shutdown_connections 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.324+0000 7f1bf3400700 1 -- 172.21.15.133:0/2532788739 wait complete. 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.326+0000 7f1bf3400700 1 Processor -- start 2024-03-29T09:08:35.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.326+0000 7f1bf3400700 1 -- start start 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec0d0d60 0x7f1bec1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec09df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bec160770 0x7f1bec0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf3400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec09ff50 con 0x7f1bec160770 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf3400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0a0230 con 0x7f1bec0d3380 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf3400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0a0510 con 0x7f1bec0d0d60 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bebe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec09df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf0a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec0d0d60 0x7f1bec1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf0a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec0d0d60 0x7f1bec1545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:45868/0 (socket says 172.21.15.133:45868) 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf0a00700 1 -- 172.21.15.133:0/3924268663 learned_addr learned my addr 172.21.15.133:0/3924268663 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bebe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec09df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39322/0 (socket says 172.21.15.133:39322) 2024-03-29T09:08:35.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.327+0000 7f1bf1400700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bec160770 0x7f1bec0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf0a00700 1 -- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 msgr2=0x7f1bec09df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf0a00700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec09df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf0a00700 1 -- 172.21.15.133:0/3924268663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bec160770 msgr2=0x7f1bec0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf0a00700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bec160770 0x7f1bec0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf0a00700 1 -- 172.21.15.133:0/3924268663 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4009b30 con 0x7f1bec0d0d60 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bebe00700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec09df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf0a00700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec0d0d60 0x7f1bec1545d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1be000e870 tx=0x7f1be000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1be8c00700 1 -- 172.21.15.133:0/3924268663 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be0009590 con 0x7f1bec0d0d60 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1be8c00700 1 -- 172.21.15.133:0/3924268663 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be0013680 con 0x7f1bec0d0d60 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1be8c00700 1 -- 172.21.15.133:0/3924268663 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be00139f0 con 0x7f1bec0d0d60 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec0a07f0 con 0x7f1bec0d0d60 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.328+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec154b10 con 0x7f1bec0d0d60 2024-03-29T09:08:35.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.329+0000 7f1be8c00700 1 -- 172.21.15.133:0/3924268663 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1be001c450 con 0x7f1bec0d0d60 2024-03-29T09:08:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.329+0000 7f1be8c00700 1 -- 172.21.15.133:0/3924268663 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f1be0012900 con 0x7f1bec0d0d60 2024-03-29T09:08:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.329+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec0a0b60 con 0x7f1bec0d0d60 2024-03-29T09:08:35.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.332+0000 7f1be8c00700 1 -- 172.21.15.133:0/3924268663 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f1be0021060 con 0x7f1bec0d0d60 2024-03-29T09:08:35.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.419+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3d441769-67e2-4cf9-8501-9a9dee172963", "id": 3} v 0) v1 -- 0x7f1bec0678b0 con 0x7f1bec0d0d60 2024-03-29T09:08:35.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.428+0000 7f1be8c00700 1 -- 172.21.15.133:0/3924268663 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3d441769-67e2-4cf9-8501-9a9dee172963", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1be0017070 con 0x7f1bec0d0d60 2024-03-29T09:08:35.427 INFO:teuthology.orchestra.run.smithi133.stdout:3 2024-03-29T09:08:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec0d0d60 msgr2=0x7f1bec1545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec0d0d60 0x7f1bec1545d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1be000e870 tx=0x7f1be000eb80 comp rx=0 tx=0).stop 2024-03-29T09:08:35.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 shutdown_connections 2024-03-29T09:08:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bec160770 0x7f1bec0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1bec0d0d60 0x7f1bec1545d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 --2- 172.21.15.133:0/3924268663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bec0d3380 0x7f1bec09df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 >> 172.21.15.133:0/3924268663 conn(0x7f1bec08afd0 msgr2=0x7f1bec06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 shutdown_connections 2024-03-29T09:08:35.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.429+0000 7f1bf3400700 1 -- 172.21.15.133:0/3924268663 wait complete. 2024-03-29T09:08:35.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 8f07961c-cadf-41ba-b375-d6f1a6ebfb1f 4 2024-03-29T09:08:35.512 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:35.512+0000 7f7c7e72d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T09:08:35.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.671+0000 7f0a14000700 1 Processor -- start 2024-03-29T09:08:35.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.672+0000 7f0a14000700 1 -- start start 2024-03-29T09:08:35.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a14000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0a0c0d08a0 0x7f0a0c06f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a14000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a0c15db20 0x7f0a0c0add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a14000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0ad870 con 0x7f0a0c15a110 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a14000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c06f580 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a14000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c0b30c0 con 0x7f0a0c15db20 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a11600700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0a0c15a110 0x7f0a0c0d0190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54232/0 (socket says 172.21.15.133:54232) 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a11600700 1 -- 172.21.15.133:0/377168882 learned_addr learned my addr 172.21.15.133:0/377168882 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a12000700 1 --2- 172.21.15.133:0/377168882 >> v2:172.21.15.133:3300/0 conn(0x7f0a0c15db20 0x7f0a0c0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.673+0000 7f0a10c00700 1 --2- 172.21.15.133:0/377168882 >> v2:172.21.15.99:3300/0 conn(0x7f0a0c0d08a0 0x7f0a0c06f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a0be00700 1 -- 172.21.15.133:0/377168882 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054059001 0 0) 0x7f0a0c0ad870 con 0x7f0a0c15a110 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a0be00700 1 -- 172.21.15.133:0/377168882 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f40035b0 con 0x7f0a0c15a110 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a10c00700 1 -- 172.21.15.133:0/377168882 >> v2:172.21.15.133:3300/0 conn(0x7f0a0c15db20 msgr2=0x7f0a0c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a10c00700 1 --2- 172.21.15.133:0/377168882 >> v2:172.21.15.133:3300/0 conn(0x7f0a0c15db20 0x7f0a0c0add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a10c00700 1 -- 172.21.15.133:0/377168882 >> v1:172.21.15.133:6789/0 conn(0x7f0a0c15a110 legacy=0x7f0a0c0d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a10c00700 1 -- 172.21.15.133:0/377168882 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c15dfe0 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a10c00700 1 --2- 172.21.15.133:0/377168882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c06f040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0af4c0 tx=0x7f09f8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d55180d05e7fa7f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:35.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a0be00700 1 -- 172.21.15.133:0/377168882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8011830 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a0be00700 1 -- 172.21.15.133:0/377168882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f8011990 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.674+0000 7f0a0be00700 1 -- 172.21.15.133:0/377168882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8011d20 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 -- 172.21.15.133:0/377168882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c0d08a0 msgr2=0x7f0a0c06f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 --2- 172.21.15.133:0/377168882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c06f040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0af4c0 tx=0x7f09f8009b20 comp rx=0 tx=0).stop 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 -- 172.21.15.133:0/377168882 shutdown_connections 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 --2- 172.21.15.133:0/377168882 >> v2:172.21.15.133:3300/0 conn(0x7f0a0c15db20 0x7f0a0c0add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 --2- 172.21.15.133:0/377168882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c06f040 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 -- 172.21.15.133:0/377168882 >> 172.21.15.133:0/377168882 conn(0x7f0a0c066720 msgr2=0x7f0a0c066b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 -- 172.21.15.133:0/377168882 shutdown_connections 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.675+0000 7f0a14000700 1 -- 172.21.15.133:0/377168882 wait complete. 2024-03-29T09:08:35.676 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.677+0000 7f0a14000700 1 Processor -- start 2024-03-29T09:08:35.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.677+0000 7f0a14000700 1 -- start start 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a14000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a14000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a0c15a110 0x7f0a0c154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a14000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c15db20 0x7f0a0c0c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a14000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c15bf80 con 0x7f0a0c15db20 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a14000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c155bd0 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a14000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c155e70 con 0x7f0a0c15a110 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a11600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a11600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39330/0 (socket says 172.21.15.133:39330) 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a10c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a0c15a110 0x7f0a0c154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a11600700 1 -- 172.21.15.133:0/3528556546 learned_addr learned my addr 172.21.15.133:0/3528556546 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:35.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.678+0000 7f0a12000700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c15db20 0x7f0a0c0c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a11600700 1 -- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a0c15a110 msgr2=0x7f0a0c154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a11600700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a0c15a110 0x7f0a0c154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a11600700 1 -- 172.21.15.133:0/3528556546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c15db20 msgr2=0x7f0a0c0c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a11600700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c15db20 0x7f0a0c0c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a11600700 1 -- 172.21.15.133:0/3528556546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f80094a0 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a10c00700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a0c15a110 0x7f0a0c154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a12000700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c15db20 0x7f0a0c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a11600700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c150c30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c700 tx=0x7f0a0000ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a09600700 1 -- 172.21.15.133:0/3528556546 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00005d70 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a09600700 1 -- 172.21.15.133:0/3528556546 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000070d0 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a09600700 1 -- 172.21.15.133:0/3528556546 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007460 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c156150 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1f3ae0 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a09600700 1 -- 172.21.15.133:0/3528556546 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0a0001c440 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.679+0000 7f0a09600700 1 -- 172.21.15.133:0/3528556546 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f0a00012910 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.680+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c0678b0 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.683+0000 7f0a09600700 1 -- 172.21.15.133:0/3528556546 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f0a00017070 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.721 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:35.721+0000 7fd4e00ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T09:08:35.768 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.769+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8f07961c-cadf-41ba-b375-d6f1a6ebfb1f", "id": 4} v 0) v1 -- 0x7f0a0c06df00 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.777+0000 7f0a09600700 1 -- 172.21.15.133:0/3528556546 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8f07961c-cadf-41ba-b375-d6f1a6ebfb1f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0a00012bf0 con 0x7f0a0c0d08a0 2024-03-29T09:08:35.776 INFO:teuthology.orchestra.run.smithi133.stdout:4 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0c0d08a0 msgr2=0x7f0a0c150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c150c30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c700 tx=0x7f0a0000ca10 comp rx=0 tx=0).stop 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 shutdown_connections 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a0c15db20 0x7f0a0c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a0c0d08a0 0x7f0a0c150c30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 --2- 172.21.15.133:0/3528556546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a0c15a110 0x7f0a0c154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 >> 172.21.15.133:0/3528556546 conn(0x7f0a0c066720 msgr2=0x7f0a0c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 shutdown_connections 2024-03-29T09:08:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.778+0000 7f0a14000700 1 -- 172.21.15.133:0/3528556546 wait complete. 2024-03-29T09:08:35.794 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 73ad4bba-ab4f-481d-b7b4-5910fe9d2f82 5 2024-03-29T09:08:35.902 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:35.902+0000 7f7c7e72d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T09:08:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.981+0000 7f92bcc00700 1 Processor -- start 2024-03-29T09:08:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.981+0000 7f92bcc00700 1 -- start start 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.982+0000 7f92bcc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92b815d780 0x7f92b80cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.982+0000 7f92bcc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b806bf50 con 0x7f92b80cfde0 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.982+0000 7f92bcc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b806c1f0 con 0x7f92b809a150 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.982+0000 7f92bcc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b806c4d0 con 0x7f92b815d780 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.982+0000 7f92b5e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92b815d780 0x7f92b80cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.982+0000 7f92b5e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92b815d780 0x7f92b80cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39338/0 (socket says 172.21.15.133:39338) 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.982+0000 7f92b5e00700 1 -- 172.21.15.133:0/2947151562 learned_addr learned my addr 172.21.15.133:0/2947151562 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.983+0000 7f92b5e00700 1 -- 172.21.15.133:0/2947151562 >> v1:172.21.15.133:6789/0 conn(0x7f92b809a150 legacy=0x7f92b80cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.983+0000 7f92b5e00700 1 -- 172.21.15.133:0/2947151562 >> v1:172.21.15.99:6789/0 conn(0x7f92b80cfde0 legacy=0x7f92b8099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.983+0000 7f92b5e00700 1 -- 172.21.15.133:0/2947151562 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b806c7b0 con 0x7f92b815d780 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.983+0000 7f92b5e00700 1 --2- 172.21.15.133:0/2947151562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b815d780 0x7f92b80cf8a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f92a80098f0 tx=0x7f92a8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118a5e48e03eefbf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.983+0000 7f92b4a00700 1 -- 172.21.15.133:0/2947151562 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800b8a0 con 0x7f92b815d780 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.983+0000 7f92b4a00700 1 -- 172.21.15.133:0/2947151562 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a800ba00 con 0x7f92b815d780 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.983+0000 7f92b4a00700 1 -- 172.21.15.133:0/2947151562 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800bd90 con 0x7f92b815d780 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.984+0000 7f92bcc00700 1 -- 172.21.15.133:0/2947151562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b815d780 msgr2=0x7f92b80cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.984+0000 7f92bcc00700 1 --2- 172.21.15.133:0/2947151562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b815d780 0x7f92b80cf8a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f92a80098f0 tx=0x7f92a8009c00 comp rx=0 tx=0).stop 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.984+0000 7f92bcc00700 1 -- 172.21.15.133:0/2947151562 shutdown_connections 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.984+0000 7f92bcc00700 1 --2- 172.21.15.133:0/2947151562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b815d780 0x7f92b80cf8a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.984+0000 7f92bcc00700 1 -- 172.21.15.133:0/2947151562 >> 172.21.15.133:0/2947151562 conn(0x7f92b80b2b20 msgr2=0x7f92b80b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.984+0000 7f92bcc00700 1 -- 172.21.15.133:0/2947151562 shutdown_connections 2024-03-29T09:08:35.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.984+0000 7f92bcc00700 1 -- 172.21.15.133:0/2947151562 wait complete. 2024-03-29T09:08:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.986+0000 7f92bcc00700 1 Processor -- start 2024-03-29T09:08:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.987+0000 7f92bcc00700 1 -- start start 2024-03-29T09:08:35.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92bcc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b809a150 0x7f92b80caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92bcc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80cfde0 0x7f92b8151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92bcc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92b815d780 0x7f92b80c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92bcc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80c8a70 con 0x7f92b80cfde0 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92bcc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80c8ca0 con 0x7f92b809a150 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92bcc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80c8f80 con 0x7f92b815d780 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b809a150 0x7f92b80caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b809a150 0x7f92b80caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39346/0 (socket says 172.21.15.133:39346) 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 -- 172.21.15.133:0/413022733 learned_addr learned my addr 172.21.15.133:0/413022733 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b6800700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92b815d780 0x7f92b80c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5400700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80cfde0 0x7f92b8151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 -- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92b815d780 msgr2=0x7f92b80c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92b815d780 0x7f92b80c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 -- 172.21.15.133:0/413022733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80cfde0 msgr2=0x7f92b8151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80cfde0 0x7f92b8151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.988+0000 7f92b5e00700 1 -- 172.21.15.133:0/413022733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a8009550 con 0x7f92b809a150 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92b5e00700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b809a150 0x7f92b80caac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009130 tx=0x7f92a8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92ae000700 1 -- 172.21.15.133:0/413022733 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80036a0 con 0x7f92b809a150 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92ae000700 1 -- 172.21.15.133:0/413022733 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a801e070 con 0x7f92b809a150 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92ae000700 1 -- 172.21.15.133:0/413022733 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8003a40 con 0x7f92b809a150 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b80c9260 con 0x7f92b809a150 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b80c9820 con 0x7f92b809a150 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92ae000700 1 -- 172.21.15.133:0/413022733 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f92a8025480 con 0x7f92b809a150 2024-03-29T09:08:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.989+0000 7f92ae000700 1 -- 172.21.15.133:0/413022733 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f92a8020070 con 0x7f92b809a150 2024-03-29T09:08:35.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.990+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b80678b0 con 0x7f92b809a150 2024-03-29T09:08:35.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:35.992+0000 7f92ae000700 1 -- 172.21.15.133:0/413022733 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f92a802a020 con 0x7f92b809a150 2024-03-29T09:08:36.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.079+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "73ad4bba-ab4f-481d-b7b4-5910fe9d2f82", "id": 5} v 0) v1 -- 0x7f92b806e540 con 0x7f92b809a150 2024-03-29T09:08:36.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.090+0000 7f92ae000700 1 -- 172.21.15.133:0/413022733 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73ad4bba-ab4f-481d-b7b4-5910fe9d2f82", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f92a8015020 con 0x7f92b809a150 2024-03-29T09:08:36.090 INFO:teuthology.orchestra.run.smithi133.stdout:5 2024-03-29T09:08:36.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.091+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b809a150 msgr2=0x7f92b80caac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.091+0000 7f92bcc00700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b809a150 0x7f92b80caac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009130 tx=0x7f92a8004750 comp rx=0 tx=0).stop 2024-03-29T09:08:36.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.091+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 shutdown_connections 2024-03-29T09:08:36.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.091+0000 7f92bcc00700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92b815d780 0x7f92b80c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.091+0000 7f92bcc00700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92b809a150 0x7f92b80caac0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.091+0000 7f92bcc00700 1 --2- 172.21.15.133:0/413022733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92b80cfde0 0x7f92b8151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.091+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 >> 172.21.15.133:0/413022733 conn(0x7f92b80b2b20 msgr2=0x7f92b80b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:36.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.092+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 shutdown_connections 2024-03-29T09:08:36.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.092+0000 7f92bcc00700 1 -- 172.21.15.133:0/413022733 wait complete. 2024-03-29T09:08:36.102 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:36.103+0000 7f7c7e72d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T09:08:36.110 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 1eac9d41-854e-4bd1-9134-2c80a6c97bc3 6 2024-03-29T09:08:36.145 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:36.146+0000 7fd4e00ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T09:08:36.203 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:36.203+0000 7f7c7e72d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T09:08:36.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.297+0000 7f58ab600700 1 Processor -- start 2024-03-29T09:08:36.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.297+0000 7f58ab600700 1 -- start start 2024-03-29T09:08:36.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58ab600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f58a40b3cc0 0x7f58a40d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58ab600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40d3810 con 0x7f58a40d3380 2024-03-29T09:08:36.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58ab600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a40d1520 con 0x7f58a4160770 2024-03-29T09:08:36.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58ab600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a40b4140 con 0x7f58a40b3cc0 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58a8c00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f58a4160770 0x7f58a40d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:54254/0 (socket says 172.21.15.133:54254) 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58a8c00700 1 -- 172.21.15.133:0/1816649286 learned_addr learned my addr 172.21.15.133:0/1816649286 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58a3e00700 1 --2- 172.21.15.133:0/1816649286 >> v2:172.21.15.99:3300/0 conn(0x7f58a40b3cc0 0x7f58a40d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58a3400700 1 -- 172.21.15.133:0/1816649286 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106603244 0 0) 0x7f58a40d1520 con 0x7f58a4160770 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58a3400700 1 -- 172.21.15.133:0/1816649286 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588c0035b0 con 0x7f58a4160770 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58a3400700 1 -- 172.21.15.133:0/1816649286 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395801649 0 0) 0x7f58a40d3810 con 0x7f58a40d3380 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.299+0000 7f58a3400700 1 -- 172.21.15.133:0/1816649286 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a40d1520 con 0x7f58a40d3380 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.300+0000 7f58a3e00700 1 -- 172.21.15.133:0/1816649286 >> v1:172.21.15.133:6789/0 conn(0x7f58a4160770 legacy=0x7f58a40d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.300+0000 7f58a3e00700 1 -- 172.21.15.133:0/1816649286 >> v1:172.21.15.99:6789/0 conn(0x7f58a40d3380 legacy=0x7f58a40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.300+0000 7f58a3e00700 1 -- 172.21.15.133:0/1816649286 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40d1800 con 0x7f58a40b3cc0 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.300+0000 7f58a3e00700 1 --2- 172.21.15.133:0/1816649286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40b3cc0 0x7f58a40d2e40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5890009840 tx=0x7f5890009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97980dfaa1560ba server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.300+0000 7f58a3400700 1 -- 172.21.15.133:0/1816649286 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890011830 con 0x7f58a40b3cc0 2024-03-29T09:08:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.300+0000 7f58a3400700 1 -- 172.21.15.133:0/1816649286 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890011990 con 0x7f58a40b3cc0 2024-03-29T09:08:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.300+0000 7f58a3400700 1 -- 172.21.15.133:0/1816649286 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890011d20 con 0x7f58a40b3cc0 2024-03-29T09:08:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.301+0000 7f58ab600700 1 -- 172.21.15.133:0/1816649286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40b3cc0 msgr2=0x7f58a40d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.301+0000 7f58ab600700 1 --2- 172.21.15.133:0/1816649286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40b3cc0 0x7f58a40d2e40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5890009840 tx=0x7f5890009b50 comp rx=0 tx=0).stop 2024-03-29T09:08:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.301+0000 7f58ab600700 1 -- 172.21.15.133:0/1816649286 shutdown_connections 2024-03-29T09:08:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.301+0000 7f58ab600700 1 --2- 172.21.15.133:0/1816649286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40b3cc0 0x7f58a40d2e40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.301+0000 7f58ab600700 1 -- 172.21.15.133:0/1816649286 >> 172.21.15.133:0/1816649286 conn(0x7f58a408afd0 msgr2=0x7f58a409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:36.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.301+0000 7f58ab600700 1 -- 172.21.15.133:0/1816649286 shutdown_connections 2024-03-29T09:08:36.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.301+0000 7f58ab600700 1 -- 172.21.15.133:0/1816649286 wait complete. 2024-03-29T09:08:36.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.303+0000 7f58ab600700 1 Processor -- start 2024-03-29T09:08:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.304+0000 7f58ab600700 1 -- start start 2024-03-29T09:08:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58ab600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58a40b3cc0 0x7f58a40cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58ab600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40d3380 0x7f58a40b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58ab600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58a4160770 0x7f58a409e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58ab600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a415c9c0 con 0x7f58a40d3380 2024-03-29T09:08:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58ab600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40ca610 con 0x7f58a40b3cc0 2024-03-29T09:08:36.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58ab600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a40ca8f0 con 0x7f58a4160770 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a8c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58a40b3cc0 0x7f58a40cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a8c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58a40b3cc0 0x7f58a40cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39348/0 (socket says 172.21.15.133:39348) 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a8c00700 1 -- 172.21.15.133:0/3285167094 learned_addr learned my addr 172.21.15.133:0/3285167094 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a3e00700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40d3380 0x7f58a40b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a9600700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58a4160770 0x7f58a409e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a3e00700 1 -- 172.21.15.133:0/3285167094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58a4160770 msgr2=0x7f58a409e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a3e00700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58a4160770 0x7f58a409e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a3e00700 1 -- 172.21.15.133:0/3285167094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58a40b3cc0 msgr2=0x7f58a40cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a3e00700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58a40b3cc0 0x7f58a40cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.305+0000 7f58a3e00700 1 -- 172.21.15.133:0/3285167094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58900094a0 con 0x7f58a40d3380 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58a3e00700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40d3380 0x7f58a40b6bd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5890016000 tx=0x7f5890004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58a0c00700 1 -- 172.21.15.133:0/3285167094 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589001a6e0 con 0x7f58a40d3380 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58a0c00700 1 -- 172.21.15.133:0/3285167094 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589001a860 con 0x7f58a40d3380 2024-03-29T09:08:36.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a40cabd0 con 0x7f58a40d3380 2024-03-29T09:08:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58a0c00700 1 -- 172.21.15.133:0/3285167094 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589001ac10 con 0x7f58a40d3380 2024-03-29T09:08:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a40cb190 con 0x7f58a40d3380 2024-03-29T09:08:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58a0c00700 1 -- 172.21.15.133:0/3285167094 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5890022070 con 0x7f58a40d3380 2024-03-29T09:08:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.306+0000 7f58a0c00700 1 -- 172.21.15.133:0/3285167094 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f589002ed30 con 0x7f58a40d3380 2024-03-29T09:08:36.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.307+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a40cf450 con 0x7f58a40d3380 2024-03-29T09:08:36.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.311+0000 7f58a0c00700 1 -- 172.21.15.133:0/3285167094 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f5890029070 con 0x7f58a40d3380 2024-03-29T09:08:36.356 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:36.357+0000 7fd4e00ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-29T09:08:36.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.395+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1eac9d41-854e-4bd1-9134-2c80a6c97bc3", "id": 6} v 0) v1 -- 0x7f58a40678b0 con 0x7f58a40d3380 2024-03-29T09:08:36.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.399+0000 7f58a0c00700 1 -- 172.21.15.133:0/3285167094 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1eac9d41-854e-4bd1-9134-2c80a6c97bc3", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5890015020 con 0x7f58a40d3380 2024-03-29T09:08:36.399 INFO:teuthology.orchestra.run.smithi133.stdout:6 2024-03-29T09:08:36.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40d3380 msgr2=0x7f58a40b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40d3380 0x7f58a40b6bd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5890016000 tx=0x7f5890004750 comp rx=0 tx=0).stop 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 shutdown_connections 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58a4160770 0x7f58a409e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58a40b3cc0 0x7f58a40cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 --2- 172.21.15.133:0/3285167094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58a40d3380 0x7f58a40b6bd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 >> 172.21.15.133:0/3285167094 conn(0x7f58a408afd0 msgr2=0x7f58a40d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 shutdown_connections 2024-03-29T09:08:36.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.400+0000 7f58ab600700 1 -- 172.21.15.133:0/3285167094 wait complete. 2024-03-29T09:08:36.417 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 2fb07c4a-1ab7-454c-9a54-f127a9e53bff 7 2024-03-29T09:08:36.461 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:36.461+0000 7fd4e00ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T09:08:36.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.645+0000 7f3153800700 1 Processor -- start 2024-03-29T09:08:36.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.645+0000 7f3153800700 1 -- start start 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f3153800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f314c15d800 0x7f314c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f3153800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c0d1e90 0x7f314c0d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f3153800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c0d38d0 con 0x7f314c15d420 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f3153800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c0d12b0 con 0x7f314c15d800 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f3153800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0afdd0 con 0x7f314c0d1e90 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f314be00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f314c15d800 0x7f314c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f3151800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c0d1e90 0x7f314c0d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f314be00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f314c15d800 0x7f314c0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39360/0 (socket says 172.21.15.133:39360) 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f3150e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f314c15d420 0x7f314c0d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.133:50104/0 (socket says 172.21.15.133:50104) 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.646+0000 7f314be00700 1 -- 172.21.15.133:0/2728445523 learned_addr learned my addr 172.21.15.133:0/2728445523 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314be00700 1 -- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c0d1e90 msgr2=0x7f314c0d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:36.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314be00700 1 --2- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c0d1e90 0x7f314c0d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314be00700 1 -- 172.21.15.133:0/2728445523 >> v1:172.21.15.99:6789/0 conn(0x7f314c15d420 legacy=0x7f314c0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314be00700 1 -- 172.21.15.133:0/2728445523 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c0d3b00 con 0x7f314c15d800 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f3151800700 1 --2- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c0d1e90 0x7f314c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314b400700 1 -- 172.21.15.133:0/2728445523 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102810065 0 0) 0x7f314c0d38d0 con 0x7f314c15d420 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314be00700 1 --2- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c15d800 0x7f314c0d2e50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f31380098f0 tx=0x7f3138009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4774a10c5621015c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314b400700 1 -- 172.21.15.133:0/2728445523 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138011830 con 0x7f314c15d800 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314b400700 1 -- 172.21.15.133:0/2728445523 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3138011990 con 0x7f314c15d800 2024-03-29T09:08:36.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.647+0000 7f314b400700 1 -- 172.21.15.133:0/2728445523 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138011d20 con 0x7f314c15d800 2024-03-29T09:08:36.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 -- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c15d800 msgr2=0x7f314c0d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 --2- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c15d800 0x7f314c0d2e50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f31380098f0 tx=0x7f3138009c00 comp rx=0 tx=0).stop 2024-03-29T09:08:36.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 -- 172.21.15.133:0/2728445523 shutdown_connections 2024-03-29T09:08:36.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 --2- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c0d1e90 0x7f314c0d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 --2- 172.21.15.133:0/2728445523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c15d800 0x7f314c0d2e50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 -- 172.21.15.133:0/2728445523 >> 172.21.15.133:0/2728445523 conn(0x7f314c08afd0 msgr2=0x7f314c09c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:36.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 -- 172.21.15.133:0/2728445523 shutdown_connections 2024-03-29T09:08:36.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.648+0000 7f3153800700 1 -- 172.21.15.133:0/2728445523 wait complete. 2024-03-29T09:08:36.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.650+0000 7f3153800700 1 Processor -- start 2024-03-29T09:08:36.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.650+0000 7f3153800700 1 -- start start 2024-03-29T09:08:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3153800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c0d1e90 0x7f314c0cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3153800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f314c15d420 0x7f314c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3153800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c1525a0 0x7f314c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3153800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0cbea0 con 0x7f314c15d420 2024-03-29T09:08:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3153800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0cc180 con 0x7f314c0d1e90 2024-03-29T09:08:36.652 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3153800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0cc460 con 0x7f314c1525a0 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3150e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c0d1e90 0x7f314c0cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3151800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c1525a0 0x7f314c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3150e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c0d1e90 0x7f314c0cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39362/0 (socket says 172.21.15.133:39362) 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f3150e00700 1 -- 172.21.15.133:0/3408576813 learned_addr learned my addr 172.21.15.133:0/3408576813 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.651+0000 7f314be00700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f314c15d420 0x7f314c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3151800700 1 -- 172.21.15.133:0/3408576813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c0d1e90 msgr2=0x7f314c0cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3151800700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c0d1e90 0x7f314c0cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3151800700 1 -- 172.21.15.133:0/3408576813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f314c15d420 msgr2=0x7f314c15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3151800700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f314c15d420 0x7f314c15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3151800700 1 -- 172.21.15.133:0/3408576813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3144009a70 con 0x7f314c1525a0 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3151800700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c1525a0 0x7f314c15af20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f314400efc0 tx=0x7f314400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3148c00700 1 -- 172.21.15.133:0/3408576813 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314400ca90 con 0x7f314c1525a0 2024-03-29T09:08:36.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3148c00700 1 -- 172.21.15.133:0/3408576813 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314400f040 con 0x7f314c1525a0 2024-03-29T09:08:36.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3138009550 con 0x7f314c1525a0 2024-03-29T09:08:36.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3148c00700 1 -- 172.21.15.133:0/3408576813 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314400ce30 con 0x7f314c1525a0 2024-03-29T09:08:36.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c0c77e0 con 0x7f314c1525a0 2024-03-29T09:08:36.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.652+0000 7f3148c00700 1 -- 172.21.15.133:0/3408576813 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3144003d20 con 0x7f314c1525a0 2024-03-29T09:08:36.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.653+0000 7f3148c00700 1 -- 172.21.15.133:0/3408576813 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f3144014480 con 0x7f314c1525a0 2024-03-29T09:08:36.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.654+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c0cbfe0 con 0x7f314c1525a0 2024-03-29T09:08:36.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.656+0000 7f3148c00700 1 -- 172.21.15.133:0/3408576813 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f314c0cbfe0 con 0x7f314c1525a0 2024-03-29T09:08:36.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.742+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2fb07c4a-1ab7-454c-9a54-f127a9e53bff", "id": 7} v 0) v1 -- 0x7f314c0678b0 con 0x7f314c1525a0 2024-03-29T09:08:36.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.754+0000 7f3148c00700 1 -- 172.21.15.133:0/3408576813 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2fb07c4a-1ab7-454c-9a54-f127a9e53bff", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f314402b0b0 con 0x7f314c1525a0 2024-03-29T09:08:36.753 INFO:teuthology.orchestra.run.smithi133.stdout:7 2024-03-29T09:08:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c1525a0 msgr2=0x7f314c15af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c1525a0 0x7f314c15af20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f314400efc0 tx=0x7f314400c3a0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 shutdown_connections 2024-03-29T09:08:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f314c1525a0 0x7f314c15af20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f314c0d1e90 0x7f314c0cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 --2- 172.21.15.133:0/3408576813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f314c15d420 0x7f314c15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 >> 172.21.15.133:0/3408576813 conn(0x7f314c08afd0 msgr2=0x7f314c0b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 shutdown_connections 2024-03-29T09:08:36.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-29T09:08:36.755+0000 7f3153800700 1 -- 172.21.15.133:0/3408576813 wait complete. 2024-03-29T09:08:36.771 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-29T09:08:36.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-29T09:08:36.775 INFO:tasks.ceph.osd.0:Started 2024-03-29T09:08:36.776 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-29T09:08:36.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-29T09:08:36.779 INFO:tasks.ceph.osd.1:Started 2024-03-29T09:08:36.779 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-29T09:08:36.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-29T09:08:36.783 INFO:tasks.ceph.osd.2:Started 2024-03-29T09:08:36.783 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-29T09:08:36.783 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-29T09:08:36.788 INFO:tasks.ceph.osd.3:Started 2024-03-29T09:08:36.789 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-29T09:08:36.789 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-29T09:08:36.793 INFO:tasks.ceph.osd.4:Started 2024-03-29T09:08:36.793 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-29T09:08:36.793 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-29T09:08:36.797 INFO:tasks.ceph.osd.5:Started 2024-03-29T09:08:36.797 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-29T09:08:36.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-29T09:08:36.800 INFO:tasks.ceph.osd.6:Started 2024-03-29T09:08:36.801 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-29T09:08:36.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-29T09:08:36.805 INFO:tasks.ceph.osd.7:Started 2024-03-29T09:08:36.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:08:37.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.068+0000 7fd12fa00700 1 Processor -- start 2024-03-29T09:08:37.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.068+0000 7fd12fa00700 1 -- start start 2024-03-29T09:08:37.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.069+0000 7fd12fa00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1280cfd70 0x7fd1281615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:37.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12fa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd128161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:37.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12fa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd128162020 0x7fd1280b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:37.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12fa00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1280d1be0 con 0x7fd128162020 2024-03-29T09:08:37.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12fa00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1280b4980 con 0x7fd1280cfd70 2024-03-29T09:08:37.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12fa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280b46a0 con 0x7fd1280d0150 2024-03-29T09:08:37.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12da00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd128162020 0x7fd1280b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1280cfd70 0x7fd1281615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd127e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd128161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1280cfd70 0x7fd1281615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:56194/0 (socket says 172.21.15.99:56194) 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd127e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd128161ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:59472/0 (socket says 172.21.15.99:59472) 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 -- 172.21.15.99:0/414937163 learned_addr learned my addr 172.21.15.99:0/414937163 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12da00700 1 --2- 172.21.15.99:0/414937163 >> v2:172.21.15.99:3300/0 conn(0x7fd128162020 0x7fd1280b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43528/0 (socket says 172.21.15.99:43528) 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 -- 172.21.15.99:0/414937163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 msgr2=0x7fd128161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 --2- 172.21.15.99:0/414937163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd128161ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 -- 172.21.15.99:0/414937163 >> v2:172.21.15.99:3300/0 conn(0x7fd128162020 msgr2=0x7fd1280b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 --2- 172.21.15.99:0/414937163 >> v2:172.21.15.99:3300/0 conn(0x7fd128162020 0x7fd1280b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.070+0000 7fd12d000700 1 -- 172.21.15.99:0/414937163 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280d1590 con 0x7fd1280cfd70 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.071+0000 7fd12d000700 1 --2- 172.21.15.99:0/414937163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281615a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd12808cca0 tx=0x7fd11c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd6bd98d718de02 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.071+0000 7fd127400700 1 -- 172.21.15.99:0/414937163 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c011830 con 0x7fd1280cfd70 2024-03-29T09:08:37.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.071+0000 7fd127400700 1 -- 172.21.15.99:0/414937163 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c011990 con 0x7fd1280cfd70 2024-03-29T09:08:37.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.071+0000 7fd127400700 1 -- 172.21.15.99:0/414937163 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c011d20 con 0x7fd1280cfd70 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 -- 172.21.15.99:0/414937163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 msgr2=0x7fd1281615a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 --2- 172.21.15.99:0/414937163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281615a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd12808cca0 tx=0x7fd11c009bd0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 -- 172.21.15.99:0/414937163 shutdown_connections 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 --2- 172.21.15.99:0/414937163 >> v2:172.21.15.99:3300/0 conn(0x7fd128162020 0x7fd1280b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 --2- 172.21.15.99:0/414937163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281615a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 --2- 172.21.15.99:0/414937163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd128161ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 -- 172.21.15.99:0/414937163 >> 172.21.15.99:0/414937163 conn(0x7fd12809c4a0 msgr2=0x7fd1280d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 -- 172.21.15.99:0/414937163 shutdown_connections 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.072+0000 7fd12fa00700 1 -- 172.21.15.99:0/414937163 wait complete. 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.074+0000 7fd12fa00700 1 Processor -- start 2024-03-29T09:08:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.074+0000 7fd12fa00700 1 -- start start 2024-03-29T09:08:37.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12fa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:37.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12fa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd1280cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12fa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd128162020 0x7fd128159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12fa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280b7340 con 0x7fd128162020 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12fa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd128155b10 con 0x7fd1280cfd70 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12fa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280b66c0 con 0x7fd1280d0150 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:56206/0 (socket says 172.21.15.99:56206) 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 -- 172.21.15.99:0/393536688 learned_addr learned my addr 172.21.15.99:0/393536688 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd127e00700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd1280cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12da00700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd128162020 0x7fd128159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 -- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 msgr2=0x7fd1280cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd1280cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 -- 172.21.15.99:0/393536688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd128162020 msgr2=0x7fd128159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd128162020 0x7fd128159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.075+0000 7fd12d000700 1 -- 172.21.15.99:0/393536688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c009550 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd12d000700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281551c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd11c000c00 tx=0x7fd11c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd124c00700 1 -- 172.21.15.99:0/393536688 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c004240 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd124c00700 1 -- 172.21.15.99:0/393536688 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c01f070 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd124c00700 1 -- 172.21.15.99:0/393536688 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0036a0 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1280b69a0 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1280b6f60 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd124c00700 1 -- 172.21.15.99:0/393536688 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd11c021030 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.076+0000 7fd124c00700 1 -- 172.21.15.99:0/393536688 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fd11c01ada0 con 0x7fd1280cfd70 2024-03-29T09:08:37.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.077+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1280677a0 con 0x7fd1280cfd70 2024-03-29T09:08:37.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.080+0000 7fd124c00700 1 -- 172.21.15.99:0/393536688 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fd11c015020 con 0x7fd1280cfd70 2024-03-29T09:08:37.129 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:37.129+0000 7f7c7e72d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T09:08:37.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.161+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1280cf840 con 0x7fd1280cfd70 2024-03-29T09:08:37.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.162+0000 7fd124c00700 1 -- 172.21.15.99:0/393536688 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd11c003c90 con 0x7fd1280cfd70 2024-03-29T09:08:37.162 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:08:37.162 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":10,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:08:36.744811+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:08:37.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 msgr2=0x7fd1281551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:37.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281551c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd11c000c00 tx=0x7fd11c004750 comp rx=0 tx=0).stop 2024-03-29T09:08:37.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 shutdown_connections 2024-03-29T09:08:37.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd128162020 0x7fd128159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1280cfd70 0x7fd1281551c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 --2- 172.21.15.99:0/393536688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1280d0150 0x7fd1280cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:37.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 >> 172.21.15.99:0/393536688 conn(0x7fd12809c4a0 msgr2=0x7fd1280b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:37.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 shutdown_connections 2024-03-29T09:08:37.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:37.163+0000 7fd12fa00700 1 -- 172.21.15.99:0/393536688 wait complete. 2024-03-29T09:08:37.181 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-29T09:08:37.181 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-29T09:08:37.237 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:37.237+0000 7f7c7e72d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T09:08:37.373 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:37.373+0000 7fd4e00ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T09:08:37.399 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:37.400+0000 7f7c7e72d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T09:08:37.477 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:37.478+0000 7fd4e00ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T09:08:37.480 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-03-29T09:08:37.480+0000 7fd61c517640 -1 Falling back to public interface 2024-03-29T09:08:37.483 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-03-29T09:08:37.484+0000 7fe877a4d640 -1 Falling back to public interface 2024-03-29T09:08:37.492 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-03-29T09:08:37.493+0000 7f5075305640 -1 Falling back to public interface 2024-03-29T09:08:37.496 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-03-29T09:08:37.496+0000 7f7175405640 -1 Falling back to public interface 2024-03-29T09:08:37.499 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-03-29T09:08:37.500+0000 7fe679cdc640 -1 Falling back to public interface 2024-03-29T09:08:37.510 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-03-29T09:08:37.510+0000 7fe03e25c640 -1 Falling back to public interface 2024-03-29T09:08:37.513 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-03-29T09:08:37.514+0000 7f45e7777640 -1 Falling back to public interface 2024-03-29T09:08:37.519 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-03-29T09:08:37.520+0000 7fc8a9b37640 -1 Falling back to public interface 2024-03-29T09:08:37.652 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:37.652+0000 7fd4e00ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T09:08:37.986 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T09:08:38.096 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:38.096+0000 7f7c7e72d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T09:08:38.156 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.156+0000 7fb3ac800700 1 Processor -- start 2024-03-29T09:08:38.156 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.156+0000 7fb3ac800700 1 -- start start 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.158+0000 7fb3ac800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a415d3f0 0x7fb3a40d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.158+0000 7fb3ac800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3a40b2050 0x7fb3a40d1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.158+0000 7fb3ac800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3a40d1c70 0x7fb3a40cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.158+0000 7fb3ac800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a40d40b0 con 0x7fb3a40b2050 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.158+0000 7fb3ac800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a40d2100 con 0x7fb3a40d1c70 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.158+0000 7fb3ac800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40afd80 con 0x7fb3a415d3f0 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3a40b2050 0x7fb3a40d1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a415d3f0 0x7fb3a40d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3a40b2050 0x7fb3a40d1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40554/0 (socket says 172.21.15.99:40554) 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3aa800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3a40d1c70 0x7fb3a40cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:38.160 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 -- 172.21.15.99:0/787809445 learned_addr learned my addr 172.21.15.99:0/787809445 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 -- 172.21.15.99:0/787809445 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a415d3f0 msgr2=0x7fb3a40d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 --2- 172.21.15.99:0/787809445 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a415d3f0 0x7fb3a40d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 -- 172.21.15.99:0/787809445 >> v2:172.21.15.133:3300/0 conn(0x7fb3a40d1c70 msgr2=0x7fb3a40cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 --2- 172.21.15.99:0/787809445 >> v2:172.21.15.133:3300/0 conn(0x7fb3a40d1c70 0x7fb3a40cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 -- 172.21.15.99:0/787809445 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40ce910 con 0x7fb3a40b2050 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.159+0000 7fb3a9400700 1 --2- 172.21.15.99:0/787809445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a40d1730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b24d0 tx=0x7fb394009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d7c6fe3b341920 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3a8a00700 1 -- 172.21.15.99:0/787809445 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394011830 con 0x7fb3a40b2050 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3a8a00700 1 -- 172.21.15.99:0/787809445 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394011990 con 0x7fb3a40b2050 2024-03-29T09:08:38.161 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3a8a00700 1 -- 172.21.15.99:0/787809445 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394011d20 con 0x7fb3a40b2050 2024-03-29T09:08:38.162 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 -- 172.21.15.99:0/787809445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 msgr2=0x7fb3a40d1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:38.162 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 --2- 172.21.15.99:0/787809445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a40d1730 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b24d0 tx=0x7fb394009c00 comp rx=0 tx=0).stop 2024-03-29T09:08:38.162 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 -- 172.21.15.99:0/787809445 shutdown_connections 2024-03-29T09:08:38.162 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 --2- 172.21.15.99:0/787809445 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a415d3f0 0x7fb3a40d3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.163 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 --2- 172.21.15.99:0/787809445 >> v2:172.21.15.133:3300/0 conn(0x7fb3a40d1c70 0x7fb3a40cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.163 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 --2- 172.21.15.99:0/787809445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a40d1730 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.163 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 -- 172.21.15.99:0/787809445 >> 172.21.15.99:0/787809445 conn(0x7fb3a409c490 msgr2=0x7fb3a40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:38.163 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.160+0000 7fb3ac800700 1 -- 172.21.15.99:0/787809445 shutdown_connections 2024-03-29T09:08:38.163 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.161+0000 7fb3ac800700 1 -- 172.21.15.99:0/787809445 wait complete. 2024-03-29T09:08:38.163 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.162+0000 7fb3ac800700 1 Processor -- start 2024-03-29T09:08:38.163 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.162+0000 7fb3ac800700 1 -- start start 2024-03-29T09:08:38.165 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.163+0000 7fb3ac800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:38.165 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.163+0000 7fb3ac800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a40d1c70 0x7fb3a41f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:38.165 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.163+0000 7fb3ac800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3a415d3f0 0x7fb3a415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:38.165 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.163+0000 7fb3ac800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40b2500 con 0x7fb3a40b2050 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.163+0000 7fb3ac800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a41f7360 con 0x7fb3a415d3f0 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.163+0000 7fb3ac800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a40b4b30 con 0x7fb3a40d1c70 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.163+0000 7fb3a9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a4159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40556/0 (socket says 172.21.15.99:40556) 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 -- 172.21.15.99:0/3639224841 learned_addr learned my addr 172.21.15.99:0/3639224841 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9400700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a40d1c70 0x7fb3a41f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3aa800700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3a415d3f0 0x7fb3a415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 -- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a40d1c70 msgr2=0x7fb3a41f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a40d1c70 0x7fb3a41f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 -- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3a415d3f0 msgr2=0x7fb3a415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3a415d3f0 0x7fb3a415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 -- 172.21.15.99:0/3639224841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb394009550 con 0x7fb3a40b2050 2024-03-29T09:08:38.166 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3a9e00700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a4159f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e870 tx=0x7fb39800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb39e000700 1 -- 172.21.15.99:0/3639224841 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398009590 con 0x7fb3a40b2050 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb39e000700 1 -- 172.21.15.99:0/3639224841 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398013680 con 0x7fb3a40b2050 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb39e000700 1 -- 172.21.15.99:0/3639224841 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3980139f0 con 0x7fb3a40b2050 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.164+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a40b4dd0 con 0x7fb3a40b2050 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.165+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a40b5390 con 0x7fb3a40b2050 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.165+0000 7fb39e000700 1 -- 172.21.15.99:0/3639224841 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb398005a20 con 0x7fb3a40b2050 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.165+0000 7fb39e000700 1 -- 172.21.15.99:0/3639224841 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb398005cd0 con 0x7fb3a40b2050 2024-03-29T09:08:38.167 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.166+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a40b2a80 con 0x7fb3a40b2050 2024-03-29T09:08:38.169 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.170+0000 7fb39e000700 1 -- 172.21.15.99:0/3639224841 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fb398021060 con 0x7fb3a40b2050 2024-03-29T09:08:38.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.244+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3a40b5520 con 0x7fb3a40b2050 2024-03-29T09:08:38.245 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.245+0000 7fb39e000700 1 -- 172.21.15.99:0/3639224841 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb398029020 con 0x7fb3a40b2050 2024-03-29T09:08:38.245 INFO:teuthology.misc.health.smithi099.stdout: 2024-03-29T09:08:38.245 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":10,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:08:36.744811+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 msgr2=0x7fb3a4159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a4159f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb39800e870 tx=0x7fb39800eb80 comp rx=0 tx=0).stop 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 shutdown_connections 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3a415d3f0 0x7fb3a415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3a40b2050 0x7fb3a4159f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 --2- 172.21.15.99:0/3639224841 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3a40d1c70 0x7fb3a41f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 >> 172.21.15.99:0/3639224841 conn(0x7fb3a409c490 msgr2=0x7fb3a40d00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 shutdown_connections 2024-03-29T09:08:38.246 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:38.246+0000 7fb3ac800700 1 -- 172.21.15.99:0/3639224841 wait complete. 2024-03-29T09:08:38.260 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T09:08:38.316 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:38.316+0000 7f7c7e72d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T09:08:38.355 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:38.356+0000 7fd4e00ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-29T09:08:38.527 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:38.527+0000 7f7c7e72d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T09:08:38.572 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:38.573+0000 7fd4e00ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T09:08:38.788 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:38.789+0000 7fd4e00ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T09:08:39.519 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:39.519+0000 7f7c7e72d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T09:08:39.792 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:39.792+0000 7fd4e00ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T09:08:39.818 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:39.819+0000 7f7c7e72d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T09:08:39.914 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:39.914+0000 7f7c7e72d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T09:08:40.117 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:40.117+0000 7fd4e00ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T09:08:40.213 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:40.213+0000 7fd4e00ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T09:08:40.424 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:40.424+0000 7f7c7e72d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T09:08:40.756 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-03-29T09:08:40.755+0000 7fd4e00ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T09:08:41.058 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-03-29T09:08:41.058+0000 7f5075305640 -1 osd.6 0 log_to_monitors true 2024-03-29T09:08:41.059 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-03-29T09:08:41.059+0000 7fc8a9b37640 -1 osd.7 0 log_to_monitors true 2024-03-29T09:08:41.086 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-03-29T09:08:41.086+0000 7fe03e25c640 -1 osd.0 0 log_to_monitors true 2024-03-29T09:08:41.088 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-03-29T09:08:41.088+0000 7fe679cdc640 -1 osd.2 0 log_to_monitors true 2024-03-29T09:08:41.238 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-03-29T09:08:41.237+0000 7fe877a4d640 -1 osd.4 0 log_to_monitors true 2024-03-29T09:08:41.252 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-03-29T09:08:41.252+0000 7fd61c517640 -1 osd.3 0 log_to_monitors true 2024-03-29T09:08:41.295 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-03-29T09:08:41.295+0000 7f45e7777640 -1 osd.5 0 log_to_monitors true 2024-03-29T09:08:41.316 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-03-29T09:08:41.317+0000 7f7175405640 -1 osd.1 0 log_to_monitors true 2024-03-29T09:08:41.446 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:41.446+0000 7f7c30400700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-29T09:08:41.446 INFO:tasks.ceph.mgr.y.smithi099.stderr:2024-03-29T09:08:41.447+0000 7f7c30400700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-29T09:08:44.858 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-03-29T09:08:44.857+0000 7f45d5400700 -1 osd.5 0 waiting for initial osdmap 2024-03-29T09:08:44.868 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-03-29T09:08:44.866+0000 7f45cac00700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:44.873 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-03-29T09:08:44.874+0000 7fd60a200700 -1 osd.3 0 waiting for initial osdmap 2024-03-29T09:08:44.890 INFO:tasks.ceph.osd.3.smithi099.stderr:2024-03-29T09:08:44.891+0000 7fd5ffa00700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:44.909 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-03-29T09:08:44.908+0000 7f5063000700 -1 osd.6 0 waiting for initial osdmap 2024-03-29T09:08:44.917 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-03-29T09:08:44.917+0000 7f5058800700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:44.936 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-03-29T09:08:44.937+0000 7fe667a00700 -1 osd.2 0 waiting for initial osdmap 2024-03-29T09:08:44.944 INFO:tasks.ceph.osd.2.smithi099.stderr:2024-03-29T09:08:44.945+0000 7fe65d200700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:44.977 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-03-29T09:08:44.978+0000 7f7163000700 -1 osd.1 0 waiting for initial osdmap 2024-03-29T09:08:44.983 INFO:tasks.ceph.osd.1.smithi099.stderr:2024-03-29T09:08:44.985+0000 7f7158800700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:44.985 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-03-29T09:08:44.985+0000 7fe865800700 -1 osd.4 0 waiting for initial osdmap 2024-03-29T09:08:44.991 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-03-29T09:08:44.991+0000 7fe85b000700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:44.992 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-03-29T09:08:44.991+0000 7fc897800700 -1 osd.7 0 waiting for initial osdmap 2024-03-29T09:08:45.007 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-03-29T09:08:45.007+0000 7fc88d000700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:45.017 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-03-29T09:08:45.018+0000 7fe02c000700 -1 osd.0 0 waiting for initial osdmap 2024-03-29T09:08:45.027 INFO:tasks.ceph.osd.0.smithi099.stderr:2024-03-29T09:08:45.028+0000 7fe021800700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T09:08:45.067 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T09:08:45.235 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.236+0000 7f3bf9000700 1 Processor -- start 2024-03-29T09:08:45.235 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.236+0000 7f3bf9000700 1 -- start start 2024-03-29T09:08:45.240 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.237+0000 7f3bf9000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bf40b3e10 0x7f3bf415cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:45.241 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.237+0000 7f3bf9000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3bf40b2c50 0x7f3bf40b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:45.241 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.237+0000 7f3bf9000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40cce10 0x7f3bf40cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:45.241 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.237+0000 7f3bf9000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf40706c0 con 0x7f3bf40b2c50 2024-03-29T09:08:45.241 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.237+0000 7f3bf9000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf40709a0 con 0x7f3bf40b3e10 2024-03-29T09:08:45.241 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.237+0000 7f3bf9000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4070c80 con 0x7f3bf40cce10 2024-03-29T09:08:45.241 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.237+0000 7f3bf1400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3bf40b2c50 0x7f3bf40b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:45.242 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bf40b3e10 0x7f3bf415cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:45.242 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf2800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40cce10 0x7f3bf40cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:45.242 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3bf40b2c50 0x7f3bf40b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40660/0 (socket says 172.21.15.99:40660) 2024-03-29T09:08:45.242 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3bf40b3e10 0x7f3bf415cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:51618/0 (socket says 172.21.15.99:51618) 2024-03-29T09:08:45.242 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 -- 172.21.15.99:0/1387122442 learned_addr learned my addr 172.21.15.99:0/1387122442 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:45.243 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 -- 172.21.15.99:0/1387122442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40cce10 msgr2=0x7f3bf40cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:45.243 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 --2- 172.21.15.99:0/1387122442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40cce10 0x7f3bf40cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.243 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 -- 172.21.15.99:0/1387122442 >> v2:172.21.15.133:3300/0 conn(0x7f3bf40b3e10 msgr2=0x7f3bf415cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:45.243 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 --2- 172.21.15.99:0/1387122442 >> v2:172.21.15.133:3300/0 conn(0x7f3bf40b3e10 0x7f3bf415cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.243 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 -- 172.21.15.99:0/1387122442 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf4070f60 con 0x7f3bf40b2c50 2024-03-29T09:08:45.243 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf1400700 1 --2- 172.21.15.99:0/1387122442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf40b0c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3bf406e770 tx=0x7f3be0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5db729b3d182fb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf0a00700 1 -- 172.21.15.99:0/1387122442 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0011830 con 0x7f3bf40b2c50 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf0a00700 1 -- 172.21.15.99:0/1387122442 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0011990 con 0x7f3bf40b2c50 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.238+0000 7f3bf0a00700 1 -- 172.21.15.99:0/1387122442 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0011d20 con 0x7f3bf40b2c50 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 -- 172.21.15.99:0/1387122442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 msgr2=0x7f3bf40b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 --2- 172.21.15.99:0/1387122442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf40b0c30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3bf406e770 tx=0x7f3be0009bd0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 -- 172.21.15.99:0/1387122442 shutdown_connections 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 --2- 172.21.15.99:0/1387122442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40cce10 0x7f3bf40cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 --2- 172.21.15.99:0/1387122442 >> v2:172.21.15.133:3300/0 conn(0x7f3bf40b3e10 0x7f3bf415cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 --2- 172.21.15.99:0/1387122442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf40b0c30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.244 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 -- 172.21.15.99:0/1387122442 >> 172.21.15.99:0/1387122442 conn(0x7f3bf4066540 msgr2=0x7f3bf4066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:45.245 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 -- 172.21.15.99:0/1387122442 shutdown_connections 2024-03-29T09:08:45.245 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.239+0000 7f3bf9000700 1 -- 172.21.15.99:0/1387122442 wait complete. 2024-03-29T09:08:45.245 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.241+0000 7f3bf9000700 1 Processor -- start 2024-03-29T09:08:45.245 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.241+0000 7f3bf9000700 1 -- start start 2024-03-29T09:08:45.247 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf9000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40b3e10 0x7f3bf4153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf9000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf40cce10 0x7f3bf414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf9000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4153b70 con 0x7f3bf40b2c50 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf9000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4153d80 con 0x7f3bf40cce10 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf9000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4154060 con 0x7f3bf40b3e10 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf415b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40668/0 (socket says 172.21.15.99:40668) 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.242+0000 7f3bf1e00700 1 -- 172.21.15.99:0/3005860239 learned_addr learned my addr 172.21.15.99:0/3005860239 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf1400700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40b3e10 0x7f3bf4153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf1e00700 1 -- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40b3e10 msgr2=0x7f3bf4153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf2800700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf40cce10 0x7f3bf414d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf1e00700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40b3e10 0x7f3bf4153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.248 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf1e00700 1 -- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf40cce10 msgr2=0x7f3bf414d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf1e00700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf40cce10 0x7f3bf414d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf1e00700 1 -- 172.21.15.99:0/3005860239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0009550 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf1e00700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf415b170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3be400e930 tx=0x7f3be400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bea000700 1 -- 172.21.15.99:0/3005860239 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4009690 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bea000700 1 -- 172.21.15.99:0/3005860239 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400c850 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf4154340 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bea000700 1 -- 172.21.15.99:0/3005860239 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400cbe0 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.243+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf414de20 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.244+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf40aca70 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.245+0000 7f3bea000700 1 -- 172.21.15.99:0/3005860239 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 4) v1 ==== 89081+0+0 (secure 0 0 0) 0x7f3be400cd40 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.246+0000 7f3bea000700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bd406ac00 0x7f3bd406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.246+0000 7f3bea000700 1 -- 172.21.15.99:0/3005860239 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f3be4088870 con 0x7f3bf40b2c50 2024-03-29T09:08:45.249 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.246+0000 7f3bf1400700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bd406ac00 0x7f3bd406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:45.250 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.247+0000 7f3bf1400700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bd406ac00 0x7f3bd406d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4066a30 tx=0x7f3be0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:45.250 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.251+0000 7f3bea000700 1 -- 172.21.15.99:0/3005860239 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3be4054950 con 0x7f3bf40b2c50 2024-03-29T09:08:45.455 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.456+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf40677a0 con 0x7f3bf40b2c50 2024-03-29T09:08:45.456 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.457+0000 7f3bea000700 1 -- 172.21.15.99:0/3005860239 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f3be4021090 con 0x7f3bf40b2c50 2024-03-29T09:08:45.456 INFO:teuthology.misc.health.smithi099.stdout: 2024-03-29T09:08:45.456 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":12,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:08:42.511975+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bd406ac00 msgr2=0x7f3bd406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bd406ac00 0x7f3bd406d0c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4066a30 tx=0x7f3be0008030 comp rx=0 tx=0).stop 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 msgr2=0x7f3bf415b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf415b170 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3be400e930 tx=0x7f3be400ecf0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 shutdown_connections 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bd406ac00 0x7f3bd406d0c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bf40cce10 0x7f3bf414d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3bf40b2c50 0x7f3bf415b170 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.459 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 --2- 172.21.15.99:0/3005860239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bf40b3e10 0x7f3bf4153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:45.460 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 >> 172.21.15.99:0/3005860239 conn(0x7f3bf4066540 msgr2=0x7f3bf40afcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:45.460 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 shutdown_connections 2024-03-29T09:08:45.460 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:45.460+0000 7f3bf9000700 1 -- 172.21.15.99:0/3005860239 wait complete. 2024-03-29T09:08:45.473 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T09:08:52.281 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T09:08:52.412 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:08:52.416 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:08:52.461 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.462+0000 7f7a42c00700 1 Processor -- start 2024-03-29T09:08:52.461 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.462+0000 7f7a42c00700 1 -- start start 2024-03-29T09:08:52.463 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.464+0000 7f7a42c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7a3c06a7d0 0x7f7a3c0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.463 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.464+0000 7f7a42c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c072620 con 0x7f7a3c15d9a0 2024-03-29T09:08:52.463 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.464+0000 7f7a42c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c15e990 con 0x7f7a3c06ea10 2024-03-29T09:08:52.463 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.464+0000 7f7a42c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c0cfe00 con 0x7f7a3c06a7d0 2024-03-29T09:08:52.465 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.464+0000 7f7a3b400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7a3c15d9a0 0x7f7a3c06abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36848/0 (socket says 172.21.15.99:36848) 2024-03-29T09:08:52.465 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.464+0000 7f7a3b400700 1 -- 172.21.15.99:0/670889185 learned_addr learned my addr 172.21.15.99:0/670889185 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:52.465 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.464+0000 7f7a3be00700 1 --2- 172.21.15.99:0/670889185 >> v2:172.21.15.133:3300/0 conn(0x7f7a3c06a7d0 0x7f7a3c0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859598407 0 0) 0x7f7a28001110 con 0x7f7a3c15d9a0 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a200035b0 con 0x7f7a3c15d9a0 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584453447 0 0) 0x7f7a3c15e990 con 0x7f7a3c06ea10 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a28001110 con 0x7f7a3c06ea10 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465683221 0 0) 0x7f7a200035b0 con 0x7f7a3c15d9a0 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 >> v2:172.21.15.133:3300/0 conn(0x7f7a3c06a7d0 msgr2=0x7f7a3c0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 --2- 172.21.15.99:0/670889185 >> v2:172.21.15.133:3300/0 conn(0x7f7a3c06a7d0 0x7f7a3c0b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 >> v1:172.21.15.133:6789/0 conn(0x7f7a3c06ea10 legacy=0x7f7a3c0d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c15de30 con 0x7f7a3c15d9a0 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f7a28001800 con 0x7f7a3c15d9a0 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 >> v1:172.21.15.99:6789/0 conn(0x7f7a3c15d9a0 legacy=0x7f7a3c06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20003b90 0x7f7a20003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.466 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20004850 0x7f7a20004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a20005210 0x7f7a2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0cfe00 con 0x7f7a20003b90 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2000c140 con 0x7f7a20004850 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2000c300 con 0x7f7a20005210 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3be00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20003b90 0x7f7a20003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a3b400700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20004850 0x7f7a20004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.465+0000 7f7a40c00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a20005210 0x7f7a2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3be00700 1 -- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a20005210 msgr2=0x7f7a2000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3be00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a20005210 0x7f7a2000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3be00700 1 -- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20004850 msgr2=0x7f7a20004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3be00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20004850 0x7f7a20004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3be00700 1 -- 172.21.15.99:0/670889185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a200035b0 con 0x7f7a20003b90 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3b400700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20004850 0x7f7a20004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:08:52.467 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a40c00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a20005210 0x7f7a2000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:08:52.468 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a40c00700 1 -- 172.21.15.99:0/670889185 reap_dead start 2024-03-29T09:08:52.468 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3be00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20003b90 0x7f7a20003f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009b60 tx=0x7f7a20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc551bf07e1c3d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:52.468 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00e6c0 con 0x7f7a20003b90 2024-03-29T09:08:52.468 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c00e820 con 0x7f7a20003b90 2024-03-29T09:08:52.468 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.466+0000 7f7a3aa00700 1 -- 172.21.15.99:0/670889185 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00eb90 con 0x7f7a20003b90 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.467+0000 7f7a42c00700 1 -- 172.21.15.99:0/670889185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20003b90 msgr2=0x7f7a20003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.467+0000 7f7a42c00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20003b90 0x7f7a20003f90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009b60 tx=0x7f7a20005880 comp rx=0 tx=0).stop 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.467+0000 7f7a42c00700 1 -- 172.21.15.99:0/670889185 shutdown_connections 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.467+0000 7f7a42c00700 1 --2- 172.21.15.99:0/670889185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a20003b90 0x7f7a20003f90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.467+0000 7f7a42c00700 1 -- 172.21.15.99:0/670889185 >> 172.21.15.99:0/670889185 conn(0x7f7a3c066df0 msgr2=0x7f7a3c06d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.467+0000 7f7a42c00700 1 -- 172.21.15.99:0/670889185 shutdown_connections 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.467+0000 7f7a42c00700 1 -- 172.21.15.99:0/670889185 wait complete. 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.469+0000 7f7a42c00700 1 Processor -- start 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.469+0000 7f7a42c00700 1 -- start start 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a42c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20003b90 0x7f7a3c0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a42c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a3c144510 0x7f7a3c15cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a42c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3c0c62d0 0x7f7a3c15d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.471 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a42c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0b03a0 con 0x7f7a3c144510 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a42c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0cbd70 con 0x7f7a20003b90 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a42c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0c6890 con 0x7f7a3c0c62d0 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a3be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20003b90 0x7f7a3c0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a3b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a3c144510 0x7f7a3c15cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a3be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20003b90 0x7f7a3c0c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:39424/0 (socket says 172.21.15.99:39424) 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a3be00700 1 -- 172.21.15.99:0/548588282 learned_addr learned my addr 172.21.15.99:0/548588282 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.470+0000 7f7a40c00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3c0c62d0 0x7f7a3c15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a3be00700 1 -- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3c0c62d0 msgr2=0x7f7a3c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a3be00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3c0c62d0 0x7f7a3c15d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a3be00700 1 -- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a3c144510 msgr2=0x7f7a3c15cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a3be00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a3c144510 0x7f7a3c15cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a3be00700 1 -- 172.21.15.99:0/548588282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c0afce0 con 0x7f7a20003b90 2024-03-29T09:08:52.472 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a3be00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20003b90 0x7f7a3c0c5d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c110 tx=0x7f7a2c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:52.473 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a33e00700 1 -- 172.21.15.99:0/548588282 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00e6c0 con 0x7f7a20003b90 2024-03-29T09:08:52.473 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a33e00700 1 -- 172.21.15.99:0/548588282 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c0038c0 con 0x7f7a20003b90 2024-03-29T09:08:52.473 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a33e00700 1 -- 172.21.15.99:0/548588282 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c003c70 con 0x7f7a20003b90 2024-03-29T09:08:52.473 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.471+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c0cbd70 con 0x7f7a20003b90 2024-03-29T09:08:52.473 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.472+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c0c6b70 con 0x7f7a20003b90 2024-03-29T09:08:52.474 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.473+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c0b0610 con 0x7f7a20003b90 2024-03-29T09:08:52.474 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.474+0000 7f7a33e00700 1 -- 172.21.15.99:0/548588282 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a2c013470 con 0x7f7a20003b90 2024-03-29T09:08:52.474 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.475+0000 7f7a33e00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a1c06ac90 0x7f7a1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.474 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.475+0000 7f7a33e00700 1 -- 172.21.15.99:0/548588282 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7a2c08a7d0 con 0x7f7a20003b90 2024-03-29T09:08:52.474 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.475+0000 7f7a3b400700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a1c06ac90 0x7f7a1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.474 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.475+0000 7f7a3b400700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a1c06ac90 0x7f7a1c06d150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c1f5f40 tx=0x7f7a28007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:52.482 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.480+0000 7f7a33e00700 1 -- 172.21.15.99:0/548588282 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a2c055d90 con 0x7f7a20003b90 2024-03-29T09:08:52.689 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.690+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a3c070030 con 0x7f7a20003b90 2024-03-29T09:08:52.690 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.691+0000 7f7a33e00700 1 -- 172.21.15.99:0/548588282 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f7a2c017090 con 0x7f7a20003b90 2024-03-29T09:08:52.690 INFO:teuthology.misc.health.smithi099.stdout: 2024-03-29T09:08:52.690 INFO:teuthology.misc.health.smithi099.stdout:{"epoch":16,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:08:48.895850+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a1c06ac90 msgr2=0x7f7a1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a1c06ac90 0x7f7a1c06d150 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c1f5f40 tx=0x7f7a28007040 comp rx=0 tx=0).stop 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20003b90 msgr2=0x7f7a3c0c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20003b90 0x7f7a3c0c5d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c110 tx=0x7f7a2c0047c0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 shutdown_connections 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a1c06ac90 0x7f7a1c06d150 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3c0c62d0 0x7f7a3c15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a20003b90 0x7f7a3c0c5d90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 --2- 172.21.15.99:0/548588282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a3c144510 0x7f7a3c15cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.693+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 >> 172.21.15.99:0/548588282 conn(0x7f7a3c066df0 msgr2=0x7f7a3c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.694+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 shutdown_connections 2024-03-29T09:08:52.693 INFO:teuthology.misc.health.smithi099.stderr:2024-03-29T09:08:52.694+0000 7f7a42c00700 1 -- 172.21.15.99:0/548588282 wait complete. 2024-03-29T09:08:52.707 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-29T09:08:52.708 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-29T09:08:52.708 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T09:08:52.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T09:08:52.712 INFO:tasks.ceph.mds.a:Started 2024-03-29T09:08:52.712 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T09:08:52.713 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T09:08:52.716 INFO:tasks.ceph.mds.c:Started 2024-03-29T09:08:52.716 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T09:08:52.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T09:08:52.720 INFO:tasks.ceph.mds.b:Started 2024-03-29T09:08:52.720 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T09:08:52.721 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T09:08:52.724 INFO:tasks.ceph.mds.d:Started 2024-03-29T09:08:52.725 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-29T09:08:52.725 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:08:52.854 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-03-29T09:08:52.854 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-03-29T09:08:52.868 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-03-29T09:08:52.870 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-03-29T09:08:52.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.926+0000 7fa4a0c00700 1 Processor -- start 2024-03-29T09:08:52.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.926+0000 7fa4a0c00700 1 -- start start 2024-03-29T09:08:52.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.927+0000 7fa4a0c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c145d70 0x7fa49c161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.927+0000 7fa4a0c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa49c146150 0x7fa49c0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.927+0000 7fa4a0c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa49c0b3480 0x7fa49c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.927+0000 7fa4a0c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c0b3990 con 0x7fa49c0b3480 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.927+0000 7fa4a0c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c0b3c70 con 0x7fa49c146150 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.927+0000 7fa4a0c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c160f40 con 0x7fa49c145d70 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa49c0b3480 0x7fa49c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa49c0b3480 0x7fa49c06a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38676/0 (socket says 172.21.15.99:38676) 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 -- 172.21.15.99:0/1197924209 learned_addr learned my addr 172.21.15.99:0/1197924209 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa499e00700 1 --2- 172.21.15.99:0/1197924209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c145d70 0x7fa49c161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa499400700 1 --2- 172.21.15.99:0/1197924209 >> v2:172.21.15.133:3300/0 conn(0x7fa49c146150 0x7fa49c0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 -- 172.21.15.99:0/1197924209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c145d70 msgr2=0x7fa49c161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 --2- 172.21.15.99:0/1197924209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c145d70 0x7fa49c161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 -- 172.21.15.99:0/1197924209 >> v2:172.21.15.133:3300/0 conn(0x7fa49c146150 msgr2=0x7fa49c0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 --2- 172.21.15.99:0/1197924209 >> v2:172.21.15.133:3300/0 conn(0x7fa49c146150 0x7fa49c0b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 -- 172.21.15.99:0/1197924209 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c161220 con 0x7fa49c0b3480 2024-03-29T09:08:52.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.928+0000 7fa49a800700 1 --2- 172.21.15.99:0/1197924209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c06a7d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d0fd0 tx=0x7fa49400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5b01ae642b78bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:52.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa498a00700 1 -- 172.21.15.99:0/1197924209 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400e070 con 0x7fa49c0b3480 2024-03-29T09:08:52.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa498a00700 1 -- 172.21.15.99:0/1197924209 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49400b9c0 con 0x7fa49c0b3480 2024-03-29T09:08:52.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa498a00700 1 -- 172.21.15.99:0/1197924209 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400bd70 con 0x7fa49c0b3480 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 -- 172.21.15.99:0/1197924209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 msgr2=0x7fa49c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/1197924209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c06a7d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d0fd0 tx=0x7fa49400b330 comp rx=0 tx=0).stop 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 -- 172.21.15.99:0/1197924209 shutdown_connections 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/1197924209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c145d70 0x7fa49c161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/1197924209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c06a7d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/1197924209 >> v2:172.21.15.133:3300/0 conn(0x7fa49c146150 0x7fa49c0b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 -- 172.21.15.99:0/1197924209 >> 172.21.15.99:0/1197924209 conn(0x7fa49c066df0 msgr2=0x7fa49c06d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 -- 172.21.15.99:0/1197924209 shutdown_connections 2024-03-29T09:08:52.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.929+0000 7fa4a0c00700 1 -- 172.21.15.99:0/1197924209 wait complete. 2024-03-29T09:08:52.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.932+0000 7fa4a0c00700 1 Processor -- start 2024-03-29T09:08:52.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.932+0000 7fa4a0c00700 1 -- start start 2024-03-29T09:08:52.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa4a0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa4a0c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa49c145d70 0x7fa49c0add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa4a0c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c0b1da0 0x7fa49c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa4a0c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0cdd70 con 0x7fa49c0b3480 2024-03-29T09:08:52.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa4a0c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c1f6df0 con 0x7fa49c145d70 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa4a0c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0cd440 con 0x7fa49c0b1da0 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa499e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa499e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c0b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38680/0 (socket says 172.21.15.99:38680) 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa499e00700 1 -- 172.21.15.99:0/704805882 learned_addr learned my addr 172.21.15.99:0/704805882 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa499400700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa49c145d70 0x7fa49c0add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.933+0000 7fa49a800700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c0b1da0 0x7fa49c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa499e00700 1 -- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c0b1da0 msgr2=0x7fa49c0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa499e00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c0b1da0 0x7fa49c0b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa499e00700 1 -- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa49c145d70 msgr2=0x7fa49c0add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa499e00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa49c145d70 0x7fa49c0add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa499e00700 1 -- 172.21.15.99:0/704805882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa494009b30 con 0x7fa49c0b3480 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa499e00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c0b05a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00e810 tx=0x7fa48c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa492000700 1 -- 172.21.15.99:0/704805882 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00c070 con 0x7fa49c0b3480 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa492000700 1 -- 172.21.15.99:0/704805882 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c013610 con 0x7fa49c0b3480 2024-03-29T09:08:52.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa492000700 1 -- 172.21.15.99:0/704805882 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0139c0 con 0x7fa49c0b3480 2024-03-29T09:08:52.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c0cd720 con 0x7fa49c0b3480 2024-03-29T09:08:52.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.934+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c0b2770 con 0x7fa49c0b3480 2024-03-29T09:08:52.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.935+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c0677a0 con 0x7fa49c0b3480 2024-03-29T09:08:52.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.936+0000 7fa492000700 1 -- 172.21.15.99:0/704805882 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa48c013b20 con 0x7fa49c0b3480 2024-03-29T09:08:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.937+0000 7fa492000700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa47c06ac90 0x7fa47c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.937+0000 7fa492000700 1 -- 172.21.15.99:0/704805882 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa48c088e00 con 0x7fa49c0b3480 2024-03-29T09:08:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.937+0000 7fa499400700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa47c06ac90 0x7fa47c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:52.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.938+0000 7fa499400700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa47c06ac90 0x7fa47c06d150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa49c141d20 tx=0x7fa488008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:52.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:52.942+0000 7fa492000700 1 -- 172.21.15.99:0/704805882 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa48c0542a0 con 0x7fa49c0b3480 2024-03-29T09:08:53.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.146+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa49c0adb00 con 0x7fa49c0b3480 2024-03-29T09:08:53.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.147+0000 7fa492000700 1 -- 172.21.15.99:0/704805882 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fa48c0578c0 con 0x7fa49c0b3480 2024-03-29T09:08:53.146 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:08:53.146 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":16,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:08:48.895850+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa47c06ac90 msgr2=0x7fa47c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa47c06ac90 0x7fa47c06d150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa49c141d20 tx=0x7fa488008040 comp rx=0 tx=0).stop 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 msgr2=0x7fa49c0b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c0b05a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00e810 tx=0x7fa48c00eb20 comp rx=0 tx=0).stop 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 shutdown_connections 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa47c06ac90 0x7fa47c06d150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa49c0b1da0 0x7fa49c0b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa49c0b3480 0x7fa49c0b05a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 --2- 172.21.15.99:0/704805882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa49c145d70 0x7fa49c0add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.149+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 >> 172.21.15.99:0/704805882 conn(0x7fa49c066df0 msgr2=0x7fa49c06d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.150+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 shutdown_connections 2024-03-29T09:08:53.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.150+0000 7fa4a0c00700 1 -- 172.21.15.99:0/704805882 wait complete. 2024-03-29T09:08:53.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:08:53.167 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:08:53.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.361+0000 7fe007200700 1 Processor -- start 2024-03-29T09:08:53.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.361+0000 7fe007200700 1 -- start start 2024-03-29T09:08:53.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.362+0000 7fe007200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe000167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.362+0000 7fe007200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000d1520 con 0x7fe000160770 2024-03-29T09:08:53.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.362+0000 7fe007200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000d1800 con 0x7fe0000b3cc0 2024-03-29T09:08:53.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.362+0000 7fe007200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000b4140 con 0x7fe0000cf090 2024-03-29T09:08:53.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fdfffe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe000160770 0x7fe0000d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36868/0 (socket says 172.21.15.99:36868) 2024-03-29T09:08:53.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fdfffe00700 1 -- 172.21.15.99:0/452609691 learned_addr learned my addr 172.21.15.99:0/452609691 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:53.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fe005200700 1 --2- 172.21.15.99:0/452609691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe000167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fdffea00700 1 -- 172.21.15.99:0/452609691 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589010507 0 0) 0x7fe0000d1520 con 0x7fe000160770 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fdffea00700 1 -- 172.21.15.99:0/452609691 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe40035b0 con 0x7fe000160770 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fdffea00700 1 -- 172.21.15.99:0/452609691 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936074177 0 0) 0x7fe0000d1800 con 0x7fe0000b3cc0 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fdffea00700 1 -- 172.21.15.99:0/452609691 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0000d1520 con 0x7fe0000b3cc0 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fe005200700 1 -- 172.21.15.99:0/452609691 >> v1:172.21.15.133:6789/0 conn(0x7fe0000b3cc0 legacy=0x7fe0000d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fe005200700 1 -- 172.21.15.99:0/452609691 >> v1:172.21.15.99:6789/0 conn(0x7fe000160770 legacy=0x7fe0000d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.363+0000 7fe005200700 1 -- 172.21.15.99:0/452609691 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000d1ae0 con 0x7fe0000cf090 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fe005200700 1 --2- 172.21.15.99:0/452609691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe000167f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009dc0 tx=0x7fdff800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81efa40db03aa53a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fdffea00700 1 -- 172.21.15.99:0/452609691 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800e070 con 0x7fe0000cf090 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fdffea00700 1 -- 172.21.15.99:0/452609691 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff800bb20 con 0x7fe0000cf090 2024-03-29T09:08:53.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fdffea00700 1 -- 172.21.15.99:0/452609691 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8012430 con 0x7fe0000cf090 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fe007200700 1 -- 172.21.15.99:0/452609691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 msgr2=0x7fe000167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fe007200700 1 --2- 172.21.15.99:0/452609691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe000167f40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009dc0 tx=0x7fdff800b330 comp rx=0 tx=0).stop 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fe007200700 1 -- 172.21.15.99:0/452609691 shutdown_connections 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fe007200700 1 --2- 172.21.15.99:0/452609691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe000167f40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fe007200700 1 -- 172.21.15.99:0/452609691 >> 172.21.15.99:0/452609691 conn(0x7fe00008afd0 msgr2=0x7fe00009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.364+0000 7fe007200700 1 -- 172.21.15.99:0/452609691 shutdown_connections 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.365+0000 7fe007200700 1 -- 172.21.15.99:0/452609691 wait complete. 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.366+0000 7fe007200700 1 Processor -- start 2024-03-29T09:08:53.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.366+0000 7fe007200700 1 -- start start 2024-03-29T09:08:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fe007200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0000b3cc0 0x7fe000152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fe007200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe0001f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fe007200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000160770 0x7fe0000d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fe007200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000152e90 con 0x7fe0000b3cc0 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fe007200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0001f6780 con 0x7fe000160770 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fe007200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0001f6a60 con 0x7fe0000cf090 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fdfffe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0000b3cc0 0x7fe000152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fdfffe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0000b3cc0 0x7fe000152950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38682/0 (socket says 172.21.15.99:38682) 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.367+0000 7fdfffe00700 1 -- 172.21.15.99:0/3475966039 learned_addr learned my addr 172.21.15.99:0/3475966039 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdfff400700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe0001f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fe005200700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000160770 0x7fe0000d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdfffe00700 1 -- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 msgr2=0x7fe0001f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdfffe00700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe0001f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdfffe00700 1 -- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000160770 msgr2=0x7fe0000d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdfffe00700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000160770 0x7fe0000d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdfffe00700 1 -- 172.21.15.99:0/3475966039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff8009a70 con 0x7fe0000b3cc0 2024-03-29T09:08:53.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdfffe00700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0000b3cc0 0x7fe000152950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdff0001eb0 tx=0x7fdff000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdff7e00700 1 -- 172.21.15.99:0/3475966039 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0005d70 con 0x7fe0000b3cc0 2024-03-29T09:08:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdff7e00700 1 -- 172.21.15.99:0/3475966039 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff00070d0 con 0x7fe0000b3cc0 2024-03-29T09:08:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0001f6d40 con 0x7fe0000b3cc0 2024-03-29T09:08:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fdff7e00700 1 -- 172.21.15.99:0/3475966039 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0007460 con 0x7fe0000b3cc0 2024-03-29T09:08:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.368+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0001f63c0 con 0x7fe0000b3cc0 2024-03-29T09:08:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.369+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0000678b0 con 0x7fe0000b3cc0 2024-03-29T09:08:53.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.370+0000 7fdff7e00700 1 -- 172.21.15.99:0/3475966039 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdff00075c0 con 0x7fe0000b3cc0 2024-03-29T09:08:53.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.371+0000 7fdff7e00700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdfe006ac90 0x7fdfe006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.371+0000 7fdff7e00700 1 -- 172.21.15.99:0/3475966039 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdff0088320 con 0x7fe0000b3cc0 2024-03-29T09:08:53.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.371+0000 7fdfff400700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdfe006ac90 0x7fdfe006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.372+0000 7fdfff400700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdfe006ac90 0x7fdfe006d150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0000d02d0 tx=0x7fdfec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:53.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.376+0000 7fdff7e00700 1 -- 172.21.15.99:0/3475966039 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdff00538e0 con 0x7fe0000b3cc0 2024-03-29T09:08:53.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.577+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0000d05d0 con 0x7fe0000b3cc0 2024-03-29T09:08:53.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.578+0000 7fdff7e00700 1 -- 172.21.15.99:0/3475966039 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdff0056f00 con 0x7fe0000b3cc0 2024-03-29T09:08:53.577 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:08:53.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdfe006ac90 msgr2=0x7fdfe006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdfe006ac90 0x7fdfe006d150 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0000d02d0 tx=0x7fdfec009040 comp rx=0 tx=0).stop 2024-03-29T09:08:53.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0000b3cc0 msgr2=0x7fe000152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0000b3cc0 0x7fe000152950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdff0001eb0 tx=0x7fdff000c7b0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 shutdown_connections 2024-03-29T09:08:53.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdfe006ac90 0x7fdfe006d150 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe000160770 0x7fe0000d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0000b3cc0 0x7fe000152950 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 --2- 172.21.15.99:0/3475966039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0000cf090 0x7fe0001f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 >> 172.21.15.99:0/3475966039 conn(0x7fe00008afd0 msgr2=0x7fe0000b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:53.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 shutdown_connections 2024-03-29T09:08:53.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.580+0000 7fe007200700 1 -- 172.21.15.99:0/3475966039 wait complete. 2024-03-29T09:08:53.597 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:08:53.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.786+0000 7f17fc400700 1 Processor -- start 2024-03-29T09:08:53.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.786+0000 7f17fc400700 1 -- start start 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.787+0000 7f17fc400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f17f415d330 0x7f17f415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.787+0000 7f17fc400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f40ccf60 con 0x7f17f40cd6b0 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.787+0000 7f17fc400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f4071750 con 0x7f17f4153d20 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.787+0000 7f17fc400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f415dd00 con 0x7f17f415d330 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17fa400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f17f40cd6b0 0x7f17f40cdab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36880/0 (socket says 172.21.15.99:36880) 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17fa400700 1 -- 172.21.15.99:0/3570951094 learned_addr learned my addr 172.21.15.99:0/3570951094 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17f9000700 1 --2- 172.21.15.99:0/3570951094 >> v2:172.21.15.133:3300/0 conn(0x7f17f415d330 0x7f17f415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171968357 0 0) 0x7f17f40ccf60 con 0x7f17f40cd6b0 2024-03-29T09:08:53.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17dc0035b0 con 0x7f17f40cd6b0 2024-03-29T09:08:53.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004202813 0 0) 0x7f17dc0035b0 con 0x7f17f40cd6b0 2024-03-29T09:08:53.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 >> v2:172.21.15.133:3300/0 conn(0x7f17f415d330 msgr2=0x7f17f415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:53.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17f3e00700 1 --2- 172.21.15.99:0/3570951094 >> v2:172.21.15.133:3300/0 conn(0x7f17f415d330 0x7f17f415d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.788+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 >> v1:172.21.15.133:6789/0 conn(0x7f17f4153d20 legacy=0x7f17f4070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f4070c50 con 0x7f17f40cd6b0 2024-03-29T09:08:53.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f17ec0019f0 con 0x7f17f40cd6b0 2024-03-29T09:08:53.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f9000700 1 --2- 172.21.15.99:0/3570951094 >> v2:172.21.15.133:3300/0 conn(0x7f17f415d330 0x7f17f415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 >> v1:172.21.15.99:6789/0 conn(0x7f17f40cd6b0 legacy=0x7f17f40cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc003a00 0x7f17dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17dc004850 0x7f17dc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17dc005210 0x7f17dc00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f415dd00 con 0x7f17dc005210 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc00e2c0 con 0x7f17dc003a00 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc00e480 con 0x7f17dc004850 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17fa400700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17dc005210 0x7f17dc00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f9a00700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc003a00 0x7f17dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f9000700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17dc004850 0x7f17dc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17fa400700 1 -- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17dc004850 msgr2=0x7f17dc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17fa400700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17dc004850 0x7f17dc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17fa400700 1 -- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc003a00 msgr2=0x7f17dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17fa400700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc003a00 0x7f17dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17fa400700 1 -- 172.21.15.99:0/3570951094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc0035b0 con 0x7f17dc005210 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f9a00700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc003a00 0x7f17dc003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17f9000700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17dc004850 0x7f17dc004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.789+0000 7f17fa400700 1 -- 172.21.15.99:0/3570951094 reap_dead start 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fa400700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17dc005210 0x7f17dc00dc20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00a830 tx=0x7f17dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70604be1c2df58f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00a2f0 con 0x7f17dc005210 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec012620 con 0x7f17dc005210 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17f3e00700 1 -- 172.21.15.99:0/3570951094 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0129d0 con 0x7f17dc005210 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fc400700 1 -- 172.21.15.99:0/3570951094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17dc005210 msgr2=0x7f17dc00dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fc400700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17dc005210 0x7f17dc00dc20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00a830 tx=0x7f17dc005880 comp rx=0 tx=0).stop 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fc400700 1 -- 172.21.15.99:0/3570951094 shutdown_connections 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fc400700 1 --2- 172.21.15.99:0/3570951094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17dc005210 0x7f17dc00dc20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fc400700 1 -- 172.21.15.99:0/3570951094 >> 172.21.15.99:0/3570951094 conn(0x7f17f4066540 msgr2=0x7f17f4066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fc400700 1 -- 172.21.15.99:0/3570951094 shutdown_connections 2024-03-29T09:08:53.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.790+0000 7f17fc400700 1 -- 172.21.15.99:0/3570951094 wait complete. 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.792+0000 7f17fc400700 1 Processor -- start 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.792+0000 7f17fc400700 1 -- start start 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17fc400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc005210 0x7f17f4155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17fc400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41567d0 0x7f17f4151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17fc400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f41524d0 0x7f17f414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17fc400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4152950 con 0x7f17f41567d0 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17fc400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4156340 con 0x7f17dc005210 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17fc400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4156c30 con 0x7f17f41524d0 2024-03-29T09:08:53.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17f9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41567d0 0x7f17f4151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17f9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41567d0 0x7f17f4151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38706/0 (socket says 172.21.15.99:38706) 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.793+0000 7f17f9000700 1 -- 172.21.15.99:0/3560013390 learned_addr learned my addr 172.21.15.99:0/3560013390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9a00700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc005210 0x7f17f4155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17fa400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f41524d0 0x7f17f414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9000700 1 -- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f41524d0 msgr2=0x7f17f414f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9000700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f41524d0 0x7f17f414f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9000700 1 -- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc005210 msgr2=0x7f17f4155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9000700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc005210 0x7f17f4155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9000700 1 -- 172.21.15.99:0/3560013390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f4155730 con 0x7f17f41567d0 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9a00700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc005210 0x7f17f4155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17fa400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f41524d0 0x7f17f414f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f9000700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41567d0 0x7f17f4151f90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17e00109e0 tx=0x7f17e0010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:53.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f1600700 1 -- 172.21.15.99:0/3560013390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e000e8d0 con 0x7f17f41567d0 2024-03-29T09:08:53.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f1600700 1 -- 172.21.15.99:0/3560013390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e000ea30 con 0x7f17f41567d0 2024-03-29T09:08:53.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17f1600700 1 -- 172.21.15.99:0/3560013390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e000eda0 con 0x7f17f41567d0 2024-03-29T09:08:53.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.794+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f4155a10 con 0x7f17f41567d0 2024-03-29T09:08:53.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.795+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f4155fd0 con 0x7f17f41567d0 2024-03-29T09:08:53.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.796+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f41564f0 con 0x7f17f41567d0 2024-03-29T09:08:53.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.797+0000 7f17f1600700 1 -- 172.21.15.99:0/3560013390 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f17e0016460 con 0x7f17f41567d0 2024-03-29T09:08:53.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.798+0000 7f17f1600700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f17d406ac90 0x7f17d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:53.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.798+0000 7f17f1600700 1 -- 172.21.15.99:0/3560013390 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f17e0089550 con 0x7f17f41567d0 2024-03-29T09:08:53.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.798+0000 7f17f9a00700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f17d406ac90 0x7f17d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:53.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.799+0000 7f17f9a00700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f17d406ac90 0x7f17d406d150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000c00 tx=0x7f17e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:53.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:53.802+0000 7f17f1600700 1 -- 172.21.15.99:0/3560013390 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f17f41564f0 con 0x7f17f41567d0 2024-03-29T09:08:54.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.055+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17f41564b0 con 0x7f17f41567d0 2024-03-29T09:08:54.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.055+0000 7f17f1600700 1 -- 172.21.15.99:0/3560013390 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f17e0024de0 con 0x7f17f41567d0 2024-03-29T09:08:54.054 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:08:54.055 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-29T09:08:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f17d406ac90 msgr2=0x7f17d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f17d406ac90 0x7f17d406d150 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000c00 tx=0x7f17e8009040 comp rx=0 tx=0).stop 2024-03-29T09:08:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41567d0 msgr2=0x7f17f4151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41567d0 0x7f17f4151f90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f17e00109e0 tx=0x7f17e0010da0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 shutdown_connections 2024-03-29T09:08:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f17d406ac90 0x7f17d406d150 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17f41524d0 0x7f17f414f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17dc005210 0x7f17f4155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 --2- 172.21.15.99:0/3560013390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f17f41567d0 0x7f17f4151f90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 >> 172.21.15.99:0/3560013390 conn(0x7f17f4066540 msgr2=0x7f17f414f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 shutdown_connections 2024-03-29T09:08:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.058+0000 7f17fc400700 1 -- 172.21.15.99:0/3560013390 wait complete. 2024-03-29T09:08:54.058 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 3 2024-03-29T09:08:54.075 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:08:54.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.264+0000 7f2058200700 1 Processor -- start 2024-03-29T09:08:54.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.264+0000 7f2058200700 1 -- start start 2024-03-29T09:08:54.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2058200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20500d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2058200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20500d1b20 0x7f20500ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2058200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20500cddc0 con 0x7f2050160cc0 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2058200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20500ce0a0 con 0x7f20500d1b20 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2058200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20500ce380 con 0x7f20501608e0 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2056200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f20500d1b20 0x7f20500ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2054e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2050160cc0 0x7f20500d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36886/0 (socket says 172.21.15.99:36886) 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2054e00700 1 -- 172.21.15.99:0/2297855981 learned_addr learned my addr 172.21.15.99:0/2297855981 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2055800700 1 --2- 172.21.15.99:0/2297855981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20500d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2056200700 1 -- 172.21.15.99:0/2297855981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 msgr2=0x7f20500d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2056200700 1 --2- 172.21.15.99:0/2297855981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20500d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2056200700 1 -- 172.21.15.99:0/2297855981 >> v1:172.21.15.99:6789/0 conn(0x7f2050160cc0 legacy=0x7f20500d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f2056200700 1 -- 172.21.15.99:0/2297855981 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500ce660 con 0x7f20500d1b20 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.265+0000 7f204fe00700 1 -- 172.21.15.99:0/2297855981 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990984994 0 0) 0x7f20500cddc0 con 0x7f2050160cc0 2024-03-29T09:08:54.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.266+0000 7f2056200700 1 --2- 172.21.15.99:0/2297855981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500d1b20 0x7f20500ceb80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2048009d90 tx=0x7f204800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eae0eb4efd94da1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:54.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.266+0000 7f204fe00700 1 -- 172.21.15.99:0/2297855981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800e070 con 0x7f20500d1b20 2024-03-29T09:08:54.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.266+0000 7f204fe00700 1 -- 172.21.15.99:0/2297855981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800bbd0 con 0x7f20500d1b20 2024-03-29T09:08:54.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.266+0000 7f204fe00700 1 -- 172.21.15.99:0/2297855981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048012430 con 0x7f20500d1b20 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.266+0000 7f2058200700 1 -- 172.21.15.99:0/2297855981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500d1b20 msgr2=0x7f20500ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.266+0000 7f2058200700 1 --2- 172.21.15.99:0/2297855981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500d1b20 0x7f20500ceb80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2048009d90 tx=0x7f204800b330 comp rx=0 tx=0).stop 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.267+0000 7f2058200700 1 -- 172.21.15.99:0/2297855981 shutdown_connections 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.267+0000 7f2058200700 1 --2- 172.21.15.99:0/2297855981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20500d1b20 0x7f20500ceb80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.267+0000 7f2058200700 1 --2- 172.21.15.99:0/2297855981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20500d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.267+0000 7f2058200700 1 -- 172.21.15.99:0/2297855981 >> 172.21.15.99:0/2297855981 conn(0x7f205009c490 msgr2=0x7f20500d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.267+0000 7f2058200700 1 -- 172.21.15.99:0/2297855981 shutdown_connections 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.267+0000 7f2058200700 1 -- 172.21.15.99:0/2297855981 wait complete. 2024-03-29T09:08:54.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.269+0000 7f2058200700 1 Processor -- start 2024-03-29T09:08:54.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.269+0000 7f2058200700 1 -- start start 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.270+0000 7f2058200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20500d1b20 0x7f205015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.270+0000 7f2058200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20501f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.270+0000 7f2058200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050160cc0 0x7f2050158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.270+0000 7f2058200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500ca350 con 0x7f2050160cc0 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.270+0000 7f2058200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500ca630 con 0x7f20500d1b20 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.270+0000 7f2058200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2050157520 con 0x7f20501608e0 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050160cc0 0x7f2050158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050160cc0 0x7f2050158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38720/0 (socket says 172.21.15.99:38720) 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 -- 172.21.15.99:0/2894071850 learned_addr learned my addr 172.21.15.99:0/2894071850 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2055800700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20500d1b20 0x7f205015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2054e00700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20501f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 -- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 msgr2=0x7f20501f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20501f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 -- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20500d1b20 msgr2=0x7f205015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20500d1b20 0x7f205015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 -- 172.21.15.99:0/2894071850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2048009a70 con 0x7f2050160cc0 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2056200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050160cc0 0x7f2050158000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2048013000 tx=0x7f2048004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f204d600700 1 -- 172.21.15.99:0/2894071850 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048012430 con 0x7f2050160cc0 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f204d600700 1 -- 172.21.15.99:0/2894071850 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2048020070 con 0x7f2050160cc0 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2050157800 con 0x7f2050160cc0 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.271+0000 7f204d600700 1 -- 172.21.15.99:0/2894071850 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048004280 con 0x7f2050160cc0 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.272+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501f6430 con 0x7f2050160cc0 2024-03-29T09:08:54.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.273+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050157d20 con 0x7f2050160cc0 2024-03-29T09:08:54.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.273+0000 7f204d600700 1 -- 172.21.15.99:0/2894071850 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f20480043e0 con 0x7f2050160cc0 2024-03-29T09:08:54.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.274+0000 7f204d600700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f203006ac90 0x7f203006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.274+0000 7f204d600700 1 -- 172.21.15.99:0/2894071850 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2048022070 con 0x7f2050160cc0 2024-03-29T09:08:54.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.274+0000 7f2055800700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f203006ac90 0x7f203006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.275+0000 7f2055800700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f203006ac90 0x7f203006d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2044006b20 tx=0x7f2044008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:54.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.278+0000 7f204d600700 1 -- 172.21.15.99:0/2894071850 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2050157d20 con 0x7f2050160cc0 2024-03-29T09:08:54.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.487+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2050157d20 con 0x7f2050160cc0 2024-03-29T09:08:54.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.487+0000 7f204d600700 1 -- 172.21.15.99:0/2894071850 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f2050157d20 con 0x7f2050160cc0 2024-03-29T09:08:54.486 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:08:54.487 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":16,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:08:48.895850+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:08:54.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f203006ac90 msgr2=0x7f203006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f203006ac90 0x7f203006d150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2044006b20 tx=0x7f2044008040 comp rx=0 tx=0).stop 2024-03-29T09:08:54.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050160cc0 msgr2=0x7f2050158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050160cc0 0x7f2050158000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2048013000 tx=0x7f2048004ad0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 shutdown_connections 2024-03-29T09:08:54.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f203006ac90 0x7f203006d150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2050160cc0 0x7f2050158000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20500d1b20 0x7f205015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 --2- 172.21.15.99:0/2894071850 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f20501608e0 0x7f20501f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 >> 172.21.15.99:0/2894071850 conn(0x7f205009c490 msgr2=0x7f20500d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:54.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 shutdown_connections 2024-03-29T09:08:54.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.490+0000 7f2058200700 1 -- 172.21.15.99:0/2894071850 wait complete. 2024-03-29T09:08:54.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:08:54.507 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:08:54.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.700+0000 7f4e54600700 1 Processor -- start 2024-03-29T09:08:54.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.700+0000 7f4e54600700 1 -- start start 2024-03-29T09:08:54.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.701+0000 7f4e54600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4e4c160770 0x7f4e4c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.701+0000 7f4e54600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e4c0d0d60 0x7f4e4c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.701+0000 7f4e54600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0d3810 con 0x7f4e4c0d3380 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.701+0000 7f4e54600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0d1520 con 0x7f4e4c160770 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.701+0000 7f4e54600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0d1800 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4e4c160770 0x7f4e4c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4e4c160770 0x7f4e4c0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38722/0 (socket says 172.21.15.99:38722) 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e52600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4e4c0d3380 0x7f4e4c0cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36898/0 (socket says 172.21.15.99:36898) 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 -- 172.21.15.99:0/1463403180 learned_addr learned my addr 172.21.15.99:0/1463403180 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51200700 1 --2- 172.21.15.99:0/1463403180 >> v2:172.21.15.133:3300/0 conn(0x7f4e4c0d0d60 0x7f4e4c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 -- 172.21.15.99:0/1463403180 >> v2:172.21.15.133:3300/0 conn(0x7f4e4c0d0d60 msgr2=0x7f4e4c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 --2- 172.21.15.99:0/1463403180 >> v2:172.21.15.133:3300/0 conn(0x7f4e4c0d0d60 0x7f4e4c0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 -- 172.21.15.99:0/1463403180 >> v1:172.21.15.99:6789/0 conn(0x7f4e4c0d3380 legacy=0x7f4e4c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 -- 172.21.15.99:0/1463403180 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0d1ae0 con 0x7f4e4c160770 2024-03-29T09:08:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e4be00700 1 -- 172.21.15.99:0/1463403180 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034241189 0 0) 0x7f4e4c0d3810 con 0x7f4e4c0d3380 2024-03-29T09:08:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.702+0000 7f4e51c00700 1 --2- 172.21.15.99:0/1463403180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c160770 0x7f4e4c0b3cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0b5cf0 tx=0x7f4e40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1a8d3ff5f65c0b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e4be00700 1 -- 172.21.15.99:0/1463403180 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000b870 con 0x7f4e4c160770 2024-03-29T09:08:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e4be00700 1 -- 172.21.15.99:0/1463403180 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4000b9d0 con 0x7f4e4c160770 2024-03-29T09:08:54.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e4be00700 1 -- 172.21.15.99:0/1463403180 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000bd60 con 0x7f4e4c160770 2024-03-29T09:08:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 -- 172.21.15.99:0/1463403180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c160770 msgr2=0x7f4e4c0b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 --2- 172.21.15.99:0/1463403180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c160770 0x7f4e4c0b3cc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0b5cf0 tx=0x7f4e40009bd0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 -- 172.21.15.99:0/1463403180 shutdown_connections 2024-03-29T09:08:54.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 --2- 172.21.15.99:0/1463403180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c160770 0x7f4e4c0b3cc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 --2- 172.21.15.99:0/1463403180 >> v2:172.21.15.133:3300/0 conn(0x7f4e4c0d0d60 0x7f4e4c0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 -- 172.21.15.99:0/1463403180 >> 172.21.15.99:0/1463403180 conn(0x7f4e4c08afd0 msgr2=0x7f4e4c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:54.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 -- 172.21.15.99:0/1463403180 shutdown_connections 2024-03-29T09:08:54.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.703+0000 7f4e54600700 1 -- 172.21.15.99:0/1463403180 wait complete. 2024-03-29T09:08:54.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.705+0000 7f4e54600700 1 Processor -- start 2024-03-29T09:08:54.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.705+0000 7f4e54600700 1 -- start start 2024-03-29T09:08:54.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e54600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0d0d60 0x7f4e4c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e54600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c0d3380 0x7f4e4c0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e54600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160770 0x7f4e4c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e54600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c1f6fa0 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e54600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c1f3fa0 con 0x7f4e4c0d3380 2024-03-29T09:08:54.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e54600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c1f4280 con 0x7f4e4c160770 2024-03-29T09:08:54.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e51c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0d0d60 0x7f4e4c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e51c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0d0d60 0x7f4e4c15a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38732/0 (socket says 172.21.15.99:38732) 2024-03-29T09:08:54.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.706+0000 7f4e51c00700 1 -- 172.21.15.99:0/2079328096 learned_addr learned my addr 172.21.15.99:0/2079328096 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:54.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e51c00700 1 -- 172.21.15.99:0/2079328096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160770 msgr2=0x7f4e4c0b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e51c00700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160770 0x7f4e4c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e51c00700 1 -- 172.21.15.99:0/2079328096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c0d3380 msgr2=0x7f4e4c0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e51c00700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c0d3380 0x7f4e4c0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e51c00700 1 -- 172.21.15.99:0/2079328096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40009550 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e51c00700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0d0d60 0x7f4e4c15a810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e40004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:54.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e49600700 1 -- 172.21.15.99:0/2079328096 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40004520 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e49600700 1 -- 172.21.15.99:0/2079328096 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4001e070 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c1f4560 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e49600700 1 -- 172.21.15.99:0/2079328096 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e400038a0 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.707+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c1f64c0 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.708+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c1f3e00 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.709+0000 7f4e49600700 1 -- 172.21.15.99:0/2079328096 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4e40003a00 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.710+0000 7f4e49600700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:54.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.710+0000 7f4e49600700 1 -- 172.21.15.99:0/2079328096 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4e40020070 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.710+0000 7f4e51200700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:54.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.711+0000 7f4e51200700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c15c800 tx=0x7f4e38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:54.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.714+0000 7f4e49600700 1 -- 172.21.15.99:0/2079328096 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4e40055fc0 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.916+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e4c1f3e00 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.916+0000 7f4e49600700 1 -- 172.21.15.99:0/2079328096 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e4c1f3e00 con 0x7f4e4c0d0d60 2024-03-29T09:08:54.915 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:08:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 msgr2=0x7f4e2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c15c800 tx=0x7f4e38008040 comp rx=0 tx=0).stop 2024-03-29T09:08:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0d0d60 msgr2=0x7f4e4c15a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0d0d60 0x7f4e4c15a810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e40004750 comp rx=0 tx=0).stop 2024-03-29T09:08:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 shutdown_connections 2024-03-29T09:08:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e4c160770 0x7f4e4c0b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0d0d60 0x7f4e4c15a810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 --2- 172.21.15.99:0/2079328096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c0d3380 0x7f4e4c0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:54.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 >> 172.21.15.99:0/2079328096 conn(0x7f4e4c08afd0 msgr2=0x7f4e4c0b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:54.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 shutdown_connections 2024-03-29T09:08:54.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:54.919+0000 7f4e54600700 1 -- 172.21.15.99:0/2079328096 wait complete. 2024-03-29T09:08:54.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T09:08:55.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.127+0000 7f5e6e600700 1 Processor -- start 2024-03-29T09:08:55.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.127+0000 7f5e6e600700 1 -- start start 2024-03-29T09:08:55.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e6e600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815d690 0x7f5e680cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e6e600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5e6815da70 0x7f5e680cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e6e600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e680afde0 con 0x7f5e680d1740 2024-03-29T09:08:55.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e6e600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e680cf660 con 0x7f5e6815da70 2024-03-29T09:08:55.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e6e600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680d01d0 con 0x7f5e6815d690 2024-03-29T09:08:55.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e66a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5e6815da70 0x7f5e680cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e67400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815d690 0x7f5e680cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e67e00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5e680d1740 0x7f5e680cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:59134/0 (socket says 172.21.15.99:59134) 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.128+0000 7f5e67e00700 1 -- 172.21.15.99:0/1447012481 learned_addr learned my addr 172.21.15.99:0/1447012481 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.129+0000 7f5e66000700 1 -- 172.21.15.99:0/1447012481 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94257669 0 0) 0x7f5e680afde0 con 0x7f5e680d1740 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.129+0000 7f5e66000700 1 -- 172.21.15.99:0/1447012481 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e4c0035b0 con 0x7f5e680d1740 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.129+0000 7f5e66a00700 1 -- 172.21.15.99:0/1447012481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815d690 msgr2=0x7f5e680cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.129+0000 7f5e66a00700 1 --2- 172.21.15.99:0/1447012481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815d690 0x7f5e680cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.129+0000 7f5e66a00700 1 -- 172.21.15.99:0/1447012481 >> v1:172.21.15.133:6789/0 conn(0x7f5e680d1740 legacy=0x7f5e680cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.129+0000 7f5e66a00700 1 -- 172.21.15.99:0/1447012481 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680d04b0 con 0x7f5e6815da70 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.129+0000 7f5e66a00700 1 --2- 172.21.15.99:0/1447012481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e6815da70 0x7f5e680cf120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009840 tx=0x7f5e54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2819e8f0d9b468f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e66000700 1 -- 172.21.15.99:0/1447012481 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54011830 con 0x7f5e6815da70 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e66000700 1 -- 172.21.15.99:0/1447012481 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e54011990 con 0x7f5e6815da70 2024-03-29T09:08:55.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e66000700 1 -- 172.21.15.99:0/1447012481 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54011d20 con 0x7f5e6815da70 2024-03-29T09:08:55.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 -- 172.21.15.99:0/1447012481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e6815da70 msgr2=0x7f5e680cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 --2- 172.21.15.99:0/1447012481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e6815da70 0x7f5e680cf120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009840 tx=0x7f5e54009b50 comp rx=0 tx=0).stop 2024-03-29T09:08:55.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 -- 172.21.15.99:0/1447012481 shutdown_connections 2024-03-29T09:08:55.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 --2- 172.21.15.99:0/1447012481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815d690 0x7f5e680cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 --2- 172.21.15.99:0/1447012481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e6815da70 0x7f5e680cf120 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 -- 172.21.15.99:0/1447012481 >> 172.21.15.99:0/1447012481 conn(0x7f5e6809c480 msgr2=0x7f5e680d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:55.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 -- 172.21.15.99:0/1447012481 shutdown_connections 2024-03-29T09:08:55.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.130+0000 7f5e6e600700 1 -- 172.21.15.99:0/1447012481 wait complete. 2024-03-29T09:08:55.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.132+0000 7f5e6e600700 1 Processor -- start 2024-03-29T09:08:55.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.132+0000 7f5e6e600700 1 -- start start 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e6e600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e680d1740 0x7f5e6815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e6e600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e6815d690 0x7f5e680cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e6e600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815da70 0x7f5e680b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e6e600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6815afc0 con 0x7f5e6815d690 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e6e600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680cccb0 con 0x7f5e680d1740 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e6e600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680cd140 con 0x7f5e6815da70 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e66a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e6815d690 0x7f5e680cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e66a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e6815d690 0x7f5e680cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38736/0 (socket says 172.21.15.99:38736) 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e66a00700 1 -- 172.21.15.99:0/2513917544 learned_addr learned my addr 172.21.15.99:0/2513917544 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e67e00700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815da70 0x7f5e680b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.133+0000 7f5e67400700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e680d1740 0x7f5e6815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e66a00700 1 -- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815da70 msgr2=0x7f5e680b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e66a00700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815da70 0x7f5e680b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e66a00700 1 -- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e680d1740 msgr2=0x7f5e6815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e66a00700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e680d1740 0x7f5e6815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e66a00700 1 -- 172.21.15.99:0/2513917544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e540094a0 con 0x7f5e6815d690 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e66a00700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e6815d690 0x7f5e680cb890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009fd0 tx=0x7f5e54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e5f400700 1 -- 172.21.15.99:0/2513917544 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004590 con 0x7f5e6815d690 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e5f400700 1 -- 172.21.15.99:0/2513917544 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5401e070 con 0x7f5e6815d690 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e680cbdd0 con 0x7f5e6815d690 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e5f400700 1 -- 172.21.15.99:0/2513917544 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540038e0 con 0x7f5e6815d690 2024-03-29T09:08:55.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.134+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e680cc380 con 0x7f5e6815d690 2024-03-29T09:08:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.136+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e680677a0 con 0x7f5e6815d690 2024-03-29T09:08:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.136+0000 7f5e5f400700 1 -- 172.21.15.99:0/2513917544 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5e54003a40 con 0x7f5e6815d690 2024-03-29T09:08:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.137+0000 7f5e5f400700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5e4806ac90 0x7f5e4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.137+0000 7f5e5f400700 1 -- 172.21.15.99:0/2513917544 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e54089df0 con 0x7f5e6815d690 2024-03-29T09:08:55.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.137+0000 7f5e67400700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5e4806ac90 0x7f5e4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.138+0000 7f5e67400700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5e4806ac90 0x7f5e4806d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e680cb3c0 tx=0x7f5e58009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:55.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.142+0000 7f5e5f400700 1 -- 172.21.15.99:0/2513917544 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5e54055330 con 0x7f5e6815d690 2024-03-29T09:08:55.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.345+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5e680ccdf0 con 0x7f5e6815d690 2024-03-29T09:08:55.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.345+0000 7f5e5f400700 1 -- 172.21.15.99:0/2513917544 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5e54020090 con 0x7f5e6815d690 2024-03-29T09:08:55.344 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:08:55.344 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-03-29T09:08:55.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:08:55.345 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-03-29T09:08:55.345 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-03-29T09:08:55.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:08:55.345 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.347+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5e4806ac90 msgr2=0x7f5e4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.347+0000 7f5e6e600700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5e4806ac90 0x7f5e4806d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e680cb3c0 tx=0x7f5e58009210 comp rx=0 tx=0).stop 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.347+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e6815d690 msgr2=0x7f5e680cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.347+0000 7f5e6e600700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e6815d690 0x7f5e680cb890 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009fd0 tx=0x7f5e54004750 comp rx=0 tx=0).stop 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 shutdown_connections 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5e4806ac90 0x7f5e4806d150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5e6815da70 0x7f5e680b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5e680d1740 0x7f5e6815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 --2- 172.21.15.99:0/2513917544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5e6815d690 0x7f5e680cb890 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 >> 172.21.15.99:0/2513917544 conn(0x7f5e6809c480 msgr2=0x7f5e6815c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 shutdown_connections 2024-03-29T09:08:55.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.348+0000 7f5e6e600700 1 -- 172.21.15.99:0/2513917544 wait complete. 2024-03-29T09:08:55.364 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T09:08:55.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T09:08:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.558+0000 7ff0dd000700 1 Processor -- start 2024-03-29T09:08:55.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.559+0000 7ff0dd000700 1 -- start start 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0dd000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff0d80d0260 0x7ff0d80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0dd000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0d80b3ba0 0x7ff0d80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0dd000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d80d1fa0 con 0x7ff0d8160b30 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0dd000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d80cf440 con 0x7ff0d80b3ba0 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0dd000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d8160f10 con 0x7ff0d80d0260 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0d6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0d80b3ba0 0x7ff0d80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0d6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0d80b3ba0 0x7ff0d80ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38750/0 (socket says 172.21.15.99:38750) 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0d6800700 1 -- 172.21.15.99:0/2305684390 learned_addr learned my addr 172.21.15.99:0/2305684390 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:55.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.560+0000 7ff0cfe00700 1 --2- 172.21.15.99:0/2305684390 >> v2:172.21.15.133:3300/0 conn(0x7ff0d80d0260 0x7ff0d80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d6800700 1 -- 172.21.15.99:0/2305684390 >> v2:172.21.15.133:3300/0 conn(0x7ff0d80d0260 msgr2=0x7ff0d80cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d6800700 1 --2- 172.21.15.99:0/2305684390 >> v2:172.21.15.133:3300/0 conn(0x7ff0d80d0260 0x7ff0d80cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d6800700 1 -- 172.21.15.99:0/2305684390 >> v1:172.21.15.133:6789/0 conn(0x7ff0d8160b30 legacy=0x7ff0d80cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d6800700 1 -- 172.21.15.99:0/2305684390 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d80d1670 con 0x7ff0d80b3ba0 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d6800700 1 --2- 172.21.15.99:0/2305684390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d80b3ba0 0x7ff0d80ce2d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80d42a0 tx=0x7ff0d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf4adff09c968e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d5400700 1 -- 172.21.15.99:0/2305684390 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d000e070 con 0x7ff0d80b3ba0 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d5400700 1 -- 172.21.15.99:0/2305684390 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d000b910 con 0x7ff0d80b3ba0 2024-03-29T09:08:55.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0d5400700 1 -- 172.21.15.99:0/2305684390 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d000bcc0 con 0x7ff0d80b3ba0 2024-03-29T09:08:55.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0dd000700 1 -- 172.21.15.99:0/2305684390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d80b3ba0 msgr2=0x7ff0d80ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.561+0000 7ff0dd000700 1 --2- 172.21.15.99:0/2305684390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d80b3ba0 0x7ff0d80ce2d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80d42a0 tx=0x7ff0d000b330 comp rx=0 tx=0).stop 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.562+0000 7ff0dd000700 1 -- 172.21.15.99:0/2305684390 shutdown_connections 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.562+0000 7ff0dd000700 1 --2- 172.21.15.99:0/2305684390 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d80b3ba0 0x7ff0d80ce2d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.562+0000 7ff0dd000700 1 --2- 172.21.15.99:0/2305684390 >> v2:172.21.15.133:3300/0 conn(0x7ff0d80d0260 0x7ff0d80cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.562+0000 7ff0dd000700 1 -- 172.21.15.99:0/2305684390 >> 172.21.15.99:0/2305684390 conn(0x7ff0d809c4d0 msgr2=0x7ff0d80d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.562+0000 7ff0dd000700 1 -- 172.21.15.99:0/2305684390 shutdown_connections 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.562+0000 7ff0dd000700 1 -- 172.21.15.99:0/2305684390 wait complete. 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.563+0000 7ff0dd000700 1 Processor -- start 2024-03-29T09:08:55.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.564+0000 7ff0dd000700 1 -- start start 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.564+0000 7ff0dd000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0d80b3ba0 0x7ff0d80b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.564+0000 7ff0dd000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d80d0260 0x7ff0d80ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.564+0000 7ff0dd000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d8160b30 0x7ff0d81f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.564+0000 7ff0dd000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80b6390 con 0x7ff0d8160b30 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.564+0000 7ff0dd000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80cd070 con 0x7ff0d80d0260 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.564+0000 7ff0dd000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d80b8ab0 con 0x7ff0d80b3ba0 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d8160b30 0x7ff0d81f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d8160b30 0x7ff0d81f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38758/0 (socket says 172.21.15.99:38758) 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 -- 172.21.15.99:0/207814987 learned_addr learned my addr 172.21.15.99:0/207814987 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 -- 172.21.15.99:0/207814987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0d80b3ba0 msgr2=0x7ff0d80b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0d80b3ba0 0x7ff0d80b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 -- 172.21.15.99:0/207814987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d80d0260 msgr2=0x7ff0d80ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d80d0260 0x7ff0d80ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 -- 172.21.15.99:0/207814987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0009b30 con 0x7ff0d8160b30 2024-03-29T09:08:55.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0d6800700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d8160b30 0x7ff0d81f61f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0d808d930 tx=0x7ff0d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:55.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0ce000700 1 -- 172.21.15.99:0/207814987 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d00099b0 con 0x7ff0d8160b30 2024-03-29T09:08:55.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0ce000700 1 -- 172.21.15.99:0/207814987 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d00092e0 con 0x7ff0d8160b30 2024-03-29T09:08:55.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d80b6630 con 0x7ff0d8160b30 2024-03-29T09:08:55.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.565+0000 7ff0ce000700 1 -- 172.21.15.99:0/207814987 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d001c640 con 0x7ff0d8160b30 2024-03-29T09:08:55.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.566+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d80b6bf0 con 0x7ff0d8160b30 2024-03-29T09:08:55.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.566+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d80678b0 con 0x7ff0d8160b30 2024-03-29T09:08:55.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.567+0000 7ff0ce000700 1 -- 172.21.15.99:0/207814987 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff0d001c7a0 con 0x7ff0d8160b30 2024-03-29T09:08:55.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.568+0000 7ff0ce000700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff0b406ac90 0x7ff0b406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:55.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.568+0000 7ff0ce000700 1 -- 172.21.15.99:0/207814987 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff0d008a950 con 0x7ff0d8160b30 2024-03-29T09:08:55.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.568+0000 7ff0d5e00700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff0b406ac90 0x7ff0b406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:55.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.569+0000 7ff0d5e00700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff0b406ac90 0x7ff0b406d150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8000f60 tx=0x7ff0c8007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:55.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.573+0000 7ff0ce000700 1 -- 172.21.15.99:0/207814987 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff0d0055f10 con 0x7ff0d8160b30 2024-03-29T09:08:55.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.776+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff0d80c9f40 con 0x7ff0d8160b30 2024-03-29T09:08:55.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.873+0000 7ff0ce000700 1 -- 172.21.15.99:0/207814987 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff0d0059530 con 0x7ff0d8160b30 2024-03-29T09:08:55.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.876+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff0b406ac90 msgr2=0x7ff0b406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.876+0000 7ff0dd000700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff0b406ac90 0x7ff0b406d150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8000f60 tx=0x7ff0c8007a40 comp rx=0 tx=0).stop 2024-03-29T09:08:55.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.876+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d8160b30 msgr2=0x7ff0d81f61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:55.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.876+0000 7ff0dd000700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d8160b30 0x7ff0d81f61f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0d808d930 tx=0x7ff0d0004ad0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.877+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 shutdown_connections 2024-03-29T09:08:55.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.877+0000 7ff0dd000700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff0b406ac90 0x7ff0b406d150 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.877+0000 7ff0dd000700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0d8160b30 0x7ff0d81f61f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.877+0000 7ff0dd000700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0d80b3ba0 0x7ff0d80b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.877+0000 7ff0dd000700 1 --2- 172.21.15.99:0/207814987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0d80d0260 0x7ff0d80ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:55.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.877+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 >> 172.21.15.99:0/207814987 conn(0x7ff0d809c4d0 msgr2=0x7ff0d80cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:55.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.878+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 shutdown_connections 2024-03-29T09:08:55.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:55.878+0000 7ff0dd000700 1 -- 172.21.15.99:0/207814987 wait complete. 2024-03-29T09:08:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-03-29T09:08:55.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-29T09:08:56.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.097+0000 7f46a7800700 1 Processor -- start 2024-03-29T09:08:56.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.097+0000 7f46a7800700 1 -- start start 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.098+0000 7f46a7800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 0x7f46a00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.098+0000 7f46a7800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00d3810 con 0x7f46a0160770 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.098+0000 7f46a7800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00d1520 con 0x7f46a00d3380 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.098+0000 7f46a7800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00b4140 con 0x7f46a00b3cc0 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.098+0000 7f46a4e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f46a0160770 0x7f46a00d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36910/0 (socket says 172.21.15.99:36910) 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.098+0000 7f46a4e00700 1 -- 172.21.15.99:0/1656647229 learned_addr learned my addr 172.21.15.99:0/1656647229 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469fe00700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 0x7f46a00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273858673 0 0) 0x7f46a00d3810 con 0x7f46a0160770 2024-03-29T09:08:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46880035b0 con 0x7f46a0160770 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 458566276 0 0) 0x7f46880035b0 con 0x7f46a0160770 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 msgr2=0x7f46a00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 0x7f46a00d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 >> v1:172.21.15.133:6789/0 conn(0x7f46a00d3380 legacy=0x7f46a00ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00d1800 con 0x7f46a0160770 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f4694001670 con 0x7f46a0160770 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 >> v1:172.21.15.99:6789/0 conn(0x7f46a0160770 legacy=0x7f46a00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4688003a30 0x7f4688003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688004850 0x7f4688004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 0x7f4688005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468800bac0 con 0x7f4688003a30 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468800bc00 con 0x7f4688004850 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468800bdc0 con 0x7f46a00b3cc0 2024-03-29T09:08:56.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.099+0000 7f46a4e00700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4688003a30 0x7f4688003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f469fe00700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688004850 0x7f4688004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f46a5800700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 0x7f4688005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f46a4e00700 1 -- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 msgr2=0x7f4688005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f46a4e00700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00b3cc0 0x7f4688005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f46a4e00700 1 -- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688004850 msgr2=0x7f4688004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f46a4e00700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688004850 0x7f4688004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f46a4e00700 1 -- 172.21.15.99:0/1656647229 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880035b0 con 0x7f4688003a30 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f46a4e00700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4688003a30 0x7f4688003e50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f469400c270 tx=0x7f4688005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfddbaa08ca3f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694007c10 con 0x7f4688003a30 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469400cdb0 con 0x7f4688003a30 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.100+0000 7f469f400700 1 -- 172.21.15.99:0/1656647229 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694005ba0 con 0x7f4688003a30 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.101+0000 7f46a7800700 1 -- 172.21.15.99:0/1656647229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4688003a30 msgr2=0x7f4688003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.101+0000 7f46a7800700 1 --2- 172.21.15.99:0/1656647229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4688003a30 0x7f4688003e50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f469400c270 tx=0x7f4688005880 comp rx=0 tx=0).stop 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.101+0000 7f46a5800700 1 -- 172.21.15.99:0/1656647229 reap_dead start 2024-03-29T09:08:56.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.101+0000 7f46a7800700 1 -- 172.21.15.99:0/1656647229 shutdown_connections 2024-03-29T09:08:56.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.101+0000 7f46a7800700 1 -- 172.21.15.99:0/1656647229 >> 172.21.15.99:0/1656647229 conn(0x7f46a008afd0 msgr2=0x7f46a009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:56.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.101+0000 7f46a7800700 1 -- 172.21.15.99:0/1656647229 shutdown_connections 2024-03-29T09:08:56.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.101+0000 7f46a7800700 1 -- 172.21.15.99:0/1656647229 wait complete. 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.103+0000 7f46a7800700 1 Processor -- start 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.103+0000 7f46a7800700 1 -- start start 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a7800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688003a30 0x7f46a0153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a7800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00ccae0 0x7f46a015a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a7800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a006bac0 0x7f46a006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a7800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a006c480 con 0x7f46a006bac0 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a7800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00cc650 con 0x7f4688003a30 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a7800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00cb5d0 con 0x7f46a00ccae0 2024-03-29T09:08:56.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a006bac0 0x7f46a006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a4e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688003a30 0x7f46a0153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a4e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688003a30 0x7f46a0153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:39586/0 (socket says 172.21.15.99:39586) 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a4e00700 1 -- 172.21.15.99:0/1403551261 learned_addr learned my addr 172.21.15.99:0/1403551261 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f46a5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a006bac0 0x7f46a006bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38786/0 (socket says 172.21.15.99:38786) 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.104+0000 7f469fe00700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00ccae0 0x7f46a015a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a5800700 1 -- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00ccae0 msgr2=0x7f46a015a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a5800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00ccae0 0x7f46a015a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a5800700 1 -- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688003a30 msgr2=0x7f46a0153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a5800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688003a30 0x7f46a0153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a5800700 1 -- 172.21.15.99:0/1403551261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00cb8b0 con 0x7f46a006bac0 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a4e00700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688003a30 0x7f46a0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a5800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a006bac0 0x7f46a006bf40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46a008d9f0 tx=0x7f469800d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f469cc00700 1 -- 172.21.15.99:0/1403551261 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698003c10 con 0x7f46a006bac0 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f469cc00700 1 -- 172.21.15.99:0/1403551261 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698003d90 con 0x7f46a006bac0 2024-03-29T09:08:56.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f469cc00700 1 -- 172.21.15.99:0/1403551261 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800d950 con 0x7f46a006bac0 2024-03-29T09:08:56.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a00cc000 con 0x7f46a006bac0 2024-03-29T09:08:56.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.105+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a015a9e0 con 0x7f46a006bac0 2024-03-29T09:08:56.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.106+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a00678b0 con 0x7f46a006bac0 2024-03-29T09:08:56.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.107+0000 7f469cc00700 1 -- 172.21.15.99:0/1403551261 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f469800dab0 con 0x7f46a006bac0 2024-03-29T09:08:56.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.108+0000 7f469cc00700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f468006ac90 0x7f468006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:56.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.108+0000 7f469cc00700 1 -- 172.21.15.99:0/1403551261 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4698089300 con 0x7f46a006bac0 2024-03-29T09:08:56.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.109+0000 7f46a4e00700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f468006ac90 0x7f468006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:56.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.109+0000 7f46a4e00700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f468006ac90 0x7f468006d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4694015040 tx=0x7f4694007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:56.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.113+0000 7f469cc00700 1 -- 172.21.15.99:0/1403551261 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4698054620 con 0x7f46a006bac0 2024-03-29T09:08:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.319+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f46a00cc790 con 0x7f46a006bac0 2024-03-29T09:08:56.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.885+0000 7f469cc00700 1 -- 172.21.15.99:0/1403551261 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f4698057c40 con 0x7f46a006bac0 2024-03-29T09:08:56.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f468006ac90 msgr2=0x7f468006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:56.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f468006ac90 0x7f468006d150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4694015040 tx=0x7f4694007000 comp rx=0 tx=0).stop 2024-03-29T09:08:56.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a006bac0 msgr2=0x7f46a006bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:56.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a006bac0 0x7f46a006bf40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f46a008d9f0 tx=0x7f469800d150 comp rx=0 tx=0).stop 2024-03-29T09:08:56.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 shutdown_connections 2024-03-29T09:08:56.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f468006ac90 0x7f468006d150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46a006bac0 0x7f46a006bf40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4688003a30 0x7f46a0153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 --2- 172.21.15.99:0/1403551261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46a00ccae0 0x7f46a015a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:56.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.903+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 >> 172.21.15.99:0/1403551261 conn(0x7f46a008afd0 msgr2=0x7f46a015e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:56.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.904+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 shutdown_connections 2024-03-29T09:08:56.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:56.905+0000 7f46a7800700 1 -- 172.21.15.99:0/1403551261 wait complete. 2024-03-29T09:08:56.909 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-03-29T09:08:56.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T09:08:57.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.124+0000 7fd50b200700 1 Processor -- start 2024-03-29T09:08:57.123 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.124+0000 7fd50b200700 1 -- start start 2024-03-29T09:08:57.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.125+0000 7fd50b200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c0cff50 0x7fd50c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.125+0000 7fd50b200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd50c0d3960 0x7fd50c0d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.125+0000 7fd50b200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c0d3e60 con 0x7fd50c160790 2024-03-29T09:08:57.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.125+0000 7fd50b200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0d0b40 con 0x7fd50c0d3960 2024-03-29T09:08:57.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.125+0000 7fd50b200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0ce7c0 con 0x7fd50c0cff50 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd50a800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd50c0d3960 0x7fd50c0d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd509400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c0cff50 0x7fd50c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd509e00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd50c160790 0x7fd50c0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:59166/0 (socket says 172.21.15.99:59166) 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd509e00700 1 -- 172.21.15.99:0/1549609754 learned_addr learned my addr 172.21.15.99:0/1549609754 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd508a00700 1 -- 172.21.15.99:0/1549609754 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490743773 0 0) 0x7fd50c0d3e60 con 0x7fd50c160790 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd50a800700 1 -- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c0cff50 msgr2=0x7fd50c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd50a800700 1 --2- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c0cff50 0x7fd50c0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd50a800700 1 -- 172.21.15.99:0/1549609754 >> v1:172.21.15.133:6789/0 conn(0x7fd50c160790 legacy=0x7fd50c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd50a800700 1 -- 172.21.15.99:0/1549609754 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c0ceaa0 con 0x7fd50c0d3960 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.126+0000 7fd509400700 1 --2- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c0cff50 0x7fd50c0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd50a800700 1 --2- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 0x7fd50c0d1510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd504009d90 tx=0x7fd50400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6ab4eff16fce3c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:57.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd508a00700 1 -- 172.21.15.99:0/1549609754 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400e070 con 0x7fd50c0d3960 2024-03-29T09:08:57.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd508a00700 1 -- 172.21.15.99:0/1549609754 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50400bb20 con 0x7fd50c0d3960 2024-03-29T09:08:57.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd508a00700 1 -- 172.21.15.99:0/1549609754 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504012430 con 0x7fd50c0d3960 2024-03-29T09:08:57.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd50b200700 1 -- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 msgr2=0x7fd50c0d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd50b200700 1 --2- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 0x7fd50c0d1510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd504009d90 tx=0x7fd50400b330 comp rx=0 tx=0).stop 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd50b200700 1 -- 172.21.15.99:0/1549609754 shutdown_connections 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd50b200700 1 --2- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 0x7fd50c0d1510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd50b200700 1 --2- 172.21.15.99:0/1549609754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c0cff50 0x7fd50c0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.127+0000 7fd50b200700 1 -- 172.21.15.99:0/1549609754 >> 172.21.15.99:0/1549609754 conn(0x7fd50c09c340 msgr2=0x7fd50c0d2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.128+0000 7fd50b200700 1 -- 172.21.15.99:0/1549609754 shutdown_connections 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.128+0000 7fd50b200700 1 -- 172.21.15.99:0/1549609754 wait complete. 2024-03-29T09:08:57.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.129+0000 7fd50b200700 1 Processor -- start 2024-03-29T09:08:57.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.129+0000 7fd50b200700 1 -- start start 2024-03-29T09:08:57.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.130+0000 7fd50b200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c0cff50 0x7fd50c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.130+0000 7fd50b200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 0x7fd50c151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.130+0000 7fd50b200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c160790 0x7fd50c151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.130+0000 7fd50b200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c150d80 con 0x7fd50c0cff50 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.130+0000 7fd50b200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c1f2900 con 0x7fd50c0d3960 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.130+0000 7fd50b200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c150170 con 0x7fd50c160790 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c0cff50 0x7fd50c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c0cff50 0x7fd50c14bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38798/0 (socket says 172.21.15.99:38798) 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 -- 172.21.15.99:0/244075150 learned_addr learned my addr 172.21.15.99:0/244075150 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd50a800700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c160790 0x7fd50c151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509400700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 0x7fd50c151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 -- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c160790 msgr2=0x7fd50c151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c160790 0x7fd50c151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 -- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 msgr2=0x7fd50c151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 0x7fd50c151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 -- 172.21.15.99:0/244075150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80094a0 con 0x7fd50c0cff50 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd509e00700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c0cff50 0x7fd50c14bde0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c550 tx=0x7fd4fc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.131+0000 7fd502000700 1 -- 172.21.15.99:0/244075150 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc004c80 con 0x7fd50c0cff50 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.132+0000 7fd502000700 1 -- 172.21.15.99:0/244075150 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc004e00 con 0x7fd50c0cff50 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.132+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd504009a70 con 0x7fd50c0cff50 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.132+0000 7fd502000700 1 -- 172.21.15.99:0/244075150 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc007310 con 0x7fd50c0cff50 2024-03-29T09:08:57.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.132+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c0b5f40 con 0x7fd50c0cff50 2024-03-29T09:08:57.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.133+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c151060 con 0x7fd50c0cff50 2024-03-29T09:08:57.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.134+0000 7fd502000700 1 -- 172.21.15.99:0/244075150 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd4fc007470 con 0x7fd50c0cff50 2024-03-29T09:08:57.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.135+0000 7fd502000700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4ec06ac90 0x7fd4ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.135+0000 7fd502000700 1 -- 172.21.15.99:0/244075150 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd4fc088f60 con 0x7fd50c0cff50 2024-03-29T09:08:57.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.135+0000 7fd509400700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4ec06ac90 0x7fd4ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.135+0000 7fd509400700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4ec06ac90 0x7fd4ec06d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0b40b0 tx=0x7fd4f8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:57.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.139+0000 7fd502000700 1 -- 172.21.15.99:0/244075150 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd4fc054100 con 0x7fd50c0cff50 2024-03-29T09:08:57.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.385+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd50c067d80 con 0x7fd50c0cff50 2024-03-29T09:08:57.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.393+0000 7fd502000700 1 -- 172.21.15.99:0/244075150 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd4fc057720 con 0x7fd50c0cff50 2024-03-29T09:08:57.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.398+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4ec06ac90 msgr2=0x7fd4ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.398+0000 7fd50b200700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4ec06ac90 0x7fd4ec06d150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0b40b0 tx=0x7fd4f8009210 comp rx=0 tx=0).stop 2024-03-29T09:08:57.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.398+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c0cff50 msgr2=0x7fd50c14bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.398+0000 7fd50b200700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c0cff50 0x7fd50c14bde0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c550 tx=0x7fd4fc00c860 comp rx=0 tx=0).stop 2024-03-29T09:08:57.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.402+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 shutdown_connections 2024-03-29T09:08:57.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.402+0000 7fd50b200700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4ec06ac90 0x7fd4ec06d150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.402+0000 7fd50b200700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd50c160790 0x7fd50c151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.402+0000 7fd50b200700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd50c0cff50 0x7fd50c14bde0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.402+0000 7fd50b200700 1 --2- 172.21.15.99:0/244075150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd50c0d3960 0x7fd50c151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.402+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 >> 172.21.15.99:0/244075150 conn(0x7fd50c09c340 msgr2=0x7fd50c0b7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:57.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.403+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 shutdown_connections 2024-03-29T09:08:57.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.403+0000 7fd50b200700 1 -- 172.21.15.99:0/244075150 wait complete. 2024-03-29T09:08:57.409 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 2 and data pool 3 2024-03-29T09:08:57.437 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-29T09:08:57.437 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-29T09:08:57.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.632+0000 7fa09b800700 1 Processor -- start 2024-03-29T09:08:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.633+0000 7fa09b800700 1 -- start start 2024-03-29T09:08:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa09b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa094160b30 0x7fa0940cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa09b800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0940d1fa0 con 0x7fa0940d0090 2024-03-29T09:08:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa09b800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa094160f10 con 0x7fa0940b1f50 2024-03-29T09:08:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa09b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940cf440 con 0x7fa094160b30 2024-03-29T09:08:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa099800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa0940d0090 0x7fa0940b32a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36922/0 (socket says 172.21.15.99:36922) 2024-03-29T09:08:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa099800700 1 -- 172.21.15.99:0/905602261 learned_addr learned my addr 172.21.15.99:0/905602261 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:08:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa098e00700 1 --2- 172.21.15.99:0/905602261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa094160b30 0x7fa0940cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa093400700 1 -- 172.21.15.99:0/905602261 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929535838 0 0) 0x7fa0940d1fa0 con 0x7fa0940d0090 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa093400700 1 -- 172.21.15.99:0/905602261 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa07c0035b0 con 0x7fa0940d0090 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa093400700 1 -- 172.21.15.99:0/905602261 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17106548 0 0) 0x7fa094160f10 con 0x7fa0940b1f50 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.634+0000 7fa093400700 1 -- 172.21.15.99:0/905602261 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0940d1fa0 con 0x7fa0940b1f50 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.635+0000 7fa098e00700 1 -- 172.21.15.99:0/905602261 >> v1:172.21.15.133:6789/0 conn(0x7fa0940b1f50 legacy=0x7fa0940b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.635+0000 7fa098e00700 1 -- 172.21.15.99:0/905602261 >> v1:172.21.15.99:6789/0 conn(0x7fa0940d0090 legacy=0x7fa0940b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.635+0000 7fa098e00700 1 -- 172.21.15.99:0/905602261 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0940d1670 con 0x7fa094160b30 2024-03-29T09:08:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.635+0000 7fa098e00700 1 --2- 172.21.15.99:0/905602261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa094160b30 0x7fa0940cfb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa088009870 tx=0x7fa088009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe17984785f5386 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.635+0000 7fa093400700 1 -- 172.21.15.99:0/905602261 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011830 con 0x7fa094160b30 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.635+0000 7fa093400700 1 -- 172.21.15.99:0/905602261 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa088011990 con 0x7fa094160b30 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.635+0000 7fa093400700 1 -- 172.21.15.99:0/905602261 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa088011d00 con 0x7fa094160b30 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.636+0000 7fa09b800700 1 -- 172.21.15.99:0/905602261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa094160b30 msgr2=0x7fa0940cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.636+0000 7fa09b800700 1 --2- 172.21.15.99:0/905602261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa094160b30 0x7fa0940cfb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa088009870 tx=0x7fa088009b80 comp rx=0 tx=0).stop 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.636+0000 7fa09b800700 1 -- 172.21.15.99:0/905602261 shutdown_connections 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.636+0000 7fa09b800700 1 --2- 172.21.15.99:0/905602261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa094160b30 0x7fa0940cfb50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.636+0000 7fa09b800700 1 -- 172.21.15.99:0/905602261 >> 172.21.15.99:0/905602261 conn(0x7fa09409c4d0 msgr2=0x7fa0940d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.636+0000 7fa09b800700 1 -- 172.21.15.99:0/905602261 shutdown_connections 2024-03-29T09:08:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.636+0000 7fa09b800700 1 -- 172.21.15.99:0/905602261 wait complete. 2024-03-29T09:08:57.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.637+0000 7fa09b800700 1 Processor -- start 2024-03-29T09:08:57.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.638+0000 7fa09b800700 1 -- start start 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa09b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0940b1f50 0x7fa09414c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa09b800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0940d0090 0x7fa09409cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa09b800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa094160b30 0x7fa09409d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa09b800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940a0b90 con 0x7fa0940d0090 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa09b800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09414d3a0 con 0x7fa094160b30 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa09b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940a05d0 con 0x7fa0940b1f50 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0940d0090 0x7fa09409cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0940d0090 0x7fa09409cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38806/0 (socket says 172.21.15.99:38806) 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 -- 172.21.15.99:0/4158901894 learned_addr learned my addr 172.21.15.99:0/4158901894 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 -- 172.21.15.99:0/4158901894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0940b1f50 msgr2=0x7fa09414c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0940b1f50 0x7fa09414c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 -- 172.21.15.99:0/4158901894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa094160b30 msgr2=0x7fa09409d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa094160b30 0x7fa09409d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 -- 172.21.15.99:0/4158901894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0880094a0 con 0x7fa0940d0090 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.639+0000 7fa093e00700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0940d0090 0x7fa09409cf40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c530 tx=0x7fa08000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.640+0000 7fa090c00700 1 -- 172.21.15.99:0/4158901894 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0800052d0 con 0x7fa0940d0090 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.640+0000 7fa090c00700 1 -- 172.21.15.99:0/4158901894 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa080005430 con 0x7fa0940d0090 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.640+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0940a08b0 con 0x7fa0940d0090 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.640+0000 7fa090c00700 1 -- 172.21.15.99:0/4158901894 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0800038c0 con 0x7fa0940d0090 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.640+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09414c9e0 con 0x7fa0940d0090 2024-03-29T09:08:57.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.641+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09406c6c0 con 0x7fa0940d0090 2024-03-29T09:08:57.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.642+0000 7fa090c00700 1 -- 172.21.15.99:0/4158901894 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa080003a20 con 0x7fa0940d0090 2024-03-29T09:08:57.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.642+0000 7fa090c00700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa07406ac90 0x7fa07406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:57.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.643+0000 7fa090c00700 1 -- 172.21.15.99:0/4158901894 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa080088800 con 0x7fa0940d0090 2024-03-29T09:08:57.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.643+0000 7fa098e00700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa07406ac90 0x7fa07406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:57.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.643+0000 7fa098e00700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa07406ac90 0x7fa07406d150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b13f0 tx=0x7fa088010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:57.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.647+0000 7fa090c00700 1 -- 172.21.15.99:0/4158901894 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa0800539d0 con 0x7fa0940d0090 2024-03-29T09:08:57.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:57.861+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa09414d4e0 con 0x7fa0940d0090 2024-03-29T09:08:58.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.396+0000 7fa090c00700 1 -- 172.21.15.99:0/4158901894 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa09414d4e0 con 0x7fa0940d0090 2024-03-29T09:08:58.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa07406ac90 msgr2=0x7fa07406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:58.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa07406ac90 0x7fa07406d150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0940b13f0 tx=0x7fa088010000 comp rx=0 tx=0).stop 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0940d0090 msgr2=0x7fa09409cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0940d0090 0x7fa09409cf40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa08000c530 tx=0x7fa08000c8f0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 shutdown_connections 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa07406ac90 0x7fa07406d150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa094160b30 0x7fa09409d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0940b1f50 0x7fa09414c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 --2- 172.21.15.99:0/4158901894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0940d0090 0x7fa09409cf40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 >> 172.21.15.99:0/4158901894 conn(0x7fa09409c4d0 msgr2=0x7fa09406c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 shutdown_connections 2024-03-29T09:08:58.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.400+0000 7fa09b800700 1 -- 172.21.15.99:0/4158901894 wait complete. 2024-03-29T09:08:58.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T09:08:58.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.622+0000 7f43f0600700 1 Processor -- start 2024-03-29T09:08:58.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.622+0000 7f43f0600700 1 -- start start 2024-03-29T09:08:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43f0600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e80b3cc0 0x7f43e80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43f0600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43e80d3380 0x7f43e80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43f0600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e80d3810 con 0x7f43e8160770 2024-03-29T09:08:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43f0600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e80b4140 con 0x7f43e80d3380 2024-03-29T09:08:58.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43f0600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80d1520 con 0x7f43e80b3cc0 2024-03-29T09:08:58.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43ee600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43e80d3380 0x7f43e80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:58.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43ee600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43e80d3380 0x7f43e80ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60770/0 (socket says 172.21.15.99:60770) 2024-03-29T09:08:58.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.623+0000 7f43ee600700 1 -- 172.21.15.99:0/3125355056 learned_addr learned my addr 172.21.15.99:0/3125355056 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:58.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43ee600700 1 -- 172.21.15.99:0/3125355056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e80b3cc0 msgr2=0x7f43e80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:58.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43ee600700 1 --2- 172.21.15.99:0/3125355056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e80b3cc0 0x7f43e80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43ee600700 1 -- 172.21.15.99:0/3125355056 >> v1:172.21.15.133:6789/0 conn(0x7f43e8160770 legacy=0x7f43e80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:08:58.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43ee600700 1 -- 172.21.15.99:0/3125355056 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e80d1800 con 0x7f43e80d3380 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43ee600700 1 --2- 172.21.15.99:0/3125355056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80d3380 0x7f43e80ce080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e50 tx=0x7f43e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc9ffea905c5bd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43e7e00700 1 -- 172.21.15.99:0/3125355056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000e070 con 0x7f43e80d3380 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43e7e00700 1 -- 172.21.15.99:0/3125355056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e0003a20 con 0x7f43e80d3380 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.624+0000 7f43e7e00700 1 -- 172.21.15.99:0/3125355056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0003dd0 con 0x7f43e80d3380 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 -- 172.21.15.99:0/3125355056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80d3380 msgr2=0x7f43e80ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 --2- 172.21.15.99:0/3125355056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80d3380 0x7f43e80ce080 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e50 tx=0x7f43e000b330 comp rx=0 tx=0).stop 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 -- 172.21.15.99:0/3125355056 shutdown_connections 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 --2- 172.21.15.99:0/3125355056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80d3380 0x7f43e80ce080 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 --2- 172.21.15.99:0/3125355056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e80b3cc0 0x7f43e80d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 -- 172.21.15.99:0/3125355056 >> 172.21.15.99:0/3125355056 conn(0x7f43e808afd0 msgr2=0x7f43e809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 -- 172.21.15.99:0/3125355056 shutdown_connections 2024-03-29T09:08:58.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.625+0000 7f43f0600700 1 -- 172.21.15.99:0/3125355056 wait complete. 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.626+0000 7f43f0600700 1 Processor -- start 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.627+0000 7f43f0600700 1 -- start start 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.627+0000 7f43f0600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80b3cc0 0x7f43e814c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.627+0000 7f43f0600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e80d3380 0x7f43e80c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.627+0000 7f43f0600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e8160770 0x7f43e81f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43f0600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e81f6f00 con 0x7f43e80b3cc0 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43f0600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e814d030 con 0x7f43e80d3380 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43f0600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e814c790 con 0x7f43e8160770 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80b3cc0 0x7f43e814c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80b3cc0 0x7f43e814c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60772/0 (socket says 172.21.15.99:60772) 2024-03-29T09:08:58.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 -- 172.21.15.99:0/3187470478 learned_addr learned my addr 172.21.15.99:0/3187470478 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 -- 172.21.15.99:0/3187470478 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e8160770 msgr2=0x7f43e81f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e8160770 0x7f43e81f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 -- 172.21.15.99:0/3187470478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e80d3380 msgr2=0x7f43e80c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e80d3380 0x7f43e80c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 -- 172.21.15.99:0/3187470478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0009b30 con 0x7f43e80b3cc0 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.628+0000 7f43edc00700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80b3cc0 0x7f43e814c100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00b470 tx=0x7f43dc00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.629+0000 7f43e5600700 1 -- 172.21.15.99:0/3187470478 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc011830 con 0x7f43e80b3cc0 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.629+0000 7f43e5600700 1 -- 172.21.15.99:0/3187470478 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc011990 con 0x7f43e80b3cc0 2024-03-29T09:08:58.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.629+0000 7f43e5600700 1 -- 172.21.15.99:0/3187470478 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc011d40 con 0x7f43e80b3cc0 2024-03-29T09:08:58.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.629+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e814ca70 con 0x7f43e80b3cc0 2024-03-29T09:08:58.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.629+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e81f2b60 con 0x7f43e80b3cc0 2024-03-29T09:08:58.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.630+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e81f2d80 con 0x7f43e80b3cc0 2024-03-29T09:08:58.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.631+0000 7f43e5600700 1 -- 172.21.15.99:0/3187470478 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f43dc01a400 con 0x7f43e80b3cc0 2024-03-29T09:08:58.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.631+0000 7f43e5600700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43c806ac90 0x7f43c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:08:58.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.631+0000 7f43e5600700 1 -- 172.21.15.99:0/3187470478 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f43dc089460 con 0x7f43e80b3cc0 2024-03-29T09:08:58.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.632+0000 7f43ed200700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43c806ac90 0x7f43c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:08:58.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.632+0000 7f43ed200700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43c806ac90 0x7f43c806d150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f43e80c9ce0 tx=0x7f43d4006d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:08:58.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.636+0000 7f43e5600700 1 -- 172.21.15.99:0/3187470478 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f43dc054550 con 0x7f43e80b3cc0 2024-03-29T09:08:58.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:08:58.840+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f43e80678b0 con 0x7f43e80b3cc0 2024-03-29T09:08:58.875 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-03-29T09:08:58.876+0000 7fa564000700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T09:09:00.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.407+0000 7f43e5600700 1 -- 172.21.15.99:0/3187470478 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f43dc057b70 con 0x7f43e80b3cc0 2024-03-29T09:09:00.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43c806ac90 msgr2=0x7f43c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:00.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43c806ac90 0x7f43c806d150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f43e80c9ce0 tx=0x7f43d4006d20 comp rx=0 tx=0).stop 2024-03-29T09:09:00.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80b3cc0 msgr2=0x7f43e814c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:00.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80b3cc0 0x7f43e814c100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00b470 tx=0x7f43dc00b830 comp rx=0 tx=0).stop 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 shutdown_connections 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43c806ac90 0x7f43c806d150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43e8160770 0x7f43e81f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43e80b3cc0 0x7f43e814c100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 --2- 172.21.15.99:0/3187470478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43e80d3380 0x7f43e80c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.414+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 >> 172.21.15.99:0/3187470478 conn(0x7f43e808afd0 msgr2=0x7f43e80b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.415+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 shutdown_connections 2024-03-29T09:09:00.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.415+0000 7f43f0600700 1 -- 172.21.15.99:0/3187470478 wait complete. 2024-03-29T09:09:00.418 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data_ec' created 2024-03-29T09:09:00.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-29T09:09:00.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.636+0000 7feff3600700 1 Processor -- start 2024-03-29T09:09:00.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.636+0000 7feff3600700 1 -- start start 2024-03-29T09:09:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7feff3600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fefec160b30 0x7fefec0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7feff3600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7feff3600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefec0d1fa0 con 0x7fefec0b1f50 2024-03-29T09:09:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7feff3600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefec0cf440 con 0x7fefec160b30 2024-03-29T09:09:00.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7feff3600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0d0500 con 0x7fefec0d0090 2024-03-29T09:09:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7fefebe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fefec0b1f50 0x7fefec0b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35390/0 (socket says 172.21.15.99:35390) 2024-03-29T09:09:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7fefebe00700 1 -- 172.21.15.99:0/79433183 learned_addr learned my addr 172.21.15.99:0/79433183 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7feff0c00700 1 --2- 172.21.15.99:0/79433183 >> v2:172.21.15.133:3300/0 conn(0x7fefec160b30 0x7fefec0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7feff1600700 1 --2- 172.21.15.99:0/79433183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7fefeb400700 1 -- 172.21.15.99:0/79433183 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553125059 0 0) 0x7fefec0d1fa0 con 0x7fefec0b1f50 2024-03-29T09:09:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.637+0000 7fefeb400700 1 -- 172.21.15.99:0/79433183 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd40035b0 con 0x7fefec0b1f50 2024-03-29T09:09:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7fefeb400700 1 -- 172.21.15.99:0/79433183 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3743233728 0 0) 0x7fefd40035b0 con 0x7fefec0b1f50 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7feff1600700 1 -- 172.21.15.99:0/79433183 >> v2:172.21.15.133:3300/0 conn(0x7fefec160b30 msgr2=0x7fefec0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7feff1600700 1 --2- 172.21.15.99:0/79433183 >> v2:172.21.15.133:3300/0 conn(0x7fefec160b30 0x7fefec0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7feff1600700 1 -- 172.21.15.99:0/79433183 >> v1:172.21.15.99:6789/0 conn(0x7fefec0b1f50 legacy=0x7fefec0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7feff1600700 1 -- 172.21.15.99:0/79433183 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec160f10 con 0x7fefec0d0090 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7feff0c00700 1 --2- 172.21.15.99:0/79433183 >> v2:172.21.15.133:3300/0 conn(0x7fefec160b30 0x7fefec0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7feff1600700 1 --2- 172.21.15.99:0/79433183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0ceda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009d90 tx=0x7fefe400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c5edfd3bff794a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7fefeb400700 1 -- 172.21.15.99:0/79433183 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe400e070 con 0x7fefec0d0090 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7fefeb400700 1 -- 172.21.15.99:0/79433183 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4003a20 con 0x7fefec0d0090 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.638+0000 7fefeb400700 1 -- 172.21.15.99:0/79433183 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd8001670 con 0x7fefec0d0090 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 -- 172.21.15.99:0/79433183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 msgr2=0x7fefec0ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 --2- 172.21.15.99:0/79433183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0ceda0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009d90 tx=0x7fefe400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 -- 172.21.15.99:0/79433183 shutdown_connections 2024-03-29T09:09:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 --2- 172.21.15.99:0/79433183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0ceda0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 --2- 172.21.15.99:0/79433183 >> v2:172.21.15.133:3300/0 conn(0x7fefec160b30 0x7fefec0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 -- 172.21.15.99:0/79433183 >> 172.21.15.99:0/79433183 conn(0x7fefec09c4d0 msgr2=0x7fefec0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 -- 172.21.15.99:0/79433183 shutdown_connections 2024-03-29T09:09:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.639+0000 7feff3600700 1 -- 172.21.15.99:0/79433183 wait complete. 2024-03-29T09:09:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.641+0000 7feff3600700 1 Processor -- start 2024-03-29T09:09:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.641+0000 7feff3600700 1 -- start start 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff3600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec0b1f50 0x7fefec0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff3600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff3600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefec160b30 0x7fefec09d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff3600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0ca650 con 0x7fefec0b1f50 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff3600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0ca930 con 0x7fefec160b30 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff3600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefec0cac10 con 0x7fefec0d0090 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec0b1f50 0x7fefec0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec0b1f50 0x7fefec0cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60780/0 (socket says 172.21.15.99:60780) 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.642+0000 7feff0c00700 1 -- 172.21.15.99:0/2791857814 learned_addr learned my addr 172.21.15.99:0/2791857814 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7fefebe00700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff1600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefec160b30 0x7fefec09d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff0c00700 1 -- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 msgr2=0x7fefec0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff0c00700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff0c00700 1 -- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefec160b30 msgr2=0x7fefec09d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff0c00700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefec160b30 0x7fefec09d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff0c00700 1 -- 172.21.15.99:0/2791857814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00094a0 con 0x7fefec0b1f50 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff1600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefec160b30 0x7fefec09d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff0c00700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec0b1f50 0x7fefec0cb8a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefe000ebe0 tx=0x7fefe0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7fefe8c00700 1 -- 172.21.15.99:0/2791857814 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000c8b0 con 0x7fefec0b1f50 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7fefe8c00700 1 -- 172.21.15.99:0/2791857814 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe000ca10 con 0x7fefec0b1f50 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe4009a70 con 0x7fefec0b1f50 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7fefe8c00700 1 -- 172.21.15.99:0/2791857814 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000cda0 con 0x7fefec0b1f50 2024-03-29T09:09:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.643+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec0cb1d0 con 0x7fefec0b1f50 2024-03-29T09:09:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.644+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefec0a0010 con 0x7fefec0b1f50 2024-03-29T09:09:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.645+0000 7fefe8c00700 1 -- 172.21.15.99:0/2791857814 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fefe0013470 con 0x7fefec0b1f50 2024-03-29T09:09:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.646+0000 7fefe8c00700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fefcc06ac90 0x7fefcc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.646+0000 7fefe8c00700 1 -- 172.21.15.99:0/2791857814 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fefe00896b0 con 0x7fefec0b1f50 2024-03-29T09:09:00.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.646+0000 7fefebe00700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fefcc06ac90 0x7fefcc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:00.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.647+0000 7fefebe00700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fefcc06ac90 0x7fefcc06d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001eb0 tx=0x7fefd8001ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:00.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.651+0000 7fefe8c00700 1 -- 172.21.15.99:0/2791857814 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fefe0054650 con 0x7fefec0b1f50 2024-03-29T09:09:00.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:00.852+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fefec0678b0 con 0x7fefec0b1f50 2024-03-29T09:09:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.414+0000 7fefe8c00700 1 -- 172.21.15.99:0/2791857814 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fefe0057c70 con 0x7fefec0b1f50 2024-03-29T09:09:01.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.419+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fefcc06ac90 msgr2=0x7fefcc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.419+0000 7feff3600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fefcc06ac90 0x7fefcc06d150 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001eb0 tx=0x7fefd8001ce0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.419+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec0b1f50 msgr2=0x7fefec0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.419+0000 7feff3600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec0b1f50 0x7fefec0cb8a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fefe000ebe0 tx=0x7fefe0009e20 comp rx=0 tx=0).stop 2024-03-29T09:09:01.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.421+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 shutdown_connections 2024-03-29T09:09:01.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.421+0000 7feff3600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fefcc06ac90 0x7fefcc06d150 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fefd8001eb0 tx=0x7fefd8001ce0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.421+0000 7feff3600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefec160b30 0x7fefec09d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.421+0000 7feff3600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fefec0b1f50 0x7fefec0cb8a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.421+0000 7feff3600700 1 --2- 172.21.15.99:0/2791857814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fefec0d0090 0x7fefec0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.421+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 >> 172.21.15.99:0/2791857814 conn(0x7fefec09c4d0 msgr2=0x7fefec06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:01.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.424+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 shutdown_connections 2024-03-29T09:09:01.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.424+0000 7feff3600700 1 -- 172.21.15.99:0/2791857814 wait complete. 2024-03-29T09:09:01.426 INFO:teuthology.orchestra.run.smithi099.stderr:set pool 4 allow_ec_overwrites to true 2024-03-29T09:09:01.458 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-29T09:09:01.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.650+0000 7f5ab2c00700 1 Processor -- start 2024-03-29T09:09:01.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.650+0000 7f5ab2c00700 1 -- start start 2024-03-29T09:09:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.651+0000 7f5ab2c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5aac160770 0x7f5aac0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:01.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.651+0000 7f5ab2c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5aac0d3550 0x7f5aac0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.651+0000 7f5ab2c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aac0d1520 con 0x7f5aac0d0d60 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.651+0000 7f5ab2c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aac0d1800 con 0x7f5aac0d3550 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.651+0000 7f5ab2c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aac0d1ae0 con 0x7f5aac160770 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5aac0d3550 0x7f5aac0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5aac0d3550 0x7f5aac0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60782/0 (socket says 172.21.15.99:60782) 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 -- 172.21.15.99:0/855996669 learned_addr learned my addr 172.21.15.99:0/855996669 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5aabe00700 1 --2- 172.21.15.99:0/855996669 >> v2:172.21.15.133:3300/0 conn(0x7f5aac160770 0x7f5aac0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 -- 172.21.15.99:0/855996669 >> v2:172.21.15.133:3300/0 conn(0x7f5aac160770 msgr2=0x7f5aac0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 --2- 172.21.15.99:0/855996669 >> v2:172.21.15.133:3300/0 conn(0x7f5aac160770 0x7f5aac0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 -- 172.21.15.99:0/855996669 >> v1:172.21.15.99:6789/0 conn(0x7f5aac0d0d60 legacy=0x7f5aac0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 -- 172.21.15.99:0/855996669 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aac0d1dc0 con 0x7f5aac0d3550 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5aaaa00700 1 -- 172.21.15.99:0/855996669 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794987472 0 0) 0x7f5aac0d1520 con 0x7f5aac0d0d60 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.652+0000 7f5ab0c00700 1 --2- 172.21.15.99:0/855996669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d3550 0x7f5aac0ce080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0b5d20 tx=0x7f5aa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d795e36584b41977 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:01.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5aaaa00700 1 -- 172.21.15.99:0/855996669 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400e070 con 0x7f5aac0d3550 2024-03-29T09:09:01.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5aaaa00700 1 -- 172.21.15.99:0/855996669 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa400b9c0 con 0x7f5aac0d3550 2024-03-29T09:09:01.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5aaaa00700 1 -- 172.21.15.99:0/855996669 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400bd70 con 0x7f5aac0d3550 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 -- 172.21.15.99:0/855996669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d3550 msgr2=0x7f5aac0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/855996669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d3550 0x7f5aac0ce080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0b5d20 tx=0x7f5aa400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 -- 172.21.15.99:0/855996669 shutdown_connections 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/855996669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d3550 0x7f5aac0ce080 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/855996669 >> v2:172.21.15.133:3300/0 conn(0x7f5aac160770 0x7f5aac0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 -- 172.21.15.99:0/855996669 >> 172.21.15.99:0/855996669 conn(0x7f5aac08afd0 msgr2=0x7f5aac09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 -- 172.21.15.99:0/855996669 shutdown_connections 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.653+0000 7f5ab2c00700 1 -- 172.21.15.99:0/855996669 wait complete. 2024-03-29T09:09:01.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.655+0000 7f5ab2c00700 1 Processor -- start 2024-03-29T09:09:01.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.655+0000 7f5ab2c00700 1 -- start start 2024-03-29T09:09:01.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5ab2c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d0d60 0x7f5aac1f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:01.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5ab2c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5aac0d3550 0x7f5aac0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5ab2c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac160770 0x7f5aac06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5ab2c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0cb6b0 con 0x7f5aac0d0d60 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5ab2c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0b6a80 con 0x7f5aac160770 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5ab2c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac0b6d60 con 0x7f5aac0d3550 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5aabe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d0d60 0x7f5aac1f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5aabe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d0d60 0x7f5aac1f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60784/0 (socket says 172.21.15.99:60784) 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5aabe00700 1 -- 172.21.15.99:0/3314727724 learned_addr learned my addr 172.21.15.99:0/3314727724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5ab0c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac160770 0x7f5aac06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.656+0000 7f5aab400700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5aac0d3550 0x7f5aac0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aabe00700 1 -- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5aac0d3550 msgr2=0x7f5aac0b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aabe00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5aac0d3550 0x7f5aac0b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aabe00700 1 -- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac160770 msgr2=0x7f5aac06bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aabe00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac160770 0x7f5aac06bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aabe00700 1 -- 172.21.15.99:0/3314727724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4009b30 con 0x7f5aac0d0d60 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5ab0c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac160770 0x7f5aac06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aab400700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5aac0d3550 0x7f5aac0b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aabe00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d0d60 0x7f5aac1f2290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00d800 tx=0x7f5a9c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aa3e00700 1 -- 172.21.15.99:0/3314727724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0125f0 con 0x7f5aac0d0d60 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aa3e00700 1 -- 172.21.15.99:0/3314727724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9c012750 con 0x7f5aac0d0d60 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac0cb900 con 0x7f5aac0d0d60 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5aa3e00700 1 -- 172.21.15.99:0/3314727724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c012b00 con 0x7f5aac0d0d60 2024-03-29T09:09:01.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.657+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac1f27d0 con 0x7f5aac0d0d60 2024-03-29T09:09:01.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.658+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac0678b0 con 0x7f5aac0d0d60 2024-03-29T09:09:01.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.659+0000 7f5aa3e00700 1 -- 172.21.15.99:0/3314727724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5a9c012c60 con 0x7f5aac0d0d60 2024-03-29T09:09:01.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.660+0000 7f5aa3e00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5a8c06ac90 0x7f5a8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:01.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.660+0000 7f5aa3e00700 1 -- 172.21.15.99:0/3314727724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f5a9c089a70 con 0x7f5aac0d0d60 2024-03-29T09:09:01.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.660+0000 7f5aab400700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5a8c06ac90 0x7f5a8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:01.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.661+0000 7f5aab400700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5a8c06ac90 0x7f5a8c06d150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0b5f40 tx=0x7f5a98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:01.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.664+0000 7f5aa3e00700 1 -- 172.21.15.99:0/3314727724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5a9c0548e0 con 0x7f5aac0d0d60 2024-03-29T09:09:01.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.912+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f5aac0d2f40 con 0x7f5aac0d0d60 2024-03-29T09:09:01.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.918+0000 7f5aa3e00700 1 -- 172.21.15.99:0/3314727724 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7f5a9c057f00 con 0x7f5aac0d0d60 2024-03-29T09:09:01.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5a8c06ac90 msgr2=0x7f5a8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5a8c06ac90 0x7f5a8c06d150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0b5f40 tx=0x7f5a98009040 comp rx=0 tx=0).stop 2024-03-29T09:09:01.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d0d60 msgr2=0x7f5aac1f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:01.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d0d60 0x7f5aac1f2290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00d800 tx=0x7f5a9c00dbc0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 shutdown_connections 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5a8c06ac90 0x7f5a8c06d150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5aac160770 0x7f5aac06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5aac0d0d60 0x7f5aac1f2290 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 --2- 172.21.15.99:0/3314727724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5aac0d3550 0x7f5aac0b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 >> 172.21.15.99:0/3314727724 conn(0x7f5aac08afd0 msgr2=0x7f5aac06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 shutdown_connections 2024-03-29T09:09:01.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:01.921+0000 7f5ab2c00700 1 -- 172.21.15.99:0/3314727724 wait complete. 2024-03-29T09:09:01.923 INFO:teuthology.orchestra.run.smithi099.stderr:added data pool 4 to fsmap 2024-03-29T09:09:01.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:09:02.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.127+0000 7f4411a00700 1 Processor -- start 2024-03-29T09:09:02.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.128+0000 7f4411a00700 1 -- start start 2024-03-29T09:09:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f4411a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f440c1608e0 0x7f440c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f4411a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0cddc0 con 0x7f440c0d1b20 2024-03-29T09:09:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f4411a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0ce0a0 con 0x7f440c160cc0 2024-03-29T09:09:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f4411a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c0ce380 con 0x7f440c1608e0 2024-03-29T09:09:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f440aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f440c1608e0 0x7f440c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f440b400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f440c0d1b20 0x7f440c0afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35398/0 (socket says 172.21.15.99:35398) 2024-03-29T09:09:02.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f440b400700 1 -- 172.21.15.99:0/3090858905 learned_addr learned my addr 172.21.15.99:0/3090858905 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f440aa00700 1 -- 172.21.15.99:0/3090858905 >> v1:172.21.15.133:6789/0 conn(0x7f440c160cc0 legacy=0x7f440c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f440aa00700 1 -- 172.21.15.99:0/3090858905 >> v1:172.21.15.99:6789/0 conn(0x7f440c0d1b20 legacy=0x7f440c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.129+0000 7f440aa00700 1 -- 172.21.15.99:0/3090858905 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0ce660 con 0x7f440c1608e0 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f440aa00700 1 --2- 172.21.15.99:0/3090858905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c1608e0 0x7f440c0d0150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009840 tx=0x7f43fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e11cfc5ce7aceeb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4409600700 1 -- 172.21.15.99:0/3090858905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc004230 con 0x7f440c1608e0 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4409600700 1 -- 172.21.15.99:0/3090858905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc004390 con 0x7f440c1608e0 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4409600700 1 -- 172.21.15.99:0/3090858905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0036a0 con 0x7f440c1608e0 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4411a00700 1 -- 172.21.15.99:0/3090858905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c1608e0 msgr2=0x7f440c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4411a00700 1 --2- 172.21.15.99:0/3090858905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c1608e0 0x7f440c0d0150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009840 tx=0x7f43fc009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4411a00700 1 -- 172.21.15.99:0/3090858905 shutdown_connections 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4411a00700 1 --2- 172.21.15.99:0/3090858905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c1608e0 0x7f440c0d0150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.130+0000 7f4411a00700 1 -- 172.21.15.99:0/3090858905 >> 172.21.15.99:0/3090858905 conn(0x7f440c09c490 msgr2=0x7f440c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.131+0000 7f4411a00700 1 -- 172.21.15.99:0/3090858905 shutdown_connections 2024-03-29T09:09:02.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.131+0000 7f4411a00700 1 -- 172.21.15.99:0/3090858905 wait complete. 2024-03-29T09:09:02.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.132+0000 7f4411a00700 1 Processor -- start 2024-03-29T09:09:02.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.132+0000 7f4411a00700 1 -- start start 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.133+0000 7f4411a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f440c0d1b20 0x7f440c1f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.133+0000 7f4411a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f440c1608e0 0x7f440c0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.133+0000 7f4411a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c160cc0 0x7f440c0b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.133+0000 7f4411a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0b7160 con 0x7f440c160cc0 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.133+0000 7f4411a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0b7580 con 0x7f440c1608e0 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.133+0000 7f4411a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0b64e0 con 0x7f440c0d1b20 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c160cc0 0x7f440c0b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c160cc0 0x7f440c0b7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60814/0 (socket says 172.21.15.99:60814) 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 -- 172.21.15.99:0/3232170417 learned_addr learned my addr 172.21.15.99:0/3232170417 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440aa00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f440c0d1b20 0x7f440c1f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440a000700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f440c1608e0 0x7f440c0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 -- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f440c0d1b20 msgr2=0x7f440c1f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f440c0d1b20 0x7f440c1f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 -- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f440c1608e0 msgr2=0x7f440c0b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f440c1608e0 0x7f440c0b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 -- 172.21.15.99:0/3232170417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0094a0 con 0x7f440c160cc0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f440b400700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c160cc0 0x7f440c0b7cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f440400cbc0 tx=0x7f440400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f4402a00700 1 -- 172.21.15.99:0/3232170417 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440400ab50 con 0x7f440c160cc0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f4402a00700 1 -- 172.21.15.99:0/3232170417 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440400acb0 con 0x7f440c160cc0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c0b67c0 con 0x7f440c160cc0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.134+0000 7f4402a00700 1 -- 172.21.15.99:0/3232170417 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404012600 con 0x7f440c160cc0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.135+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c0b6d80 con 0x7f440c160cc0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.136+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c0b72a0 con 0x7f440c160cc0 2024-03-29T09:09:02.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.136+0000 7f4402a00700 1 -- 172.21.15.99:0/3232170417 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f440401b3b0 con 0x7f440c160cc0 2024-03-29T09:09:02.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.137+0000 7f4402a00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43ec06ac90 0x7f43ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.137+0000 7f4402a00700 1 -- 172.21.15.99:0/3232170417 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4404089270 con 0x7f440c160cc0 2024-03-29T09:09:02.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.137+0000 7f440aa00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43ec06ac90 0x7f43ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.138+0000 7f440aa00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43ec06ac90 0x7f43ec06d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f43fc000c00 tx=0x7f43fc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:02.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.142+0000 7f4402a00700 1 -- 172.21.15.99:0/3232170417 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f440c0b72a0 con 0x7f440c160cc0 2024-03-29T09:09:02.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.393+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f440c0b72a0 con 0x7f440c160cc0 2024-03-29T09:09:02.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.393+0000 7f4402a00700 1 -- 172.21.15.99:0/3232170417 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f440c0b72a0 con 0x7f440c160cc0 2024-03-29T09:09:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:02.393 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:01.914465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T09:09:02.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43ec06ac90 msgr2=0x7f43ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43ec06ac90 0x7f43ec06d150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f43fc000c00 tx=0x7f43fc01c040 comp rx=0 tx=0).stop 2024-03-29T09:09:02.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c160cc0 msgr2=0x7f440c0b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c160cc0 0x7f440c0b7cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f440400cbc0 tx=0x7f440400cf80 comp rx=0 tx=0).stop 2024-03-29T09:09:02.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 shutdown_connections 2024-03-29T09:09:02.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f43ec06ac90 0x7f43ec06d150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c160cc0 0x7f440c0b7cc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f440c0d1b20 0x7f440c1f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 --2- 172.21.15.99:0/3232170417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f440c1608e0 0x7f440c0b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 >> 172.21.15.99:0/3232170417 conn(0x7f440c09c490 msgr2=0x7f440c0d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 shutdown_connections 2024-03-29T09:09:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.396+0000 7f4411a00700 1 -- 172.21.15.99:0/3232170417 wait complete. 2024-03-29T09:09:02.397 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-03-29T09:09:02.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:02.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.603+0000 7f9de8e00700 1 Processor -- start 2024-03-29T09:09:02.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.603+0000 7f9de8e00700 1 -- start start 2024-03-29T09:09:02.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.604+0000 7f9de8e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9de41608e0 0x7f9de40d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.604+0000 7f9de8e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9de40d1b20 0x7f9de40ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de8e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de40cddc0 con 0x7f9de4160cc0 2024-03-29T09:09:02.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de8e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de40ce0a0 con 0x7f9de40d1b20 2024-03-29T09:09:02.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de8e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de40ce380 con 0x7f9de41608e0 2024-03-29T09:09:02.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9de40d1b20 0x7f9de40ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de1e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9de41608e0 0x7f9de40d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9de40d1b20 0x7f9de40ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60826/0 (socket says 172.21.15.99:60826) 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de1e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9de41608e0 0x7f9de40d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:48358/0 (socket says 172.21.15.99:48358) 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 -- 172.21.15.99:0/29909660 learned_addr learned my addr 172.21.15.99:0/29909660 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de1400700 1 --1- 172.21.15.99:0/29909660 >> v1:172.21.15.133:6789/0 conn(0x7f9de4160cc0 0x7f9de40d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:41722/0 (socket says 172.21.15.99:41722) 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 -- 172.21.15.99:0/29909660 >> v2:172.21.15.133:3300/0 conn(0x7f9de41608e0 msgr2=0x7f9de40d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 --2- 172.21.15.99:0/29909660 >> v2:172.21.15.133:3300/0 conn(0x7f9de41608e0 0x7f9de40d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 -- 172.21.15.99:0/29909660 >> v1:172.21.15.133:6789/0 conn(0x7f9de4160cc0 legacy=0x7f9de40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 -- 172.21.15.99:0/29909660 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40ce660 con 0x7f9de40d1b20 2024-03-29T09:09:02.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.605+0000 7f9de2800700 1 --2- 172.21.15.99:0/29909660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de40d1b20 0x7f9de40ceb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9de408d790 tx=0x7f9ddc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fca706e6834fc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de0a00700 1 -- 172.21.15.99:0/29909660 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00e070 con 0x7f9de40d1b20 2024-03-29T09:09:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de0a00700 1 -- 172.21.15.99:0/29909660 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc00b910 con 0x7f9de40d1b20 2024-03-29T09:09:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de0a00700 1 -- 172.21.15.99:0/29909660 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc00bcc0 con 0x7f9de40d1b20 2024-03-29T09:09:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 -- 172.21.15.99:0/29909660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de40d1b20 msgr2=0x7f9de40ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 --2- 172.21.15.99:0/29909660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de40d1b20 0x7f9de40ceb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9de408d790 tx=0x7f9ddc00b330 comp rx=0 tx=0).stop 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 -- 172.21.15.99:0/29909660 shutdown_connections 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 --2- 172.21.15.99:0/29909660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de40d1b20 0x7f9de40ceb80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 --2- 172.21.15.99:0/29909660 >> v2:172.21.15.133:3300/0 conn(0x7f9de41608e0 0x7f9de40d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 -- 172.21.15.99:0/29909660 >> 172.21.15.99:0/29909660 conn(0x7f9de409c490 msgr2=0x7f9de40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 -- 172.21.15.99:0/29909660 shutdown_connections 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.606+0000 7f9de8e00700 1 -- 172.21.15.99:0/29909660 wait complete. 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.608+0000 7f9de8e00700 1 Processor -- start 2024-03-29T09:09:02.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.608+0000 7f9de8e00700 1 -- start start 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de8e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de40d1b20 0x7f9de40b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de8e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9de41608e0 0x7f9de415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de4160cc0 0x7f9de40b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de8e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40b5950 con 0x7f9de4160cc0 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de8e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40b4d40 con 0x7f9de40d1b20 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de8e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de40b5020 con 0x7f9de41608e0 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de4160cc0 0x7f9de40b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de1e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de40d1b20 0x7f9de40b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de4160cc0 0x7f9de40b2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60830/0 (socket says 172.21.15.99:60830) 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de2800700 1 -- 172.21.15.99:0/1508756284 learned_addr learned my addr 172.21.15.99:0/1508756284 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.609+0000 7f9de1400700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9de41608e0 0x7f9de415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de2800700 1 -- 172.21.15.99:0/1508756284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9de41608e0 msgr2=0x7f9de415c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de2800700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9de41608e0 0x7f9de415c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de2800700 1 -- 172.21.15.99:0/1508756284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de40d1b20 msgr2=0x7f9de40b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de2800700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de40d1b20 0x7f9de40b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de2800700 1 -- 172.21.15.99:0/1508756284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc009b30 con 0x7f9de4160cc0 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de2800700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de4160cc0 0x7f9de40b2420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9de408c9a0 tx=0x7f9ddc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9dda000700 1 -- 172.21.15.99:0/1508756284 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc004880 con 0x7f9de4160cc0 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9dda000700 1 -- 172.21.15.99:0/1508756284 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc0099b0 con 0x7f9de4160cc0 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de40b5300 con 0x7f9de4160cc0 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9dda000700 1 -- 172.21.15.99:0/1508756284 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc01c680 con 0x7f9de4160cc0 2024-03-29T09:09:02.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.610+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de40b8a20 con 0x7f9de4160cc0 2024-03-29T09:09:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.611+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de40b5b00 con 0x7f9de4160cc0 2024-03-29T09:09:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.612+0000 7f9dda000700 1 -- 172.21.15.99:0/1508756284 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ddc01c7e0 con 0x7f9de4160cc0 2024-03-29T09:09:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.613+0000 7f9dda000700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9dc406ac90 0x7f9dc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:02.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.614+0000 7f9dda000700 1 -- 172.21.15.99:0/1508756284 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ddc08b240 con 0x7f9de4160cc0 2024-03-29T09:09:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.614+0000 7f9de1e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9dc406ac90 0x7f9dc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:02.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.614+0000 7f9de1e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9dc406ac90 0x7f9dc406d150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9de415d7b0 tx=0x7f9dd4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:02.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.618+0000 7f9dda000700 1 -- 172.21.15.99:0/1508756284 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9de40b5b00 con 0x7f9de4160cc0 2024-03-29T09:09:02.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.822+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9de40677a0 con 0x7f9de4160cc0 2024-03-29T09:09:02.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.823+0000 7f9dda000700 1 -- 172.21.15.99:0/1508756284 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f9ddc056120 con 0x7f9de4160cc0 2024-03-29T09:09:02.823 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:02.824 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:02.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9dc406ac90 msgr2=0x7f9dc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9dc406ac90 0x7f9dc406d150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9de415d7b0 tx=0x7f9dd4009210 comp rx=0 tx=0).stop 2024-03-29T09:09:02.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de4160cc0 msgr2=0x7f9de40b2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:02.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de4160cc0 0x7f9de40b2420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9de408c9a0 tx=0x7f9ddc004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 shutdown_connections 2024-03-29T09:09:02.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9dc406ac90 0x7f9dc406d150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9de4160cc0 0x7f9de40b2420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9de40d1b20 0x7f9de40b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 --2- 172.21.15.99:0/1508756284 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9de41608e0 0x7f9de415c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:02.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 >> 172.21.15.99:0/1508756284 conn(0x7f9de409c490 msgr2=0x7f9de415f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:02.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 shutdown_connections 2024-03-29T09:09:02.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:02.826+0000 7f9de8e00700 1 -- 172.21.15.99:0/1508756284 wait complete. 2024-03-29T09:09:02.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:03.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.033+0000 7f3ca6a00700 1 Processor -- start 2024-03-29T09:09:03.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.033+0000 7f3ca6a00700 1 -- start start 2024-03-29T09:09:03.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3ca6a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ca01608e0 0x7f3ca00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3ca6a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca00d1b20 0x7f3ca00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3ca6a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ca00cddc0 con 0x7f3ca0160cc0 2024-03-29T09:09:03.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3ca6a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca00ce0a0 con 0x7f3ca01608e0 2024-03-29T09:09:03.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3ca6a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00ce380 con 0x7f3ca00d1b20 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3c9fe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ca01608e0 0x7f3ca00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3c97e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3ca0160cc0 0x7f3ca00d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35404/0 (socket says 172.21.15.99:35404) 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3c9fe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ca01608e0 0x7f3ca00d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60838/0 (socket says 172.21.15.99:60838) 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.034+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3344069561 learned_addr learned my addr 172.21.15.99:0/3344069561 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3ca4a00700 1 --2- 172.21.15.99:0/3344069561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca00d1b20 0x7f3ca00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3344069561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca00d1b20 msgr2=0x7f3ca00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9fe00700 1 --2- 172.21.15.99:0/3344069561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca00d1b20 0x7f3ca00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9f400700 1 -- 172.21.15.99:0/3344069561 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497547806 0 0) 0x7f3ca00cddc0 con 0x7f3ca0160cc0 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3344069561 >> v1:172.21.15.99:6789/0 conn(0x7f3ca0160cc0 legacy=0x7f3ca00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3344069561 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00ce660 con 0x7f3ca01608e0 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3ca4a00700 1 --2- 172.21.15.99:0/3344069561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca00d1b20 0x7f3ca00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9fe00700 1 --2- 172.21.15.99:0/3344069561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca01608e0 0x7f3ca00d0150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c900098f0 tx=0x7f3c90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8122b228156f011 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9f400700 1 -- 172.21.15.99:0/3344069561 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000b8a0 con 0x7f3ca01608e0 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9f400700 1 -- 172.21.15.99:0/3344069561 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9000ba00 con 0x7f3ca01608e0 2024-03-29T09:09:03.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.035+0000 7f3c9f400700 1 -- 172.21.15.99:0/3344069561 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9000bd90 con 0x7f3ca01608e0 2024-03-29T09:09:03.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3344069561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca01608e0 msgr2=0x7f3ca00d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3344069561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca01608e0 0x7f3ca00d0150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3c900098f0 tx=0x7f3c90009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:03.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3344069561 shutdown_connections 2024-03-29T09:09:03.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3344069561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca00d1b20 0x7f3ca00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3344069561 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca01608e0 0x7f3ca00d0150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3344069561 >> 172.21.15.99:0/3344069561 conn(0x7f3ca009c490 msgr2=0x7f3ca00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:03.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3344069561 shutdown_connections 2024-03-29T09:09:03.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.036+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3344069561 wait complete. 2024-03-29T09:09:03.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.038+0000 7f3ca6a00700 1 Processor -- start 2024-03-29T09:09:03.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.038+0000 7f3ca6a00700 1 -- start start 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3ca6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca00d1b20 0x7f3ca014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3ca6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca01608e0 0x7f3ca00cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3ca6a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca0160cc0 0x7f3ca0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3ca6a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00cc3d0 con 0x7f3ca00d1b20 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3ca6a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca00c9610 con 0x7f3ca01608e0 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3ca6a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca014c550 con 0x7f3ca0160cc0 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca00d1b20 0x7f3ca014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca00d1b20 0x7f3ca014c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60848/0 (socket says 172.21.15.99:60848) 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3898095903 learned_addr learned my addr 172.21.15.99:0/3898095903 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3898095903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca0160cc0 msgr2=0x7f3ca0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca0160cc0 0x7f3ca0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3898095903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca01608e0 msgr2=0x7f3ca00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca01608e0 0x7f3ca00cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.039+0000 7f3c9fe00700 1 -- 172.21.15.99:0/3898095903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90009550 con 0x7f3ca00d1b20 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.040+0000 7f3c9fe00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca00d1b20 0x7f3ca014c010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ca008c9a0 tx=0x7f3c90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.040+0000 7f3c9cc00700 1 -- 172.21.15.99:0/3898095903 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90004480 con 0x7f3ca00d1b20 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.040+0000 7f3c9cc00700 1 -- 172.21.15.99:0/3898095903 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9001f070 con 0x7f3ca00d1b20 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.040+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca00cb750 con 0x7f3ca00d1b20 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.040+0000 7f3c9cc00700 1 -- 172.21.15.99:0/3898095903 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c900038a0 con 0x7f3ca00d1b20 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.040+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca00cbd10 con 0x7f3ca00d1b20 2024-03-29T09:09:03.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.041+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca00cc640 con 0x7f3ca00d1b20 2024-03-29T09:09:03.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.042+0000 7f3c9cc00700 1 -- 172.21.15.99:0/3898095903 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3c90021030 con 0x7f3ca00d1b20 2024-03-29T09:09:03.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.042+0000 7f3c9cc00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3c7c06ac90 0x7f3c7c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.043+0000 7f3c9cc00700 1 -- 172.21.15.99:0/3898095903 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3c9008af30 con 0x7f3ca00d1b20 2024-03-29T09:09:03.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.043+0000 7f3c97e00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3c7c06ac90 0x7f3c7c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.043+0000 7f3c97e00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3c7c06ac90 0x7f3c7c06d150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ca01f6480 tx=0x7f3c880079f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.047+0000 7f3c9cc00700 1 -- 172.21.15.99:0/3898095903 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3c90055da0 con 0x7f3ca00d1b20 2024-03-29T09:09:03.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.251+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ca00677a0 con 0x7f3ca00d1b20 2024-03-29T09:09:03.252 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.252+0000 7f3c9cc00700 1 -- 172.21.15.99:0/3898095903 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f3c90015020 con 0x7f3ca00d1b20 2024-03-29T09:09:03.253 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:03.253 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:03.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3c7c06ac90 msgr2=0x7f3c7c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3c7c06ac90 0x7f3c7c06d150 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ca01f6480 tx=0x7f3c880079f0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca00d1b20 msgr2=0x7f3ca014c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca00d1b20 0x7f3ca014c010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ca008c9a0 tx=0x7f3c90004750 comp rx=0 tx=0).stop 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 shutdown_connections 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3c7c06ac90 0x7f3c7c06d150 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ca0160cc0 0x7f3ca0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ca00d1b20 0x7f3ca014c010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 --2- 172.21.15.99:0/3898095903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ca01608e0 0x7f3ca00cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 >> 172.21.15.99:0/3898095903 conn(0x7f3ca009c490 msgr2=0x7f3ca00d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 shutdown_connections 2024-03-29T09:09:03.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.255+0000 7f3ca6a00700 1 -- 172.21.15.99:0/3898095903 wait complete. 2024-03-29T09:09:03.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:09:03.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.475+0000 7f5c7c800700 1 Processor -- start 2024-03-29T09:09:03.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.475+0000 7f5c7c800700 1 -- start start 2024-03-29T09:09:03.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.476+0000 7f5c7c800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c741608e0 0x7f5c740d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.476+0000 7f5c7c800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 0x7f5c740ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.476+0000 7f5c7c800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c740cddc0 con 0x7f5c74160cc0 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.476+0000 7f5c7c800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740ce0a0 con 0x7f5c741608e0 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.476+0000 7f5c7c800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740ce380 con 0x7f5c740d1b20 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.476+0000 7f5c79400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5c74160cc0 0x7f5c740d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35406/0 (socket says 172.21.15.99:35406) 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.476+0000 7f5c79400700 1 -- 172.21.15.99:0/2968214939 learned_addr learned my addr 172.21.15.99:0/2968214939 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c7a800700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 0x7f5c740ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c79e00700 1 --2- 172.21.15.99:0/2968214939 >> v2:172.21.15.133:3300/0 conn(0x7f5c741608e0 0x7f5c740d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204257758 0 0) 0x7f5c740cddc0 con 0x7f5c74160cc0 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c5c0035b0 con 0x7f5c74160cc0 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2159605207 0 0) 0x7f5c5c0035b0 con 0x7f5c74160cc0 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 msgr2=0x7f5c740ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 0x7f5c740ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 >> v2:172.21.15.133:3300/0 conn(0x7f5c741608e0 msgr2=0x7f5c740d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 --2- 172.21.15.99:0/2968214939 >> v2:172.21.15.133:3300/0 conn(0x7f5c741608e0 0x7f5c740d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c740ce660 con 0x7f5c74160cc0 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f5c64001670 con 0x7f5c74160cc0 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 >> v1:172.21.15.99:6789/0 conn(0x7f5c74160cc0 legacy=0x7f5c740d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c5c003a00 0x7f5c5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 0x7f5c5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5c004d90 0x7f5c5c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740ce0a0 con 0x7f5c5c004d90 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740ce380 con 0x7f5c5c003a00 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.477+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c00ba10 con 0x7f5c740d1b20 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79e00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c5c003a00 0x7f5c5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c7a800700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5c004d90 0x7f5c5c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79400700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 0x7f5c5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79e00700 1 -- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 msgr2=0x7f5c5c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79e00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c740d1b20 0x7f5c5c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79e00700 1 -- 172.21.15.99:0/2968214939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5c004d90 msgr2=0x7f5c5c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79e00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c5c004d90 0x7f5c5c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79e00700 1 -- 172.21.15.99:0/2968214939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0035b0 con 0x7f5c5c003a00 2024-03-29T09:09:03.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c79e00700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c5c003a00 0x7f5c5c003e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b60 tx=0x7f5c5c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f895a10d6cae4786 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.478+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800e6c0 con 0x7f5c5c003a00 2024-03-29T09:09:03.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800e820 con 0x7f5c5c003a00 2024-03-29T09:09:03.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c78a00700 1 -- 172.21.15.99:0/2968214939 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800eb90 con 0x7f5c5c003a00 2024-03-29T09:09:03.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c7c800700 1 -- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c5c003a00 msgr2=0x7f5c5c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c7c800700 1 --2- 172.21.15.99:0/2968214939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c5c003a00 0x7f5c5c003e20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b60 tx=0x7f5c5c005880 comp rx=0 tx=0).stop 2024-03-29T09:09:03.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c7a800700 1 -- 172.21.15.99:0/2968214939 reap_dead start 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c7c800700 1 -- 172.21.15.99:0/2968214939 shutdown_connections 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c7c800700 1 -- 172.21.15.99:0/2968214939 >> 172.21.15.99:0/2968214939 conn(0x7f5c7409c490 msgr2=0x7f5c740d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c7c800700 1 -- 172.21.15.99:0/2968214939 shutdown_connections 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.479+0000 7f5c7c800700 1 -- 172.21.15.99:0/2968214939 wait complete. 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.481+0000 7f5c7c800700 1 Processor -- start 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.481+0000 7f5c7c800700 1 -- start start 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c7c800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c5c003a00 0x7f5c74158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c7c800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c74148f20 0x7f5c7415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c7c800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c74148260 0x7f5c741486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c7c800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74148c20 con 0x7f5c74148f20 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c7c800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740ca350 con 0x7f5c74148260 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c7c800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740ca5f0 con 0x7f5c5c003a00 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c79400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c74148f20 0x7f5c7415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c79400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c74148f20 0x7f5c7415c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60868/0 (socket says 172.21.15.99:60868) 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c79400700 1 -- 172.21.15.99:0/3851670771 learned_addr learned my addr 172.21.15.99:0/3851670771 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c79e00700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c5c003a00 0x7f5c74158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.482+0000 7f5c7a800700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c74148260 0x7f5c741486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c79400700 1 -- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c5c003a00 msgr2=0x7f5c74158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c79400700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c5c003a00 0x7f5c74158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c79400700 1 -- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c74148260 msgr2=0x7f5c741486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c79400700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c74148260 0x7f5c741486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c79400700 1 -- 172.21.15.99:0/3851670771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74157e80 con 0x7f5c74148f20 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c79400700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c74148f20 0x7f5c7415c5b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c64007340 tx=0x7f5c64007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c6e000700 1 -- 172.21.15.99:0/3851670771 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400f6c0 con 0x7f5c74148f20 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c6e000700 1 -- 172.21.15.99:0/3851670771 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6400f840 con 0x7f5c74148f20 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c74158160 con 0x7f5c74148f20 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c6e000700 1 -- 172.21.15.99:0/3851670771 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6400fbf0 con 0x7f5c74148f20 2024-03-29T09:09:03.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.483+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c740ca8d0 con 0x7f5c74148f20 2024-03-29T09:09:03.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.484+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c741583a0 con 0x7f5c74148f20 2024-03-29T09:09:03.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.485+0000 7f5c6e000700 1 -- 172.21.15.99:0/3851670771 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5c6400fd50 con 0x7f5c74148f20 2024-03-29T09:09:03.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.487+0000 7f5c6e000700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c5806ac90 0x7f5c5806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.487+0000 7f5c6e000700 1 -- 172.21.15.99:0/3851670771 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5c64089610 con 0x7f5c74148f20 2024-03-29T09:09:03.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.487+0000 7f5c79e00700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c5806ac90 0x7f5c5806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.488+0000 7f5c79e00700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c5806ac90 0x7f5c5806d150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c680090b0 tx=0x7f5c6801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.491+0000 7f5c6e000700 1 -- 172.21.15.99:0/3851670771 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5c741583a0 con 0x7f5c74148f20 2024-03-29T09:09:03.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.738+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c741583a0 con 0x7f5c74148f20 2024-03-29T09:09:03.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.739+0000 7f5c6e000700 1 -- 172.21.15.99:0/3851670771 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f5c741583a0 con 0x7f5c74148f20 2024-03-29T09:09:03.739 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:03.739 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:01.914465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-29T09:09:03.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c5806ac90 msgr2=0x7f5c5806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c5806ac90 0x7f5c5806d150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c680090b0 tx=0x7f5c6801f040 comp rx=0 tx=0).stop 2024-03-29T09:09:03.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c74148f20 msgr2=0x7f5c7415c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c74148f20 0x7f5c7415c5b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5c64007340 tx=0x7f5c64007650 comp rx=0 tx=0).stop 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 shutdown_connections 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c5806ac90 0x7f5c5806d150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c74148260 0x7f5c741486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c5c003a00 0x7f5c74158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 --2- 172.21.15.99:0/3851670771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c74148f20 0x7f5c7415c5b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 >> 172.21.15.99:0/3851670771 conn(0x7f5c7409c490 msgr2=0x7f5c740cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 shutdown_connections 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.741+0000 7f5c7c800700 1 -- 172.21.15.99:0/3851670771 wait complete. 2024-03-29T09:09:03.742 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 6 2024-03-29T09:09:03.759 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:03.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.950+0000 7fd4ac200700 1 Processor -- start 2024-03-29T09:09:03.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.950+0000 7fd4ac200700 1 -- start start 2024-03-29T09:09:03.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.951+0000 7fd4ac200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd4a4160cc0 0x7fd4a40d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.951+0000 7fd4ac200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a40cddc0 con 0x7fd4a41608e0 2024-03-29T09:09:03.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.951+0000 7fd4ac200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a40ce0a0 con 0x7fd4a40d1b20 2024-03-29T09:09:03.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.951+0000 7fd4ac200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a40ce380 con 0x7fd4a4160cc0 2024-03-29T09:09:03.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.951+0000 7fd4a9800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd4a41608e0 0x7fd4a40d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35412/0 (socket says 172.21.15.99:35412) 2024-03-29T09:09:03.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.951+0000 7fd4a9800700 1 -- 172.21.15.99:0/3674978855 learned_addr learned my addr 172.21.15.99:0/3674978855 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:03.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4aa200700 1 --2- 172.21.15.99:0/3674978855 >> v2:172.21.15.133:3300/0 conn(0x7fd4a4160cc0 0x7fd4a40d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291612267 0 0) 0x7fd4a40cddc0 con 0x7fd4a41608e0 2024-03-29T09:09:03.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd48c0035b0 con 0x7fd4a41608e0 2024-03-29T09:09:03.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3770341956 0 0) 0x7fd48c0035b0 con 0x7fd4a41608e0 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 >> v2:172.21.15.133:3300/0 conn(0x7fd4a4160cc0 msgr2=0x7fd4a40d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 --2- 172.21.15.99:0/3674978855 >> v2:172.21.15.133:3300/0 conn(0x7fd4a4160cc0 0x7fd4a40d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 >> v1:172.21.15.133:6789/0 conn(0x7fd4a40d1b20 legacy=0x7fd4a40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a40ce660 con 0x7fd4a41608e0 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fd498001670 con 0x7fd4a41608e0 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 >> v1:172.21.15.99:6789/0 conn(0x7fd4a41608e0 legacy=0x7fd4a40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c003a00 0x7fd48c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c004850 0x7fd48c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd48c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40ce380 con 0x7fd48c005210 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c00c0c0 con 0x7fd48c004850 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.952+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c00c280 con 0x7fd48c003a00 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd48c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4a9800700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c003a00 0x7fd48c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4a8e00700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c004850 0x7fd48c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 -- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c003a00 msgr2=0x7fd48c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c003a00 0x7fd48c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 -- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c004850 msgr2=0x7fd48c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c004850 0x7fd48c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 -- 172.21.15.99:0/3674978855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0035b0 con 0x7fd48c005210 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 -- 172.21.15.99:0/3674978855 reap_dead start 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4aa200700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd48c00ba10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009fc0 tx=0x7fd48c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4205e57420697 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c003a20 con 0x7fd48c005210 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c00f040 con 0x7fd48c005210 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.953+0000 7fd4a3e00700 1 -- 172.21.15.99:0/3674978855 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c003da0 con 0x7fd48c005210 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.954+0000 7fd4ac200700 1 -- 172.21.15.99:0/3674978855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 msgr2=0x7fd48c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.954+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd48c00ba10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009fc0 tx=0x7fd48c005880 comp rx=0 tx=0).stop 2024-03-29T09:09:03.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.954+0000 7fd4ac200700 1 -- 172.21.15.99:0/3674978855 shutdown_connections 2024-03-29T09:09:03.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.954+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3674978855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd48c00ba10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.954+0000 7fd4ac200700 1 -- 172.21.15.99:0/3674978855 >> 172.21.15.99:0/3674978855 conn(0x7fd4a409c490 msgr2=0x7fd4a40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:03.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.954+0000 7fd4ac200700 1 -- 172.21.15.99:0/3674978855 shutdown_connections 2024-03-29T09:09:03.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.954+0000 7fd4ac200700 1 -- 172.21.15.99:0/3674978855 wait complete. 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.956+0000 7fd4ac200700 1 Processor -- start 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.956+0000 7fd4ac200700 1 -- start start 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4ac200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd4a4156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4ac200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a41535f0 0x7fd4a40cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4ac200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4a40ccb40 0x7fd4a40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4ac200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a415a020 con 0x7fd48c005210 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4ac200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a4154ab0 con 0x7fd4a41535f0 2024-03-29T09:09:03.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4ac200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40caf70 con 0x7fd4a40ccb40 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd4a4156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd4a4156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60882/0 (socket says 172.21.15.99:60882) 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 -- 172.21.15.99:0/3131309192 learned_addr learned my addr 172.21.15.99:0/3131309192 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4aa200700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4a40ccb40 0x7fd4a40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a8e00700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a41535f0 0x7fd4a40cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 -- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4a40ccb40 msgr2=0x7fd4a40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4a40ccb40 0x7fd4a40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 -- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a41535f0 msgr2=0x7fd4a40cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a41535f0 0x7fd4a40cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.957+0000 7fd4a9800700 1 -- 172.21.15.99:0/3131309192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a415a270 con 0x7fd48c005210 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.958+0000 7fd4a9800700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd4a4156850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd498007290 tx=0x7fd4980075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.958+0000 7fd4a1600700 1 -- 172.21.15.99:0/3131309192 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498011580 con 0x7fd48c005210 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.958+0000 7fd4a1600700 1 -- 172.21.15.99:0/3131309192 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4980116e0 con 0x7fd48c005210 2024-03-29T09:09:03.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.958+0000 7fd4a1600700 1 -- 172.21.15.99:0/3131309192 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498011a90 con 0x7fd48c005210 2024-03-29T09:09:03.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.958+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a415a550 con 0x7fd48c005210 2024-03-29T09:09:03.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.958+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a40cab90 con 0x7fd48c005210 2024-03-29T09:09:03.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.959+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a415a710 con 0x7fd48c005210 2024-03-29T09:09:03.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.960+0000 7fd4a1600700 1 -- 172.21.15.99:0/3131309192 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd498011bf0 con 0x7fd48c005210 2024-03-29T09:09:03.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.961+0000 7fd4a1600700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd48406ac90 0x7fd48406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:03.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.961+0000 7fd4a1600700 1 -- 172.21.15.99:0/3131309192 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4980895e0 con 0x7fd48c005210 2024-03-29T09:09:03.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.961+0000 7fd4a8e00700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd48406ac90 0x7fd48406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:03.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.961+0000 7fd4a8e00700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd48406ac90 0x7fd48406d150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4153a70 tx=0x7fd490009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:03.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:03.965+0000 7fd4a1600700 1 -- 172.21.15.99:0/3131309192 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd4980544c0 con 0x7fd48c005210 2024-03-29T09:09:04.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.167+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4a40677a0 con 0x7fd48c005210 2024-03-29T09:09:04.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.169+0000 7fd4a1600700 1 -- 172.21.15.99:0/3131309192 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7fd498057ae0 con 0x7fd48c005210 2024-03-29T09:09:04.169 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:04.169 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd48406ac90 msgr2=0x7fd48406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd48406ac90 0x7fd48406d150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4153a70 tx=0x7fd490009040 comp rx=0 tx=0).stop 2024-03-29T09:09:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 msgr2=0x7fd4a4156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd4a4156850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd498007290 tx=0x7fd4980075a0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 shutdown_connections 2024-03-29T09:09:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd48406ac90 0x7fd48406d150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4a40ccb40 0x7fd4a40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c005210 0x7fd4a4156850 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 --2- 172.21.15.99:0/3131309192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4a41535f0 0x7fd4a40cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 >> 172.21.15.99:0/3131309192 conn(0x7fd4a409c490 msgr2=0x7fd4a40c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 shutdown_connections 2024-03-29T09:09:04.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.171+0000 7fd4ac200700 1 -- 172.21.15.99:0/3131309192 wait complete. 2024-03-29T09:09:04.593 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T09:08:57.386923+0000', 'modified': '2024-03-29T09:09:01.914465+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6835/440339259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 440339259}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 440339259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-29T09:09:04.593 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T09:09:04.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-29T09:09:04.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.783+0000 7fc3a3800700 1 Processor -- start 2024-03-29T09:09:04.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.783+0000 7fc3a3800700 1 -- start start 2024-03-29T09:09:04.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.784+0000 7fc3a3800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c1608e0 0x7fc39c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.784+0000 7fc3a3800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc39c160cc0 0x7fc39c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.784+0000 7fc3a3800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc39c0cddc0 con 0x7fc39c0d3a70 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc3a3800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc39c0ce0a0 con 0x7fc39c160cc0 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc3a3800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c0ce380 con 0x7fc39c1608e0 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc39c160cc0 0x7fc39c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc39c160cc0 0x7fc39c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60884/0 (socket says 172.21.15.99:60884) 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 -- 172.21.15.99:0/1947017509 learned_addr learned my addr 172.21.15.99:0/1947017509 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/1947017509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c1608e0 0x7fc39c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 -- 172.21.15.99:0/1947017509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c1608e0 msgr2=0x7fc39c0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 --2- 172.21.15.99:0/1947017509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c1608e0 0x7fc39c0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 -- 172.21.15.99:0/1947017509 >> v1:172.21.15.133:6789/0 conn(0x7fc39c0d3a70 legacy=0x7fc39c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 -- 172.21.15.99:0/1947017509 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c0ce660 con 0x7fc39c160cc0 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/1947017509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c1608e0 0x7fc39c0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:04.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.785+0000 7fc39be00700 1 --2- 172.21.15.99:0/1947017509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c160cc0 0x7fc39c0ceb80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098f0 tx=0x7fc388009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0934261a5b3341 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:04.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc39b400700 1 -- 172.21.15.99:0/1947017509 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38800b8a0 con 0x7fc39c160cc0 2024-03-29T09:09:04.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc39b400700 1 -- 172.21.15.99:0/1947017509 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38800ba00 con 0x7fc39c160cc0 2024-03-29T09:09:04.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc39b400700 1 -- 172.21.15.99:0/1947017509 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38800bd90 con 0x7fc39c160cc0 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 -- 172.21.15.99:0/1947017509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c160cc0 msgr2=0x7fc39c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 --2- 172.21.15.99:0/1947017509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c160cc0 0x7fc39c0ceb80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc3880098f0 tx=0x7fc388009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 -- 172.21.15.99:0/1947017509 shutdown_connections 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 --2- 172.21.15.99:0/1947017509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c1608e0 0x7fc39c0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 --2- 172.21.15.99:0/1947017509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c160cc0 0x7fc39c0ceb80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 -- 172.21.15.99:0/1947017509 >> 172.21.15.99:0/1947017509 conn(0x7fc39c09c490 msgr2=0x7fc39c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 -- 172.21.15.99:0/1947017509 shutdown_connections 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.786+0000 7fc3a3800700 1 -- 172.21.15.99:0/1947017509 wait complete. 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.788+0000 7fc3a3800700 1 Processor -- start 2024-03-29T09:09:04.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.788+0000 7fc3a3800700 1 -- start start 2024-03-29T09:09:04.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a3800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc39c0d3a70 0x7fc39c0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a3800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c1608e0 0x7fc39c1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a3800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c160cc0 0x7fc39c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a3800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c158960 con 0x7fc39c1608e0 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a3800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c157a30 con 0x7fc39c0d3a70 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a3800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c157ce0 con 0x7fc39c160cc0 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc39be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c1608e0 0x7fc39c1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc39be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c1608e0 0x7fc39c1574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60894/0 (socket says 172.21.15.99:60894) 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc39be00700 1 -- 172.21.15.99:0/970041309 learned_addr learned my addr 172.21.15.99:0/970041309 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a1800700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c160cc0 0x7fc39c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.789+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc39c0d3a70 0x7fc39c0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc39be00700 1 -- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c160cc0 msgr2=0x7fc39c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc39be00700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c160cc0 0x7fc39c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc39be00700 1 -- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc39c0d3a70 msgr2=0x7fc39c0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc39be00700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc39c0d3a70 0x7fc39c0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc39be00700 1 -- 172.21.15.99:0/970041309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388009550 con 0x7fc39c1608e0 2024-03-29T09:09:04.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc39be00700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c1608e0 0x7fc39c1574f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc388009c40 tx=0x7fc388004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:04.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc398c00700 1 -- 172.21.15.99:0/970041309 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388004510 con 0x7fc39c1608e0 2024-03-29T09:09:04.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc398c00700 1 -- 172.21.15.99:0/970041309 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38801e070 con 0x7fc39c1608e0 2024-03-29T09:09:04.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc398c00700 1 -- 172.21.15.99:0/970041309 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880038e0 con 0x7fc39c1608e0 2024-03-29T09:09:04.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39c157fc0 con 0x7fc39c1608e0 2024-03-29T09:09:04.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.790+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c158580 con 0x7fc39c1608e0 2024-03-29T09:09:04.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.792+0000 7fc398c00700 1 -- 172.21.15.99:0/970041309 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc388003a40 con 0x7fc39c1608e0 2024-03-29T09:09:04.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.793+0000 7fc398c00700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc37c06ac90 0x7fc37c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.793+0000 7fc398c00700 1 -- 172.21.15.99:0/970041309 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc388020070 con 0x7fc39c1608e0 2024-03-29T09:09:04.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.794+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc37c06ac90 0x7fc37c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.794+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc37c06ac90 0x7fc37c06d150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc390006f10 tx=0x7fc390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:04.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.795+0000 7fc3b723eb80 1 Processor -- start 2024-03-29T09:09:04.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.795+0000 7fc3b723eb80 1 -- start start 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d8fc0 0x55d9e27af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27ae980 0x55d9e27afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e2708940 con 0x55d9e27ae980 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e2706300 con 0x55d9e27d3c20 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9e2701bf0 con 0x55d9e27d8fc0 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27ae980 0x55d9e27afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27ae980 0x55d9e27afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60898/0 (socket says 172.21.15.99:60898) 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.796+0000 7fc3a1800700 1 -- 172.21.15.99:0/519594374 learned_addr learned my addr 172.21.15.99:0/519594374 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:04.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc39be00700 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d8fc0 0x55d9e27af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc3a1800700 1 -- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d8fc0 msgr2=0x55d9e27af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc3a1800700 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d8fc0 0x55d9e27af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc3a1800700 1 -- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 msgr2=0x55d9e27d7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc3a1800700 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27d7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc3a1800700 1 -- 172.21.15.99:0/519594374 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9e252fb10 con 0x55d9e27ae980 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc3a1800700 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27ae980 0x55d9e27afb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3940021e0 tx=0x7fc394001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439991e1200b99fa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc38d600700 1 -- 172.21.15.99:0/519594374 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39400fd20 con 0x55d9e27ae980 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc38d600700 1 -- 172.21.15.99:0/519594374 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39400fe80 con 0x55d9e27ae980 2024-03-29T09:09:04.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.797+0000 7fc38d600700 1 -- 172.21.15.99:0/519594374 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394003c00 con 0x55d9e27ae980 2024-03-29T09:09:04.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 -- 172.21.15.99:0/519594374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27ae980 msgr2=0x55d9e27afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27ae980 0x55d9e27afb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3940021e0 tx=0x7fc394001b10 comp rx=0 tx=0).stop 2024-03-29T09:09:04.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 -- 172.21.15.99:0/519594374 shutdown_connections 2024-03-29T09:09:04.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27ae980 0x55d9e27afb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27d7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/519594374 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d8fc0 0x55d9e27af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 -- 172.21.15.99:0/519594374 >> 172.21.15.99:0/519594374 conn(0x55d9e270f650 msgr2=0x55d9e27f63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:04.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 -- 172.21.15.99:0/519594374 shutdown_connections 2024-03-29T09:09:04.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.798+0000 7fc3b723eb80 1 -- 172.21.15.99:0/519594374 wait complete. 2024-03-29T09:09:04.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.799+0000 7fc3b723eb80 1 -- start start 2024-03-29T09:09:04.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.799+0000 7fc3b723eb80 1 Processor -- start 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e27dec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e269dd30 con 0x55d9e27d8fc0 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e269d9d0 con 0x55d9e27d3c20 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9e2585650 con 0x55d9e27ae980 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3b723eb80 10 client.0 _ll_register_callbacks cb 0x55d9e26a7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3a0e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e27dec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc39be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27dad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60904/0 (socket says 172.21.15.99:60904) 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.800+0000 7fc3a1800700 1 -- 172.21.15.99:0/3603731427 learned_addr learned my addr 172.21.15.99:0/3603731427 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc39be00700 1 -- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 msgr2=0x55d9e27dec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc39be00700 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e27dec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc39be00700 1 -- 172.21.15.99:0/3603731427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 msgr2=0x55d9e27db2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc39be00700 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27db2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc39be00700 1 -- 172.21.15.99:0/3603731427 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c0cddc0 con 0x55d9e27d3c20 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc39be00700 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27dad90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc388088cc0 tx=0x7fc388059d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64527ad934e08ce server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc382000700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc382000700 1 -- 172.21.15.99:0/3603731427 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880619d0 con 0x55d9e27d3c20 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9e26c2330 con 0x55d9e27d3c20 2024-03-29T09:09:04.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc382000700 1 -- 172.21.15.99:0/3603731427 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc38806edc0 con 0x55d9e27d3c20 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc382000700 1 -- 172.21.15.99:0/3603731427 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc388055cf0 con 0x55d9e27d3c20 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc3b723eb80 10 client.4390 fetch_fsmap learned FSMap version 6 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.801+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d9e26c7e50 con 0x55d9e27d3c20 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc382000700 1 -- 172.21.15.99:0/3603731427 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d9e26c2330 con 0x55d9e27d3c20 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc382000700 1 -- 172.21.15.99:0/3603731427 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fc388059840 con 0x55d9e27d3c20 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc3b723eb80 10 client.4390 fetch_fsmap finished waiting for FSMap version 6 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc3b723eb80 10 client.4390 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc3b723eb80 20 client.4390 populate_metadata read hostname 'smithi099' 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e2809420 0x55d9e2809800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc3b723eb80 4 client.4390 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 --> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9e26fa6b0 con 0x55d9e2809420 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.802+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e2809420 0x55d9e2809800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.803+0000 7fc382000700 10 client.4390 ms_handle_connect on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:04.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.803+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e2809420 0x55d9e2809800 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4cb2ea19b9bdc01 server_cookie=327455421d422221 in_seq=0 out_seq=0 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc382000700 1 -- 172.21.15.99:0/3603731427 <== mds.0 v2:172.21.15.133:6834/440339259 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d9e26fa6b0 con 0x55d9e2809420 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc382000700 10 client.4390 handle_command_reply: tid=0 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 client.4390 shutdown 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 msgr2=0x55d9e27dad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27dad90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc388088cc0 tx=0x7fc388059d60 comp rx=0 tx=0).stop 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 shutdown_connections 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e2809420 0x55d9e2809800 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e27dec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3603731427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27dad90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc382000700 0 client.4390 ms_handle_reset on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 >> 172.21.15.99:0/3603731427 conn(0x55d9e270f650 msgr2=0x55d9e27f6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:04.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 shutdown_connections 2024-03-29T09:09:04.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.804+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3603731427 wait complete. 2024-03-29T09:09:04.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.805+0000 7fc3b723eb80 20 client.4390 trim_cache size 0 max 16384 2024-03-29T09:09:04.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.822+0000 7fc3b723eb80 1 Processor -- start 2024-03-29T09:09:04.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.822+0000 7fc3b723eb80 1 -- start start 2024-03-29T09:09:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.823+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27ae980 0x55d9e27f6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.823+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d3c20 0x55d9e27d7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.823+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27ab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.823+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e2805610 con 0x55d9e27d8fc0 2024-03-29T09:09:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.823+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e27d7cc0 con 0x55d9e27ae980 2024-03-29T09:09:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.823+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9e27df050 con 0x55d9e27d3c20 2024-03-29T09:09:04.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.823+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27ab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60908/0 (socket says 172.21.15.99:60908) 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 -- 172.21.15.99:0/1960688458 learned_addr learned my addr 172.21.15.99:0/1960688458 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27ae980 0x55d9e27f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc39be00700 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d3c20 0x55d9e27d7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 -- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d3c20 msgr2=0x55d9e27d7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d3c20 0x55d9e27d7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 -- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27ae980 msgr2=0x55d9e27f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27ae980 0x55d9e27f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 -- 172.21.15.99:0/1960688458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9e27d3640 con 0x55d9e27d8fc0 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc3a1800700 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27ab560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc394014000 tx=0x7fc39400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e1125267704204 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc38e000700 1 -- 172.21.15.99:0/1960688458 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394001850 con 0x55d9e27d8fc0 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc38e000700 1 -- 172.21.15.99:0/1960688458 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394003a00 con 0x55d9e27d8fc0 2024-03-29T09:09:04.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.824+0000 7fc38e000700 1 -- 172.21.15.99:0/1960688458 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394003db0 con 0x55d9e27d8fc0 2024-03-29T09:09:04.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 -- 172.21.15.99:0/1960688458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 msgr2=0x55d9e27ab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27ab560 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc394014000 tx=0x7fc39400a000 comp rx=0 tx=0).stop 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 -- 172.21.15.99:0/1960688458 shutdown_connections 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27ab560 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27ae980 0x55d9e27f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/1960688458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27d3c20 0x55d9e27d7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 -- 172.21.15.99:0/1960688458 >> 172.21.15.99:0/1960688458 conn(0x55d9e270f650 msgr2=0x55d9e27f6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 -- 172.21.15.99:0/1960688458 shutdown_connections 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.825+0000 7fc3b723eb80 1 -- 172.21.15.99:0/1960688458 wait complete. 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.826+0000 7fc3b723eb80 1 -- start start 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.826+0000 7fc3b723eb80 1 Processor -- start 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e285d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3b723eb80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e27f3bf0 con 0x55d9e27d8fc0 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9e27f4540 con 0x55d9e27d3c20 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3b723eb80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9e2818460 con 0x55d9e27ae980 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3b723eb80 10 client.0 _ll_register_callbacks cb 0x55d9e26a7420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc39be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60910/0 (socket says 172.21.15.99:60910) 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 -- 172.21.15.99:0/3215944695 learned_addr learned my addr 172.21.15.99:0/3215944695 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e285d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 -- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 msgr2=0x55d9e285d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e285d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 -- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 msgr2=0x55d9e27af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.827+0000 7fc3a1800700 1 -- 172.21.15.99:0/3215944695 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9e252fb10 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3a1800700 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27af700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc394000f80 tx=0x7fc394001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e12c1b50a8da888 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc382000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc382000700 1 -- 172.21.15.99:0/3215944695 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3940054e0 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc382000700 1 -- 172.21.15.99:0/3215944695 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc394005640 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc39400fe80 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc382000700 1 -- 172.21.15.99:0/3215944695 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc394005800 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 10 client.4416 fetch_fsmap learned FSMap version 6 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d9e2587c80 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc382000700 1 -- 172.21.15.99:0/3215944695 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39400fe80 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc382000700 1 -- 172.21.15.99:0/3215944695 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 6) v1 ==== 3122+0+0 (secure 0 0 0) 0x7fc394012070 con 0x55d9e27d8fc0 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 10 client.4416 fetch_fsmap finished waiting for FSMap version 6 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 10 client.4416 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 20 client.4416 populate_metadata read hostname 'smithi099' 2024-03-29T09:09:04.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e27b0690 0x55d9e27f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:04.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 4 client.4416 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T09:09:04.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.828+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 --> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d9e2550ca0 con 0x55d9e27b0690 2024-03-29T09:09:04.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.829+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e27b0690 0x55d9e27f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.829+0000 7fc382000700 10 client.4416 ms_handle_connect on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.829+0000 7fc3a0e00700 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e27b0690 0x55d9e27f2cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6f06af3284d19c server_cookie=66e9f02df588a91b in_seq=0 out_seq=0 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc382000700 1 -- 172.21.15.99:0/3215944695 <== mds.0 v2:172.21.15.133:6834/440339259 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d9e2550ca0 con 0x55d9e27b0690 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc382000700 10 client.4416 handle_command_reply: tid=0 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 client.4416 shutdown 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 msgr2=0x55d9e27af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27af700 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc394000f80 tx=0x7fc394001250 comp rx=0 tx=0).stop 2024-03-29T09:09:04.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 shutdown_connections 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x55d9e27b0690 0x55d9e27f2cd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d9e27d8fc0 0x55d9e27af700 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9e27ae980 0x55d9e285d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc382000700 0 client.4416 ms_handle_reset on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 --2- 172.21.15.99:0/3215944695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9e27d3c20 0x55d9e27af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 >> 172.21.15.99:0/3215944695 conn(0x55d9e270f650 msgr2=0x55d9e27f6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 shutdown_connections 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 1 -- 172.21.15.99:0/3215944695 wait complete. 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.830+0000 7fc3b723eb80 20 client.4416 trim_cache size 0 max 16384 2024-03-29T09:09:04.834 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "83e3f490-14b9-4a33-8e2d-bbf792b299cc", 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4285, 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 7.4399650020000001, 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 6, 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 24, 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 23, 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 11.954383175 2024-03-29T09:09:04.835 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:09:04.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc37c06ac90 msgr2=0x7fc37c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc37c06ac90 0x7fc37c06d150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc390006f10 tx=0x7fc390008040 comp rx=0 tx=0).stop 2024-03-29T09:09:04.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c1608e0 msgr2=0x7fc39c1574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:04.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c1608e0 0x7fc39c1574f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc388009c40 tx=0x7fc388004750 comp rx=0 tx=0).stop 2024-03-29T09:09:04.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 shutdown_connections 2024-03-29T09:09:04.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc37c06ac90 0x7fc37c06d150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc39c160cc0 0x7fc39c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc39c0d3a70 0x7fc39c0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 --2- 172.21.15.99:0/970041309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc39c1608e0 0x7fc39c1574f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 >> 172.21.15.99:0/970041309 conn(0x7fc39c09c490 msgr2=0x7fc39c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 shutdown_connections 2024-03-29T09:09:04.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:04.831+0000 7fc3a3800700 1 -- 172.21.15.99:0/970041309 wait complete. 2024-03-29T09:09:04.849 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T09:09:04.849 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:04.849 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:09:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.039+0000 7fbcfee00700 1 Processor -- start 2024-03-29T09:09:05.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.039+0000 7fbcfee00700 1 -- start start 2024-03-29T09:09:05.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.040+0000 7fbcfee00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbcf815d5f0 0x7fbcf80ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcfee00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf80cdc50 con 0x7fbcf80d1690 2024-03-29T09:09:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcfee00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf80d06a0 con 0x7fbcf80b3510 2024-03-29T09:09:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcfee00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf80d0980 con 0x7fbcf815d5f0 2024-03-29T09:09:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbcf815d5f0 0x7fbcf80ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbcf815d5f0 0x7fbcf80ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60926/0 (socket says 172.21.15.99:60926) 2024-03-29T09:09:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbcf80d1690 0x7fbcf80b2e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:35414/0 (socket says 172.21.15.99:35414) 2024-03-29T09:09:05.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7e00700 1 -- 172.21.15.99:0/1993774492 learned_addr learned my addr 172.21.15.99:0/1993774492 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7e00700 1 -- 172.21.15.99:0/1993774492 >> v1:172.21.15.133:6789/0 conn(0x7fbcf80b3510 legacy=0x7fbcf80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:05.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7e00700 1 -- 172.21.15.99:0/1993774492 >> v1:172.21.15.99:6789/0 conn(0x7fbcf80d1690 legacy=0x7fbcf80b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7e00700 1 -- 172.21.15.99:0/1993774492 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf80cde80 con 0x7fbcf815d5f0 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.041+0000 7fbcf7e00700 1 --2- 172.21.15.99:0/1993774492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf80ce960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbce80098f0 tx=0x7fbce8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df06445c2cce8d7f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcf6a00700 1 -- 172.21.15.99:0/1993774492 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8004230 con 0x7fbcf815d5f0 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcf6a00700 1 -- 172.21.15.99:0/1993774492 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce8004390 con 0x7fbcf815d5f0 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcf6a00700 1 -- 172.21.15.99:0/1993774492 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80036a0 con 0x7fbcf815d5f0 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcfee00700 1 -- 172.21.15.99:0/1993774492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 msgr2=0x7fbcf80ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcfee00700 1 --2- 172.21.15.99:0/1993774492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf80ce960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbce80098f0 tx=0x7fbce8009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcfee00700 1 -- 172.21.15.99:0/1993774492 shutdown_connections 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcfee00700 1 --2- 172.21.15.99:0/1993774492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf80ce960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcfee00700 1 -- 172.21.15.99:0/1993774492 >> 172.21.15.99:0/1993774492 conn(0x7fbcf808cb40 msgr2=0x7fbcf809c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcfee00700 1 -- 172.21.15.99:0/1993774492 shutdown_connections 2024-03-29T09:09:05.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.042+0000 7fbcfee00700 1 -- 172.21.15.99:0/1993774492 wait complete. 2024-03-29T09:09:05.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.044+0000 7fbcfee00700 1 Processor -- start 2024-03-29T09:09:05.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.044+0000 7fbcfee00700 1 -- start start 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfee00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf80b3510 0x7fbcf8160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf80d1690 0x7fbcf81f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf8160d80 con 0x7fbcf815d5f0 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfee00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf815a7e0 con 0x7fbcf80b3510 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf80b7180 con 0x7fbcf80d1690 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60928/0 (socket says 172.21.15.99:60928) 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcf7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf80b3510 0x7fbcf8160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcfce00700 1 -- 172.21.15.99:0/4033547173 learned_addr learned my addr 172.21.15.99:0/4033547173 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.045+0000 7fbcf7400700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf80d1690 0x7fbcf81f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfce00700 1 -- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf80d1690 msgr2=0x7fbcf81f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfce00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf80d1690 0x7fbcf81f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:05.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfce00700 1 -- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf80b3510 msgr2=0x7fbcf8160840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfce00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf80b3510 0x7fbcf8160840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfce00700 1 -- 172.21.15.99:0/4033547173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8009550 con 0x7fbcf815d5f0 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcf7e00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf80b3510 0x7fbcf8160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfce00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf815ac70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000cc10 tx=0x7fbcf000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcefe00700 1 -- 172.21.15.99:0/4033547173 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf000aa00 con 0x7fbcf815d5f0 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcefe00700 1 -- 172.21.15.99:0/4033547173 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf000ab60 con 0x7fbcf815d5f0 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf80b6640 con 0x7fbcf815d5f0 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcefe00700 1 -- 172.21.15.99:0/4033547173 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0012420 con 0x7fbcf815d5f0 2024-03-29T09:09:05.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.046+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf80b6bc0 con 0x7fbcf815d5f0 2024-03-29T09:09:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.047+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf80677a0 con 0x7fbcf815d5f0 2024-03-29T09:09:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.048+0000 7fbcefe00700 1 -- 172.21.15.99:0/4033547173 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbcf0012580 con 0x7fbcf815d5f0 2024-03-29T09:09:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.049+0000 7fbcefe00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbcd806ac90 0x7fbcd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.049+0000 7fbcefe00700 1 -- 172.21.15.99:0/4033547173 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbcf0089940 con 0x7fbcf815d5f0 2024-03-29T09:09:05.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.049+0000 7fbcf7e00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbcd806ac90 0x7fbcd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:05.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.050+0000 7fbcf7e00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbcd806ac90 0x7fbcd806d150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009130 tx=0x7fbce8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:05.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.053+0000 7fbcefe00700 1 -- 172.21.15.99:0/4033547173 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbcf0054790 con 0x7fbcf815d5f0 2024-03-29T09:09:05.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.303+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbcf815a920 con 0x7fbcf815d5f0 2024-03-29T09:09:05.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.304+0000 7fbcefe00700 1 -- 172.21.15.99:0/4033547173 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7fbcf0057db0 con 0x7fbcf815d5f0 2024-03-29T09:09:05.304 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbcd806ac90 msgr2=0x7fbcd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbcd806ac90 0x7fbcd806d150 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009130 tx=0x7fbce8008030 comp rx=0 tx=0).stop 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 msgr2=0x7fbcf815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf815ac70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbcf000cc10 tx=0x7fbcf000cf20 comp rx=0 tx=0).stop 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 shutdown_connections 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbcd806ac90 0x7fbcd806d150 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbcf815d5f0 0x7fbcf815ac70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbcf80b3510 0x7fbcf8160840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 --2- 172.21.15.99:0/4033547173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbcf80d1690 0x7fbcf81f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 >> 172.21.15.99:0/4033547173 conn(0x7fbcf808cb40 msgr2=0x7fbcf809c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 shutdown_connections 2024-03-29T09:09:05.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:05.306+0000 7fbcfee00700 1 -- 172.21.15.99:0/4033547173 wait complete. 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:09:05.324 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:09:05.325 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T09:09:05.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:05.325 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:09:05.366 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42243sec preferred_lft 42243sec 2024-03-29T09:09:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:09:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:05.369 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T09:09:05.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:05.369 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:05.369 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:05.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2024-03-29T09:09:05.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2024-03-29T09:09:05.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2024-03-29T09:09:05.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T09:09:05.369 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:05.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:05.865 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:09:05.931 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:09:05.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:05.934 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:09:06.003 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:09:06.003 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:09:06.003 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:09:06.003 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:09:06.003 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:09:06.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:06.005 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:06.005 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:06.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:09:06.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:09:06.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:09:06.005 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:06.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:06.687 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:09:06.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:06.729 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-03-29T09:09:06.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:06.811 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:06.811 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:06.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T09:09:06.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-29T09:09:06.811 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:06.986 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-29T09:09:06.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-29T09:09:06.987 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:07.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:07.261 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:07.261 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:07.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:09:07.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T09:09:07.261 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:07.425 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-29T09:09:07.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:07.426 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:07.466 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-29T09:09:07.467 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:07.467 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:07.547 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:09:07.653 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:07.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:07.716 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:07.718 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:07.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:07.719 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:09:07.760 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:09:07.760 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-29T09:09:07.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:09:07.872 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:07.872 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:07.935 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:07.938 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:07.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:07.938 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:09:07.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi099.stderr:2024-03-29T09:09:07.970+0000 7fb1c44c55c0 -1 init, newargv = 0x5631f6af3800 newargc=15 2024-03-29T09:09:07.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi099.stderr:ceph-fuse[49336]: starting ceph client 2024-03-29T09:09:07.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi099.stderr:ceph-fuse[49336]: starting fuse 2024-03-29T09:09:09.082 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:09:09.155 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:09.155 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:09.221 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:09.224 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:09.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:09.224 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:09:09.265 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:09:09.266 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:09:09.267 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:09:09.268 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:09:09.268 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:09.268 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.admin")) 2024-03-29T09:09:09.268 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:09:09.446 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.admin.49336.asok 2024-03-29T09:09:09.622 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.49336.asok 2024-03-29T09:09:09.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:09.622 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.49336.asok status 2024-03-29T09:09:09.840 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:09:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:09:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:09:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:09:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "admin", 2024-03-29T09:09:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:09:09.840 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "49336", 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4431, 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4431 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:09:09.841 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2324244636 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2324244636 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4431 192.168.0.1:0/2324244636", 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2324244636", 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 6, 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 24, 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:09:09.842 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:09:09.843 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:09:09.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:09.854 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:09:09.854 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:09.896 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:09:09.896 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:09.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:09.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:10.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:10.003 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-29T09:09:10.140 INFO:teuthology.orchestra.run.smithi099.stdout:# file: . 2024-03-29T09:09:10.140 INFO:teuthology.orchestra.run.smithi099.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-29T09:09:10.140 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:10.141 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:09:10.141 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:09:10.183 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:10.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.411+0000 7ff544c00700 1 Processor -- start 2024-03-29T09:09:10.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.411+0000 7ff544c00700 1 -- start start 2024-03-29T09:09:10.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff544c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff540143940 0x7ff540070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff544c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540070ca0 0x7ff54015db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff544c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5400cd820 con 0x7ff54015e040 2024-03-29T09:09:10.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff544c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff540143d20 con 0x7ff540143940 2024-03-29T09:09:10.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff544c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015eaf0 con 0x7ff540070ca0 2024-03-29T09:09:10.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff53de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff540143940 0x7ff540070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff53de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff540143940 0x7ff540070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59476/0 (socket says 172.21.15.99:59476) 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff53de00700 1 -- 172.21.15.99:0/1679105451 learned_addr learned my addr 172.21.15.99:0/1679105451 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff537e00700 1 --2- 172.21.15.99:0/1679105451 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540070ca0 0x7ff54015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff53de00700 1 -- 172.21.15.99:0/1679105451 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540070ca0 msgr2=0x7ff54015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff53de00700 1 --2- 172.21.15.99:0/1679105451 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540070ca0 0x7ff54015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff53de00700 1 -- 172.21.15.99:0/1679105451 >> v1:172.21.15.133:6789/0 conn(0x7ff54015e040 legacy=0x7ff5400ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff53de00700 1 -- 172.21.15.99:0/1679105451 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54015e530 con 0x7ff540143940 2024-03-29T09:09:10.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.413+0000 7ff537e00700 1 --2- 172.21.15.99:0/1679105451 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540070ca0 0x7ff54015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:10.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.414+0000 7ff53de00700 1 --2- 172.21.15.99:0/1679105451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540143940 0x7ff540070760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098f0 tx=0x7ff530009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b5b10a0e200d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:10.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.414+0000 7ff53d400700 1 -- 172.21.15.99:0/1679105451 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000b8a0 con 0x7ff540143940 2024-03-29T09:09:10.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.414+0000 7ff53d400700 1 -- 172.21.15.99:0/1679105451 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53000ba00 con 0x7ff540143940 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.414+0000 7ff53d400700 1 -- 172.21.15.99:0/1679105451 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53000bd90 con 0x7ff540143940 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.414+0000 7ff544c00700 1 -- 172.21.15.99:0/1679105451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540143940 msgr2=0x7ff540070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.414+0000 7ff544c00700 1 --2- 172.21.15.99:0/1679105451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540143940 0x7ff540070760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff5300098f0 tx=0x7ff530009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.415+0000 7ff544c00700 1 -- 172.21.15.99:0/1679105451 shutdown_connections 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.415+0000 7ff544c00700 1 --2- 172.21.15.99:0/1679105451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff540143940 0x7ff540070760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.415+0000 7ff544c00700 1 --2- 172.21.15.99:0/1679105451 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540070ca0 0x7ff54015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.415+0000 7ff544c00700 1 -- 172.21.15.99:0/1679105451 >> 172.21.15.99:0/1679105451 conn(0x7ff540066540 msgr2=0x7ff540066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.415+0000 7ff544c00700 1 -- 172.21.15.99:0/1679105451 shutdown_connections 2024-03-29T09:09:10.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.415+0000 7ff544c00700 1 -- 172.21.15.99:0/1679105451 wait complete. 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.416+0000 7ff544c00700 1 Processor -- start 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.416+0000 7ff544c00700 1 -- start start 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.417+0000 7ff544c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff540070ca0 0x7ff5401e4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.417+0000 7ff544c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540143940 0x7ff540141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.417+0000 7ff544c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54015e040 0x7ff540158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.417+0000 7ff544c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540141b40 con 0x7ff54015e040 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.417+0000 7ff544c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540141e20 con 0x7ff540070ca0 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.417+0000 7ff544c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff540142100 con 0x7ff540143940 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.417+0000 7ff53e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54015e040 0x7ff540158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54015e040 0x7ff540158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59478/0 (socket says 172.21.15.99:59478) 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 -- 172.21.15.99:0/3701868968 learned_addr learned my addr 172.21.15.99:0/3701868968 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53de00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff540070ca0 0x7ff5401e4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff537e00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540143940 0x7ff540141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 -- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540143940 msgr2=0x7ff540141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540143940 0x7ff540141310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 -- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff540070ca0 msgr2=0x7ff5401e4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff540070ca0 0x7ff5401e4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 -- 172.21.15.99:0/3701868968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530009550 con 0x7ff54015e040 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff53e800700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54015e040 0x7ff540158fb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff53800cb50 tx=0x7ff53800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff536000700 1 -- 172.21.15.99:0/3701868968 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538003c00 con 0x7ff54015e040 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff536000700 1 -- 172.21.15.99:0/3701868968 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538003d60 con 0x7ff54015e040 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5401423e0 con 0x7ff54015e040 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.418+0000 7ff536000700 1 -- 172.21.15.99:0/3701868968 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800aa20 con 0x7ff54015e040 2024-03-29T09:09:10.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.419+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540141850 con 0x7ff54015e040 2024-03-29T09:09:10.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.420+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5400cd1d0 con 0x7ff54015e040 2024-03-29T09:09:10.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.420+0000 7ff536000700 1 -- 172.21.15.99:0/3701868968 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff53800ab80 con 0x7ff54015e040 2024-03-29T09:09:10.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.421+0000 7ff536000700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff51c06ac90 0x7ff51c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.421+0000 7ff536000700 1 -- 172.21.15.99:0/3701868968 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff538089660 con 0x7ff54015e040 2024-03-29T09:09:10.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.421+0000 7ff53de00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff51c06ac90 0x7ff51c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.422+0000 7ff53de00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff51c06ac90 0x7ff51c06d150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff530009c40 tx=0x7ff530008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:10.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.426+0000 7ff536000700 1 -- 172.21.15.99:0/3701868968 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff5380544b0 con 0x7ff54015e040 2024-03-29T09:09:10.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.629+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff540142900 con 0x7ff54015e040 2024-03-29T09:09:10.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.631+0000 7ff536000700 1 -- 172.21.15.99:0/3701868968 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7ff540142900 con 0x7ff54015e040 2024-03-29T09:09:10.631 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:10.631 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:10.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff51c06ac90 msgr2=0x7ff51c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff51c06ac90 0x7ff51c06d150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff530009c40 tx=0x7ff530008030 comp rx=0 tx=0).stop 2024-03-29T09:09:10.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54015e040 msgr2=0x7ff540158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54015e040 0x7ff540158fb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff53800cb50 tx=0x7ff53800ce60 comp rx=0 tx=0).stop 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 shutdown_connections 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff51c06ac90 0x7ff51c06d150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54015e040 0x7ff540158fb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff540070ca0 0x7ff5401e4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 --2- 172.21.15.99:0/3701868968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff540143940 0x7ff540141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 >> 172.21.15.99:0/3701868968 conn(0x7ff540066540 msgr2=0x7ff5400ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 shutdown_connections 2024-03-29T09:09:10.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.633+0000 7ff544c00700 1 -- 172.21.15.99:0/3701868968 wait complete. 2024-03-29T09:09:10.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:09:10.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:09:10.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.847+0000 7fbc1aa00700 1 Processor -- start 2024-03-29T09:09:10.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.847+0000 7fbc1aa00700 1 -- start start 2024-03-29T09:09:10.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc1aa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc14160770 0x7fbc140b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc1aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbc140d3550 0x7fbc140ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc1aa00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140d1520 con 0x7fbc140d0d60 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc1aa00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc140d1800 con 0x7fbc140d3550 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc1aa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140d1ae0 con 0x7fbc14160770 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc18a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbc140d3550 0x7fbc140ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc13400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbc140d0d60 0x7fbc140d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37508/0 (socket says 172.21.15.99:37508) 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc13400700 1 -- 172.21.15.99:0/1491822327 learned_addr learned my addr 172.21.15.99:0/1491822327 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.848+0000 7fbc13e00700 1 --2- 172.21.15.99:0/1491822327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc14160770 0x7fbc140b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc12a00700 1 -- 172.21.15.99:0/1491822327 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609521830 0 0) 0x7fbc140d1520 con 0x7fbc140d0d60 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc12a00700 1 -- 172.21.15.99:0/1491822327 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf80035b0 con 0x7fbc140d0d60 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc18a00700 1 -- 172.21.15.99:0/1491822327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc14160770 msgr2=0x7fbc140b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc18a00700 1 --2- 172.21.15.99:0/1491822327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc14160770 0x7fbc140b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc18a00700 1 -- 172.21.15.99:0/1491822327 >> v1:172.21.15.99:6789/0 conn(0x7fbc140d0d60 legacy=0x7fbc140d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc18a00700 1 -- 172.21.15.99:0/1491822327 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc140d1dc0 con 0x7fbc140d3550 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc18a00700 1 --2- 172.21.15.99:0/1491822327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc140d3550 0x7fbc140ce080 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009d90 tx=0x7fbc0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31be8b2562491faa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc12a00700 1 -- 172.21.15.99:0/1491822327 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00e070 con 0x7fbc140d3550 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc12a00700 1 -- 172.21.15.99:0/1491822327 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c00bb20 con 0x7fbc140d3550 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.849+0000 7fbc12a00700 1 -- 172.21.15.99:0/1491822327 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c012430 con 0x7fbc140d3550 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1491822327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc140d3550 msgr2=0x7fbc140ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1491822327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc140d3550 0x7fbc140ce080 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009d90 tx=0x7fbc0c00b330 comp rx=0 tx=0).stop 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1491822327 shutdown_connections 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1491822327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc140d3550 0x7fbc140ce080 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1491822327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc14160770 0x7fbc140b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1491822327 >> 172.21.15.99:0/1491822327 conn(0x7fbc1408afd0 msgr2=0x7fbc1409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:10.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1491822327 shutdown_connections 2024-03-29T09:09:10.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.850+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1491822327 wait complete. 2024-03-29T09:09:10.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.852+0000 7fbc1aa00700 1 Processor -- start 2024-03-29T09:09:10.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.852+0000 7fbc1aa00700 1 -- start start 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.853+0000 7fbc1aa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc140d0d60 0x7fbc140b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc1aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc140d3550 0x7fbc140b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc1aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc14160770 0x7fbc140b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc1aa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140b16c0 con 0x7fbc14160770 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc1aa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140b0ab0 con 0x7fbc140d3550 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc1aa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140b0d90 con 0x7fbc140d0d60 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc14160770 0x7fbc140b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc14160770 0x7fbc140b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59494/0 (socket says 172.21.15.99:59494) 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 -- 172.21.15.99:0/1013837151 learned_addr learned my addr 172.21.15.99:0/1013837151 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc13400700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc140d3550 0x7fbc140b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc13e00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc140d0d60 0x7fbc140b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 -- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc140d0d60 msgr2=0x7fbc140b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc140d0d60 0x7fbc140b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 -- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc140d3550 msgr2=0x7fbc140b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc140d3550 0x7fbc140b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 -- 172.21.15.99:0/1013837151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c009a70 con 0x7fbc14160770 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.854+0000 7fbc18a00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc14160770 0x7fbc140b6290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c014040 tx=0x7fbc0c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.855+0000 7fbc0be00700 1 -- 172.21.15.99:0/1013837151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c012a70 con 0x7fbc14160770 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.855+0000 7fbc0be00700 1 -- 172.21.15.99:0/1013837151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c020070 con 0x7fbc14160770 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.855+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc140b1070 con 0x7fbc14160770 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.855+0000 7fbc0be00700 1 -- 172.21.15.99:0/1013837151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c012e30 con 0x7fbc14160770 2024-03-29T09:09:10.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.855+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc140b8b00 con 0x7fbc14160770 2024-03-29T09:09:10.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.856+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc140b1870 con 0x7fbc14160770 2024-03-29T09:09:10.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.857+0000 7fbc0be00700 1 -- 172.21.15.99:0/1013837151 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbc0c004040 con 0x7fbc14160770 2024-03-29T09:09:10.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.857+0000 7fbc0be00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbf406ac90 0x7fbbf406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:10.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.858+0000 7fbc0be00700 1 -- 172.21.15.99:0/1013837151 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbc0c022070 con 0x7fbc14160770 2024-03-29T09:09:10.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.858+0000 7fbc13e00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbf406ac90 0x7fbbf406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:10.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.858+0000 7fbc13e00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbf406ac90 0x7fbbf406d150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009b70 tx=0x7fbc04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:10.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:10.862+0000 7fbc0be00700 1 -- 172.21.15.99:0/1013837151 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbc140b1870 con 0x7fbc14160770 2024-03-29T09:09:11.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.068+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc140678b0 con 0x7fbc14160770 2024-03-29T09:09:11.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.069+0000 7fbc0be00700 1 -- 172.21.15.99:0/1013837151 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc0c059cd0 con 0x7fbc14160770 2024-03-29T09:09:11.069 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:09:11.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbf406ac90 msgr2=0x7fbbf406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbf406ac90 0x7fbbf406d150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009b70 tx=0x7fbc04008040 comp rx=0 tx=0).stop 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc14160770 msgr2=0x7fbc140b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc14160770 0x7fbc140b6290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c014040 tx=0x7fbc0c004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 shutdown_connections 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbf406ac90 0x7fbbf406d150 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbc14160770 0x7fbc140b6290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbc140d0d60 0x7fbc140b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 --2- 172.21.15.99:0/1013837151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc140d3550 0x7fbc140b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 >> 172.21.15.99:0/1013837151 conn(0x7fbc1408afd0 msgr2=0x7fbc1415e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 shutdown_connections 2024-03-29T09:09:11.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.071+0000 7fbc1aa00700 1 -- 172.21.15.99:0/1013837151 wait complete. 2024-03-29T09:09:11.089 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:09:11.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.281+0000 7fe193000700 1 Processor -- start 2024-03-29T09:09:11.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.281+0000 7fe193000700 1 -- start start 2024-03-29T09:09:11.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.282+0000 7fe193000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe18c15d360 0x7fe18c0cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.282+0000 7fe193000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d1f10 0x7fe18c0b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.282+0000 7fe193000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c0cf1b0 con 0x7fe18c0d0110 2024-03-29T09:09:11.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.282+0000 7fe193000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18c0b60c0 con 0x7fe18c15d360 2024-03-29T09:09:11.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.282+0000 7fe193000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0d0690 con 0x7fe18c0d1f10 2024-03-29T09:09:11.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe18be00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe18c15d360 0x7fe18c0cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d1f10 0x7fe18c0b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d1f10 0x7fe18c0b6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:33620/0 (socket says 172.21.15.99:33620) 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 -- 172.21.15.99:0/1709955954 learned_addr learned my addr 172.21.15.99:0/1709955954 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe18b400700 1 --1- 172.21.15.99:0/1709955954 >> v1:172.21.15.133:6789/0 conn(0x7fe18c0d0110 0x7fe18c0d1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:44898/0 (socket says 172.21.15.99:44898) 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 -- 172.21.15.99:0/1709955954 >> v2:172.21.15.133:3300/0 conn(0x7fe18c15d360 msgr2=0x7fe18c0cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 --2- 172.21.15.99:0/1709955954 >> v2:172.21.15.133:3300/0 conn(0x7fe18c15d360 0x7fe18c0cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 -- 172.21.15.99:0/1709955954 >> v1:172.21.15.133:6789/0 conn(0x7fe18c0d0110 legacy=0x7fe18c0d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 -- 172.21.15.99:0/1709955954 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c0b62d0 con 0x7fe18c0d1f10 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe18be00700 1 --2- 172.21.15.99:0/1709955954 >> v2:172.21.15.133:3300/0 conn(0x7fe18c15d360 0x7fe18c0cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:11.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.283+0000 7fe191000700 1 --2- 172.21.15.99:0/1709955954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d1f10 0x7fe18c0b6db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe184009e50 tx=0x7fe18400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0f9240c057941c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:11.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe18aa00700 1 -- 172.21.15.99:0/1709955954 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18400e070 con 0x7fe18c0d1f10 2024-03-29T09:09:11.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe18aa00700 1 -- 172.21.15.99:0/1709955954 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18400b910 con 0x7fe18c0d1f10 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe18aa00700 1 -- 172.21.15.99:0/1709955954 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18400bcc0 con 0x7fe18c0d1f10 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 -- 172.21.15.99:0/1709955954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d1f10 msgr2=0x7fe18c0b6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 --2- 172.21.15.99:0/1709955954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d1f10 0x7fe18c0b6db0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe184009e50 tx=0x7fe18400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 -- 172.21.15.99:0/1709955954 shutdown_connections 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 --2- 172.21.15.99:0/1709955954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d1f10 0x7fe18c0b6db0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 --2- 172.21.15.99:0/1709955954 >> v2:172.21.15.133:3300/0 conn(0x7fe18c15d360 0x7fe18c0cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 -- 172.21.15.99:0/1709955954 >> 172.21.15.99:0/1709955954 conn(0x7fe18c09c530 msgr2=0x7fe18c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 -- 172.21.15.99:0/1709955954 shutdown_connections 2024-03-29T09:09:11.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.284+0000 7fe193000700 1 -- 172.21.15.99:0/1709955954 wait complete. 2024-03-29T09:09:11.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.286+0000 7fe193000700 1 Processor -- start 2024-03-29T09:09:11.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.286+0000 7fe193000700 1 -- start start 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe193000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d0110 0x7fe18c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe193000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18c0d1f10 0x7fe18c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe193000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe18c15d360 0x7fe18c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe193000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15e370 con 0x7fe18c0d1f10 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe193000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15e140 con 0x7fe18c15d360 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe193000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0c6fb0 con 0x7fe18c0d0110 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe18b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18c0d1f10 0x7fe18c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe18b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18c0d1f10 0x7fe18c15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59500/0 (socket says 172.21.15.99:59500) 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe18b400700 1 -- 172.21.15.99:0/2828961379 learned_addr learned my addr 172.21.15.99:0/2828961379 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe191000700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe18c15d360 0x7fe18c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.287+0000 7fe18be00700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d0110 0x7fe18c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe18b400700 1 -- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d0110 msgr2=0x7fe18c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe18b400700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d0110 0x7fe18c153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe18b400700 1 -- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe18c15d360 msgr2=0x7fe18c1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe18b400700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe18c15d360 0x7fe18c1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe18b400700 1 -- 172.21.15.99:0/2828961379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0094a0 con 0x7fe18c0d1f10 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe18b400700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18c0d1f10 0x7fe18c15dc00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c4f0 tx=0x7fe17800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe183e00700 1 -- 172.21.15.99:0/2828961379 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178005cb0 con 0x7fe18c0d1f10 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe183e00700 1 -- 172.21.15.99:0/2828961379 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178005e10 con 0x7fe18c0d1f10 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe184009b30 con 0x7fe18c0d1f10 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe183e00700 1 -- 172.21.15.99:0/2828961379 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1780072d0 con 0x7fe18c0d1f10 2024-03-29T09:09:11.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.288+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c0ccb70 con 0x7fe18c0d1f10 2024-03-29T09:09:11.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.289+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c0c7120 con 0x7fe18c0d1f10 2024-03-29T09:09:11.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.290+0000 7fe183e00700 1 -- 172.21.15.99:0/2828961379 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe178007430 con 0x7fe18c0d1f10 2024-03-29T09:09:11.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.292+0000 7fe183e00700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe16c06ac90 0x7fe16c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.292+0000 7fe183e00700 1 -- 172.21.15.99:0/2828961379 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe178088920 con 0x7fe18c0d1f10 2024-03-29T09:09:11.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.292+0000 7fe18be00700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe16c06ac90 0x7fe16c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.292+0000 7fe18be00700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe16c06ac90 0x7fe16c06d150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009cd0 tx=0x7fe17c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:11.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.296+0000 7fe183e00700 1 -- 172.21.15.99:0/2828961379 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe178053830 con 0x7fe18c0d1f10 2024-03-29T09:09:11.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.500+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe18c0678b0 con 0x7fe18c0d1f10 2024-03-29T09:09:11.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.500+0000 7fe183e00700 1 -- 172.21.15.99:0/2828961379 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1780170a0 con 0x7fe18c0d1f10 2024-03-29T09:09:11.500 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:11.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe16c06ac90 msgr2=0x7fe16c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe16c06ac90 0x7fe16c06d150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009cd0 tx=0x7fe17c009210 comp rx=0 tx=0).stop 2024-03-29T09:09:11.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18c0d1f10 msgr2=0x7fe18c15dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18c0d1f10 0x7fe18c15dc00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c4f0 tx=0x7fe17800c800 comp rx=0 tx=0).stop 2024-03-29T09:09:11.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 shutdown_connections 2024-03-29T09:09:11.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe16c06ac90 0x7fe16c06d150 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe18c15d360 0x7fe18c1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe18c0d0110 0x7fe18c153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 --2- 172.21.15.99:0/2828961379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe18c0d1f10 0x7fe18c15dc00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 >> 172.21.15.99:0/2828961379 conn(0x7fe18c09c530 msgr2=0x7fe18c0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:11.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 shutdown_connections 2024-03-29T09:09:11.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.503+0000 7fe193000700 1 -- 172.21.15.99:0/2828961379 wait complete. 2024-03-29T09:09:11.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:09:11.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.712+0000 7f1dfe400700 1 Processor -- start 2024-03-29T09:09:11.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.712+0000 7f1dfe400700 1 -- start start 2024-03-29T09:09:11.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.713+0000 7f1dfe400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df80d1e90 0x7f1df8168130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.713+0000 7f1dfe400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80afdd0 con 0x7f1df815d420 2024-03-29T09:09:11.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.713+0000 7f1dfe400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80d3b00 con 0x7f1df815d800 2024-03-29T09:09:11.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.713+0000 7f1dfe400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80d12b0 con 0x7f1df80d1e90 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.713+0000 7f1df7400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1df815d420 0x7f1df80d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37510/0 (socket says 172.21.15.99:37510) 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.713+0000 7f1df7400700 1 -- 172.21.15.99:0/3786855549 learned_addr learned my addr 172.21.15.99:0/3786855549 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df7e00700 1 --2- 172.21.15.99:0/3786855549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df80d1e90 0x7f1df8168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df6000700 1 -- 172.21.15.99:0/3786855549 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382754894 0 0) 0x7f1df80afdd0 con 0x7f1df815d420 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df6000700 1 -- 172.21.15.99:0/3786855549 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ddc0035b0 con 0x7f1df815d420 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df6000700 1 -- 172.21.15.99:0/3786855549 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610225468 0 0) 0x7f1df80d3b00 con 0x7f1df815d800 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df6000700 1 -- 172.21.15.99:0/3786855549 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df80afdd0 con 0x7f1df815d800 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df7e00700 1 -- 172.21.15.99:0/3786855549 >> v1:172.21.15.133:6789/0 conn(0x7f1df815d800 legacy=0x7f1df80d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df7e00700 1 -- 172.21.15.99:0/3786855549 >> v1:172.21.15.99:6789/0 conn(0x7f1df815d420 legacy=0x7f1df80d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df7e00700 1 -- 172.21.15.99:0/3786855549 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80b0070 con 0x7f1df80d1e90 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.714+0000 7f1df7e00700 1 --2- 172.21.15.99:0/3786855549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df80d1e90 0x7f1df8168130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009dc0 tx=0x7f1df000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18792ef5960f8e38 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:11.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1df6000700 1 -- 172.21.15.99:0/3786855549 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df000e070 con 0x7f1df80d1e90 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1df6000700 1 -- 172.21.15.99:0/3786855549 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df000bb20 con 0x7f1df80d1e90 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1df6000700 1 -- 172.21.15.99:0/3786855549 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df000be90 con 0x7f1df80d1e90 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1dfe400700 1 -- 172.21.15.99:0/3786855549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df80d1e90 msgr2=0x7f1df8168130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3786855549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df80d1e90 0x7f1df8168130 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009dc0 tx=0x7f1df000b330 comp rx=0 tx=0).stop 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1dfe400700 1 -- 172.21.15.99:0/3786855549 shutdown_connections 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3786855549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df80d1e90 0x7f1df8168130 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1dfe400700 1 -- 172.21.15.99:0/3786855549 >> 172.21.15.99:0/3786855549 conn(0x7f1df808afd0 msgr2=0x7f1df809c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1dfe400700 1 -- 172.21.15.99:0/3786855549 shutdown_connections 2024-03-29T09:09:11.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.715+0000 7f1dfe400700 1 -- 172.21.15.99:0/3786855549 wait complete. 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.717+0000 7f1dfe400700 1 Processor -- start 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.717+0000 7f1dfe400700 1 -- start start 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1dfe400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df80d1e90 0x7f1df80b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1dfe400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df815d420 0x7f1df8152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1dfe400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df815e600 0x7f1df815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1dfe400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8152e90 con 0x7f1df80d1e90 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1dfe400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80b6f70 con 0x7f1df815d420 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1dfe400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80ccc70 con 0x7f1df815e600 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1df7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df80d1e90 0x7f1df80b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1df7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df80d1e90 0x7f1df80b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59502/0 (socket says 172.21.15.99:59502) 2024-03-29T09:09:11.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1df6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df815d420 0x7f1df8152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1df7400700 1 -- 172.21.15.99:0/3547946245 learned_addr learned my addr 172.21.15.99:0/3547946245 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.718+0000 7f1df7e00700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df815e600 0x7f1df815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1df7400700 1 -- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df815e600 msgr2=0x7f1df815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1df7400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df815e600 0x7f1df815eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1df7400700 1 -- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df815d420 msgr2=0x7f1df8152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1df7400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df815d420 0x7f1df8152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1df7400700 1 -- 172.21.15.99:0/3547946245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df0009a70 con 0x7f1df80d1e90 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1df6a00700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df815d420 0x7f1df8152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1df7400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df80d1e90 0x7f1df80b7fa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c700 tx=0x7f1de800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1def400700 1 -- 172.21.15.99:0/3547946245 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8005d70 con 0x7f1df80d1e90 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1def400700 1 -- 172.21.15.99:0/3547946245 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de80070d0 con 0x7f1df80d1e90 2024-03-29T09:09:11.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1def400700 1 -- 172.21.15.99:0/3547946245 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8007440 con 0x7f1df80d1e90 2024-03-29T09:09:11.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df80b71e0 con 0x7f1df80d1e90 2024-03-29T09:09:11.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.719+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df80b77a0 con 0x7f1df80d1e90 2024-03-29T09:09:11.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.720+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df80678b0 con 0x7f1df80d1e90 2024-03-29T09:09:11.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.721+0000 7f1def400700 1 -- 172.21.15.99:0/3547946245 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1de80075a0 con 0x7f1df80d1e90 2024-03-29T09:09:11.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.722+0000 7f1def400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1dd806ac90 0x7f1dd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:11.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.722+0000 7f1def400700 1 -- 172.21.15.99:0/3547946245 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1de8088950 con 0x7f1df80d1e90 2024-03-29T09:09:11.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.722+0000 7f1df6a00700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1dd806ac90 0x7f1dd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:11.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.723+0000 7f1df6a00700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1dd806ac90 0x7f1dd806d150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1de4001b80 tx=0x7f1de4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:11.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.727+0000 7f1def400700 1 -- 172.21.15.99:0/3547946245 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1de80538e0 con 0x7f1df80d1e90 2024-03-29T09:09:11.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.928+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1df80b23d0 con 0x7f1df80d1e90 2024-03-29T09:09:11.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.929+0000 7f1def400700 1 -- 172.21.15.99:0/3547946245 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1de80170a0 con 0x7f1df80d1e90 2024-03-29T09:09:11.929 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1dd806ac90 msgr2=0x7f1dd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1dd806ac90 0x7f1dd806d150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1de4001b80 tx=0x7f1de4009040 comp rx=0 tx=0).stop 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df80d1e90 msgr2=0x7f1df80b7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df80d1e90 0x7f1df80b7fa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1de800c700 tx=0x7f1de800ca10 comp rx=0 tx=0).stop 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 shutdown_connections 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1dd806ac90 0x7f1dd806d150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1df815e600 0x7f1df815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1df80d1e90 0x7f1df80b7fa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 --2- 172.21.15.99:0/3547946245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1df815d420 0x7f1df8152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:11.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 >> 172.21.15.99:0/3547946245 conn(0x7f1df808afd0 msgr2=0x7f1df80d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:11.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 shutdown_connections 2024-03-29T09:09:11.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:11.931+0000 7f1dfe400700 1 -- 172.21.15.99:0/3547946245 wait complete. 2024-03-29T09:09:11.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:09:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.142+0000 7f969ee00700 1 Processor -- start 2024-03-29T09:09:12.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.143+0000 7f969ee00700 1 -- start start 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ee00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96980b3cc0 0x7f96980d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ee00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96980d3380 0x7f96980ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ee00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96980d3810 con 0x7f9698160770 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ee00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980b4140 con 0x7f96980d3380 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ee00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96980d1520 con 0x7f96980b3cc0 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ce00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96980d3380 0x7f96980ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f9697e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9698160770 0x7f96980d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37516/0 (socket says 172.21.15.99:37516) 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f9697e00700 1 -- 172.21.15.99:0/291359482 learned_addr learned my addr 172.21.15.99:0/291359482 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f968fe00700 1 --2- 172.21.15.99:0/291359482 >> v2:172.21.15.133:3300/0 conn(0x7f96980b3cc0 0x7f96980d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ce00700 1 -- 172.21.15.99:0/291359482 >> v2:172.21.15.133:3300/0 conn(0x7f96980b3cc0 msgr2=0x7f96980d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ce00700 1 --2- 172.21.15.99:0/291359482 >> v2:172.21.15.133:3300/0 conn(0x7f96980b3cc0 0x7f96980d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ce00700 1 -- 172.21.15.99:0/291359482 >> v1:172.21.15.99:6789/0 conn(0x7f9698160770 legacy=0x7f96980d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f969ce00700 1 -- 172.21.15.99:0/291359482 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96980d1800 con 0x7f96980d3380 2024-03-29T09:09:12.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.144+0000 7f9697400700 1 -- 172.21.15.99:0/291359482 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900358157 0 0) 0x7f96980d3810 con 0x7f9698160770 2024-03-29T09:09:12.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f969ce00700 1 --2- 172.21.15.99:0/291359482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96980d3380 0x7f96980ce080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96980b5cf0 tx=0x7f969000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4d55d22d45e2fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:12.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f9697400700 1 -- 172.21.15.99:0/291359482 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969000e070 con 0x7f96980d3380 2024-03-29T09:09:12.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f9697400700 1 -- 172.21.15.99:0/291359482 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969000bb20 con 0x7f96980d3380 2024-03-29T09:09:12.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f9697400700 1 -- 172.21.15.99:0/291359482 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690012430 con 0x7f96980d3380 2024-03-29T09:09:12.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f969ee00700 1 -- 172.21.15.99:0/291359482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96980d3380 msgr2=0x7f96980ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f969ee00700 1 --2- 172.21.15.99:0/291359482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96980d3380 0x7f96980ce080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f96980b5cf0 tx=0x7f969000b330 comp rx=0 tx=0).stop 2024-03-29T09:09:12.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f969ee00700 1 -- 172.21.15.99:0/291359482 shutdown_connections 2024-03-29T09:09:12.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f969ee00700 1 --2- 172.21.15.99:0/291359482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96980d3380 0x7f96980ce080 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.145+0000 7f969ee00700 1 --2- 172.21.15.99:0/291359482 >> v2:172.21.15.133:3300/0 conn(0x7f96980b3cc0 0x7f96980d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.146+0000 7f969ee00700 1 -- 172.21.15.99:0/291359482 >> 172.21.15.99:0/291359482 conn(0x7f969808afd0 msgr2=0x7f969809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:12.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.146+0000 7f969ee00700 1 -- 172.21.15.99:0/291359482 shutdown_connections 2024-03-29T09:09:12.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.146+0000 7f969ee00700 1 -- 172.21.15.99:0/291359482 wait complete. 2024-03-29T09:09:12.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.147+0000 7f969ee00700 1 Processor -- start 2024-03-29T09:09:12.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.147+0000 7f969ee00700 1 -- start start 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.148+0000 7f969ee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980b3cc0 0x7f96980b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.148+0000 7f969ee00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96980d3380 0x7f969815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.148+0000 7f969ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9698160770 0x7f969815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.148+0000 7f969ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980cacb0 con 0x7f9698160770 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.148+0000 7f969ee00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96980caf20 con 0x7f96980d3380 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.148+0000 7f969ee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96980cb200 con 0x7f96980b3cc0 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.148+0000 7f969ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9698160770 0x7f969815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9698160770 0x7f969815dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59528/0 (socket says 172.21.15.99:59528) 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 -- 172.21.15.99:0/3206539646 learned_addr learned my addr 172.21.15.99:0/3206539646 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f9697e00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980b3cc0 0x7f96980b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f968fe00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96980d3380 0x7f969815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 -- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980b3cc0 msgr2=0x7f96980b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980b3cc0 0x7f96980b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 -- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96980d3380 msgr2=0x7f969815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96980d3380 0x7f969815d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 -- 172.21.15.99:0/3206539646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009a70 con 0x7f9698160770 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ce00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9698160770 0x7f969815dc30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96980d0d10 tx=0x7f9690004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f9694c00700 1 -- 172.21.15.99:0/3206539646 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690012430 con 0x7f9698160770 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f9694c00700 1 -- 172.21.15.99:0/3206539646 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9690020070 con 0x7f9698160770 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96980cb4e0 con 0x7f9698160770 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.149+0000 7f9694c00700 1 -- 172.21.15.99:0/3206539646 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690004280 con 0x7f9698160770 2024-03-29T09:09:12.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.150+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96980b15c0 con 0x7f9698160770 2024-03-29T09:09:12.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.150+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96980cba00 con 0x7f9698160770 2024-03-29T09:09:12.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.151+0000 7f9694c00700 1 -- 172.21.15.99:0/3206539646 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f96900043e0 con 0x7f9698160770 2024-03-29T09:09:12.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.152+0000 7f9694c00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f967406acc0 0x7f967406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.152+0000 7f9694c00700 1 -- 172.21.15.99:0/3206539646 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9690022070 con 0x7f9698160770 2024-03-29T09:09:12.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.152+0000 7f9697e00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f967406acc0 0x7f967406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.153+0000 7f9697e00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f967406acc0 0x7f967406d180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9688000f60 tx=0x7f9688007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:12.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.157+0000 7f9694c00700 1 -- 172.21.15.99:0/3206539646 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f96980cba00 con 0x7f9698160770 2024-03-29T09:09:12.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.360+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f96980cba00 con 0x7f9698160770 2024-03-29T09:09:12.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.360+0000 7f9694c00700 1 -- 172.21.15.99:0/3206539646 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f96980cba00 con 0x7f9698160770 2024-03-29T09:09:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f967406acc0 msgr2=0x7f967406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f967406acc0 0x7f967406d180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9688000f60 tx=0x7f9688007e70 comp rx=0 tx=0).stop 2024-03-29T09:09:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9698160770 msgr2=0x7f969815dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9698160770 0x7f969815dc30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f96980d0d10 tx=0x7f9690004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 shutdown_connections 2024-03-29T09:09:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f967406acc0 0x7f967406d180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9698160770 0x7f969815dc30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980b3cc0 0x7f96980b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 --2- 172.21.15.99:0/3206539646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96980d3380 0x7f969815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 >> 172.21.15.99:0/3206539646 conn(0x7f969808afd0 msgr2=0x7f969806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 shutdown_connections 2024-03-29T09:09:12.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.363+0000 7f969ee00700 1 -- 172.21.15.99:0/3206539646 wait complete. 2024-03-29T09:09:12.382 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:09:12.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.570+0000 7efeeb200700 1 Processor -- start 2024-03-29T09:09:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.571+0000 7efeeb200700 1 -- start start 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efeeb200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee415d980 0x7efee40cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efeeb200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efee40b1780 0x7efee40ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efeeb200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee40d1a50 con 0x7efee40cfd70 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efeeb200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee40d2260 con 0x7efee40b1780 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efeeb200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee40d1ca0 con 0x7efee415d980 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efee9200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efee40b1780 0x7efee40ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efee9200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efee40b1780 0x7efee40ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59536/0 (socket says 172.21.15.99:59536) 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efee9200700 1 -- 172.21.15.99:0/2413789196 learned_addr learned my addr 172.21.15.99:0/2413789196 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efedbe00700 1 --2- 172.21.15.99:0/2413789196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee415d980 0x7efee40cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efee9200700 1 -- 172.21.15.99:0/2413789196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee415d980 msgr2=0x7efee40cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efee9200700 1 --2- 172.21.15.99:0/2413789196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee415d980 0x7efee40cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efee9200700 1 -- 172.21.15.99:0/2413789196 >> v1:172.21.15.133:6789/0 conn(0x7efee40cfd70 legacy=0x7efee415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efee9200700 1 -- 172.21.15.99:0/2413789196 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee40d1f80 con 0x7efee40b1780 2024-03-29T09:09:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.572+0000 7efedbe00700 1 --2- 172.21.15.99:0/2413789196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee415d980 0x7efee40cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efee9200700 1 --2- 172.21.15.99:0/2413789196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee40b1780 0x7efee40ce270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efedc009e50 tx=0x7efedc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f509703ca9a432 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efee3400700 1 -- 172.21.15.99:0/2413789196 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc00e070 con 0x7efee40b1780 2024-03-29T09:09:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efee3400700 1 -- 172.21.15.99:0/2413789196 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc00b9c0 con 0x7efee40b1780 2024-03-29T09:09:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efee3400700 1 -- 172.21.15.99:0/2413789196 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc00bd70 con 0x7efee40b1780 2024-03-29T09:09:12.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efeeb200700 1 -- 172.21.15.99:0/2413789196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee40b1780 msgr2=0x7efee40ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efeeb200700 1 --2- 172.21.15.99:0/2413789196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee40b1780 0x7efee40ce270 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efedc009e50 tx=0x7efedc00b330 comp rx=0 tx=0).stop 2024-03-29T09:09:12.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efeeb200700 1 -- 172.21.15.99:0/2413789196 shutdown_connections 2024-03-29T09:09:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efeeb200700 1 --2- 172.21.15.99:0/2413789196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee40b1780 0x7efee40ce270 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efeeb200700 1 --2- 172.21.15.99:0/2413789196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee415d980 0x7efee40cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.573+0000 7efeeb200700 1 -- 172.21.15.99:0/2413789196 >> 172.21.15.99:0/2413789196 conn(0x7efee409c470 msgr2=0x7efee40d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.574+0000 7efeeb200700 1 -- 172.21.15.99:0/2413789196 shutdown_connections 2024-03-29T09:09:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.574+0000 7efeeb200700 1 -- 172.21.15.99:0/2413789196 wait complete. 2024-03-29T09:09:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.575+0000 7efeeb200700 1 Processor -- start 2024-03-29T09:09:12.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.575+0000 7efeeb200700 1 -- start start 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.576+0000 7efeeb200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee40b1780 0x7efee415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.576+0000 7efeeb200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee40cfd70 0x7efee40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.576+0000 7efeeb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee415d980 0x7efee40cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.576+0000 7efeeb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee41f39a0 con 0x7efee415d980 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.576+0000 7efeeb200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee415c0f0 con 0x7efee40b1780 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.576+0000 7efeeb200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee40ca320 con 0x7efee40cfd70 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.576+0000 7efee9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee415d980 0x7efee40cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee3e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee40b1780 0x7efee415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efedbe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee40cfd70 0x7efee40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee415d980 0x7efee40cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59542/0 (socket says 172.21.15.99:59542) 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 -- 172.21.15.99:0/1333240058 learned_addr learned my addr 172.21.15.99:0/1333240058 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 -- 172.21.15.99:0/1333240058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee40cfd70 msgr2=0x7efee40cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee40cfd70 0x7efee40cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 -- 172.21.15.99:0/1333240058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee40b1780 msgr2=0x7efee415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee40b1780 0x7efee415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 -- 172.21.15.99:0/1333240058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc009b30 con 0x7efee415d980 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee9200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee415d980 0x7efee40cbac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efee408d8d0 tx=0x7efedc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee0c00700 1 -- 172.21.15.99:0/1333240058 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc004800 con 0x7efee415d980 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee0c00700 1 -- 172.21.15.99:0/1333240058 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc004960 con 0x7efee415d980 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efee0c00700 1 -- 172.21.15.99:0/1333240058 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc01c680 con 0x7efee415d980 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.577+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee40ca600 con 0x7efee415d980 2024-03-29T09:09:12.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.578+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee40cabc0 con 0x7efee415d980 2024-03-29T09:09:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.578+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee40b4a30 con 0x7efee415d980 2024-03-29T09:09:12.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.579+0000 7efee0c00700 1 -- 172.21.15.99:0/1333240058 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efedc01c7e0 con 0x7efee415d980 2024-03-29T09:09:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.580+0000 7efee0c00700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efec006ac90 0x7efec006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.580+0000 7efee0c00700 1 -- 172.21.15.99:0/1333240058 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efedc08b220 con 0x7efee415d980 2024-03-29T09:09:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.580+0000 7efee3e00700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efec006ac90 0x7efec006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:12.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.581+0000 7efee3e00700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efec006ac90 0x7efec006d150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efee415d2f0 tx=0x7efed4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:12.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.584+0000 7efee0c00700 1 -- 172.21.15.99:0/1333240058 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efedc056100 con 0x7efee415d980 2024-03-29T09:09:12.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.805+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efee4067d80 con 0x7efee415d980 2024-03-29T09:09:12.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.806+0000 7efee0c00700 1 -- 172.21.15.99:0/1333240058 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7efedc059720 con 0x7efee415d980 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efec006ac90 msgr2=0x7efec006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efec006ac90 0x7efec006d150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efee415d2f0 tx=0x7efed4009040 comp rx=0 tx=0).stop 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee415d980 msgr2=0x7efee40cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee415d980 0x7efee40cbac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efee408d8d0 tx=0x7efedc004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 shutdown_connections 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efec006ac90 0x7efec006d150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee415d980 0x7efee40cbac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee40b1780 0x7efee415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 --2- 172.21.15.99:0/1333240058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efee40cfd70 0x7efee40cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 >> 172.21.15.99:0/1333240058 conn(0x7efee409c470 msgr2=0x7efee40b4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 shutdown_connections 2024-03-29T09:09:12.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:12.808+0000 7efeeb200700 1 -- 172.21.15.99:0/1333240058 wait complete. 2024-03-29T09:09:12.810 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:09:12.826 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:09:12.826 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:09:12.868 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.098+0000 7f745da00700 1 Processor -- start 2024-03-29T09:09:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.098+0000 7f745da00700 1 -- start start 2024-03-29T09:09:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f745da00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f74580d1b20 0x7f74580d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f745da00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7458160cc0 0x7f74580ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f745da00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580cddc0 con 0x7f74581608e0 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f745da00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580ce0a0 con 0x7f7458160cc0 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f745da00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74580ce380 con 0x7f74580d1b20 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7457400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7458160cc0 0x7f74580ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7456a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f74581608e0 0x7f74580d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37524/0 (socket says 172.21.15.99:37524) 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7457400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7458160cc0 0x7f74580ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59546/0 (socket says 172.21.15.99:59546) 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7456a00700 1 -- 172.21.15.99:0/3120120287 learned_addr learned my addr 172.21.15.99:0/3120120287 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7456000700 1 --2- 172.21.15.99:0/3120120287 >> v2:172.21.15.133:3300/0 conn(0x7f74580d1b20 0x7f74580d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7457400700 1 -- 172.21.15.99:0/3120120287 >> v2:172.21.15.133:3300/0 conn(0x7f74580d1b20 msgr2=0x7f74580d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7457400700 1 --2- 172.21.15.99:0/3120120287 >> v2:172.21.15.133:3300/0 conn(0x7f74580d1b20 0x7f74580d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7457400700 1 -- 172.21.15.99:0/3120120287 >> v1:172.21.15.99:6789/0 conn(0x7f74581608e0 legacy=0x7f74580d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7457400700 1 -- 172.21.15.99:0/3120120287 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580ce660 con 0x7f7458160cc0 2024-03-29T09:09:13.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.100+0000 7f7456000700 1 --2- 172.21.15.99:0/3120120287 >> v2:172.21.15.133:3300/0 conn(0x7f74580d1b20 0x7f74580d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:13.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f7457400700 1 --2- 172.21.15.99:0/3120120287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458160cc0 0x7f74580ceb80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f745806ac50 tx=0x7f745000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b473485f2777705 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f7455600700 1 -- 172.21.15.99:0/3120120287 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745000e070 con 0x7f7458160cc0 2024-03-29T09:09:13.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f7455600700 1 -- 172.21.15.99:0/3120120287 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745000bbd0 con 0x7f7458160cc0 2024-03-29T09:09:13.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f7455600700 1 -- 172.21.15.99:0/3120120287 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450012430 con 0x7f7458160cc0 2024-03-29T09:09:13.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f745da00700 1 -- 172.21.15.99:0/3120120287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458160cc0 msgr2=0x7f74580ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f745da00700 1 --2- 172.21.15.99:0/3120120287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458160cc0 0x7f74580ceb80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f745806ac50 tx=0x7f745000b330 comp rx=0 tx=0).stop 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f745da00700 1 -- 172.21.15.99:0/3120120287 shutdown_connections 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f745da00700 1 --2- 172.21.15.99:0/3120120287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7458160cc0 0x7f74580ceb80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f745da00700 1 --2- 172.21.15.99:0/3120120287 >> v2:172.21.15.133:3300/0 conn(0x7f74580d1b20 0x7f74580d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.101+0000 7f745da00700 1 -- 172.21.15.99:0/3120120287 >> 172.21.15.99:0/3120120287 conn(0x7f745809c490 msgr2=0x7f74580d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.102+0000 7f745da00700 1 -- 172.21.15.99:0/3120120287 shutdown_connections 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.102+0000 7f745da00700 1 -- 172.21.15.99:0/3120120287 wait complete. 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.103+0000 7f745da00700 1 Processor -- start 2024-03-29T09:09:13.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.103+0000 7f745da00700 1 -- start start 2024-03-29T09:09:13.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.104+0000 7f745da00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74580d1b20 0x7f745815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.104+0000 7f745da00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74581608e0 0x7f74580ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.104+0000 7f745da00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7458160cc0 0x7f74581580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.104+0000 7f745da00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580cae50 con 0x7f74580d1b20 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.104+0000 7f745da00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c9610 con 0x7f7458160cc0 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.104+0000 7f745da00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7458157c30 con 0x7f74581608e0 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74580d1b20 0x7f745815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74580d1b20 0x7f745815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59550/0 (socket says 172.21.15.99:59550) 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 -- 172.21.15.99:0/3176297062 learned_addr learned my addr 172.21.15.99:0/3176297062 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7457400700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7458160cc0 0x7f74581580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456000700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74581608e0 0x7f74580ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 -- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74581608e0 msgr2=0x7f74580ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74581608e0 0x7f74580ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 -- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7458160cc0 msgr2=0x7f74581580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7458160cc0 0x7f74581580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 -- 172.21.15.99:0/3176297062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450009a70 con 0x7f74580d1b20 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f7456a00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74580d1b20 0x7f745815a790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f744800c550 tx=0x7f744800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f744ea00700 1 -- 172.21.15.99:0/3176297062 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448005d70 con 0x7f74580d1b20 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f744ea00700 1 -- 172.21.15.99:0/3176297062 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74480070d0 con 0x7f74580d1b20 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f744ea00700 1 -- 172.21.15.99:0/3176297062 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448007440 con 0x7f74580d1b20 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.105+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745815acd0 con 0x7f74580d1b20 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.106+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580cd2e0 con 0x7f74580d1b20 2024-03-29T09:09:13.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.107+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74580b2a70 con 0x7f74580d1b20 2024-03-29T09:09:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.108+0000 7f744ea00700 1 -- 172.21.15.99:0/3176297062 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f74480075a0 con 0x7f74580d1b20 2024-03-29T09:09:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.108+0000 7f744ea00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f743806ac90 0x7f743806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.108+0000 7f744ea00700 1 -- 172.21.15.99:0/3176297062 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7448088a40 con 0x7f74580d1b20 2024-03-29T09:09:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.109+0000 7f7456000700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f743806ac90 0x7f743806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.109+0000 7f7456000700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f743806ac90 0x7f743806d150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7458159730 tx=0x7f7444008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.113+0000 7f744ea00700 1 -- 172.21.15.99:0/3176297062 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f74480539c0 con 0x7f74580d1b20 2024-03-29T09:09:13.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.317+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7458157f10 con 0x7f74580d1b20 2024-03-29T09:09:13.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.318+0000 7f744ea00700 1 -- 172.21.15.99:0/3176297062 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f7448017090 con 0x7f74580d1b20 2024-03-29T09:09:13.319 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:13.319 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f743806ac90 msgr2=0x7f743806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f743806ac90 0x7f743806d150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7458159730 tx=0x7f7444008040 comp rx=0 tx=0).stop 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74580d1b20 msgr2=0x7f745815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74580d1b20 0x7f745815a790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f744800c550 tx=0x7f744800c910 comp rx=0 tx=0).stop 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 shutdown_connections 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f743806ac90 0x7f743806d150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7458160cc0 0x7f74581580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74580d1b20 0x7f745815a790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 --2- 172.21.15.99:0/3176297062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74581608e0 0x7f74580ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 >> 172.21.15.99:0/3176297062 conn(0x7f745809c490 msgr2=0x7f7458155dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:13.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 shutdown_connections 2024-03-29T09:09:13.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.321+0000 7f745da00700 1 -- 172.21.15.99:0/3176297062 wait complete. 2024-03-29T09:09:13.339 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:09:13.340 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:09:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.536+0000 7f2ee0000700 1 Processor -- start 2024-03-29T09:09:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.536+0000 7f2ee0000700 1 -- start start 2024-03-29T09:09:13.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.537+0000 7f2ee0000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2ed8160770 0x7f2ed80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.537+0000 7f2ee0000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed80d3550 0x7f2ed80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.537+0000 7f2ee0000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed80d1520 con 0x7f2ed80d0d60 2024-03-29T09:09:13.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.537+0000 7f2ee0000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed80d1800 con 0x7f2ed8160770 2024-03-29T09:09:13.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.537+0000 7f2ee0000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80d1ae0 con 0x7f2ed80d3550 2024-03-29T09:09:13.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2edcc00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2ed80d0d60 0x7f2ed80d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37530/0 (socket says 172.21.15.99:37530) 2024-03-29T09:09:13.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2edcc00700 1 -- 172.21.15.99:0/1060304192 learned_addr learned my addr 172.21.15.99:0/1060304192 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:13.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2edd600700 1 --2- 172.21.15.99:0/1060304192 >> v2:172.21.15.133:3300/0 conn(0x7f2ed8160770 0x7f2ed80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ede000700 1 --2- 172.21.15.99:0/1060304192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed80d3550 0x7f2ed80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ed7e00700 1 -- 172.21.15.99:0/1060304192 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817573256 0 0) 0x7f2ed80d1520 con 0x7f2ed80d0d60 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ed7e00700 1 -- 172.21.15.99:0/1060304192 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ec00035b0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ede000700 1 -- 172.21.15.99:0/1060304192 >> v2:172.21.15.133:3300/0 conn(0x7f2ed8160770 msgr2=0x7f2ed80b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ede000700 1 --2- 172.21.15.99:0/1060304192 >> v2:172.21.15.133:3300/0 conn(0x7f2ed8160770 0x7f2ed80b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ede000700 1 -- 172.21.15.99:0/1060304192 >> v1:172.21.15.99:6789/0 conn(0x7f2ed80d0d60 legacy=0x7f2ed80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ede000700 1 -- 172.21.15.99:0/1060304192 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed80d1dc0 con 0x7f2ed80d3550 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2edd600700 1 --2- 172.21.15.99:0/1060304192 >> v2:172.21.15.133:3300/0 conn(0x7f2ed8160770 0x7f2ed80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.538+0000 7f2ede000700 1 --2- 172.21.15.99:0/1060304192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed80d3550 0x7f2ed80ce080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009d90 tx=0x7f2ed000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebf583565730fead server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ed7e00700 1 -- 172.21.15.99:0/1060304192 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000e070 con 0x7f2ed80d3550 2024-03-29T09:09:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ed7e00700 1 -- 172.21.15.99:0/1060304192 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed000bb20 con 0x7f2ed80d3550 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ed7e00700 1 -- 172.21.15.99:0/1060304192 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0012430 con 0x7f2ed80d3550 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 -- 172.21.15.99:0/1060304192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed80d3550 msgr2=0x7f2ed80ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 --2- 172.21.15.99:0/1060304192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed80d3550 0x7f2ed80ce080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009d90 tx=0x7f2ed000b330 comp rx=0 tx=0).stop 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 -- 172.21.15.99:0/1060304192 shutdown_connections 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 --2- 172.21.15.99:0/1060304192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed80d3550 0x7f2ed80ce080 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 --2- 172.21.15.99:0/1060304192 >> v2:172.21.15.133:3300/0 conn(0x7f2ed8160770 0x7f2ed80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 -- 172.21.15.99:0/1060304192 >> 172.21.15.99:0/1060304192 conn(0x7f2ed808afd0 msgr2=0x7f2ed809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 -- 172.21.15.99:0/1060304192 shutdown_connections 2024-03-29T09:09:13.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.539+0000 7f2ee0000700 1 -- 172.21.15.99:0/1060304192 wait complete. 2024-03-29T09:09:13.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.541+0000 7f2ee0000700 1 Processor -- start 2024-03-29T09:09:13.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.541+0000 7f2ee0000700 1 -- start start 2024-03-29T09:09:13.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2ee0000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed80d0d60 0x7f2ed80b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2ee0000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed80d3550 0x7f2ed80afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2ee0000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed8160770 0x7f2ed80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2ee0000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed809e7c0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2ee0000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed80cd3e0 con 0x7f2ed80d3550 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2ee0000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed814d700 con 0x7f2ed8160770 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2edd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed80d0d60 0x7f2ed80b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2edd600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed80d0d60 0x7f2ed80b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59556/0 (socket says 172.21.15.99:59556) 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.542+0000 7f2edd600700 1 -- 172.21.15.99:0/263813607 learned_addr learned my addr 172.21.15.99:0/263813607 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2edcc00700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed80d3550 0x7f2ed80afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2ede000700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed8160770 0x7f2ed80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2edd600700 1 -- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed8160770 msgr2=0x7f2ed80b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2edd600700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed8160770 0x7f2ed80b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2edd600700 1 -- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed80d3550 msgr2=0x7f2ed80afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2edd600700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed80d3550 0x7f2ed80afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2edd600700 1 -- 172.21.15.99:0/263813607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc0094a0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2edd600700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed80d0d60 0x7f2ed80b1190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00ebe0 tx=0x7f2ecc009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2ed5600700 1 -- 172.21.15.99:0/263813607 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00c8b0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2ed5600700 1 -- 172.21.15.99:0/263813607 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc00ca10 con 0x7f2ed80d0d60 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2ed5600700 1 -- 172.21.15.99:0/263813607 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc00cda0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.543+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed0009a70 con 0x7f2ed80d0d60 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.544+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed814dcc0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.545+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed814dee0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.546+0000 7f2ed5600700 1 -- 172.21.15.99:0/263813607 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2ecc013470 con 0x7f2ed80d0d60 2024-03-29T09:09:13.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.547+0000 7f2ed5600700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2eb806ac90 0x7f2eb806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.547+0000 7f2ed5600700 1 -- 172.21.15.99:0/263813607 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2ecc089880 con 0x7f2ed80d0d60 2024-03-29T09:09:13.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.547+0000 7f2edcc00700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2eb806ac90 0x7f2eb806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.548+0000 7f2edcc00700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2eb806ac90 0x7f2eb806d150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80b8d20 tx=0x7f2ec4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.551+0000 7f2ed5600700 1 -- 172.21.15.99:0/263813607 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2ecc0546e0 con 0x7f2ed80d0d60 2024-03-29T09:09:13.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.752+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ed806c620 con 0x7f2ed80d0d60 2024-03-29T09:09:13.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.753+0000 7f2ed5600700 1 -- 172.21.15.99:0/263813607 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ecc057d00 con 0x7f2ed80d0d60 2024-03-29T09:09:13.753 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:09:13.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2eb806ac90 msgr2=0x7f2eb806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2eb806ac90 0x7f2eb806d150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80b8d20 tx=0x7f2ec4009040 comp rx=0 tx=0).stop 2024-03-29T09:09:13.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed80d0d60 msgr2=0x7f2ed80b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed80d0d60 0x7f2ed80b1190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc00ebe0 tx=0x7f2ecc009e20 comp rx=0 tx=0).stop 2024-03-29T09:09:13.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 shutdown_connections 2024-03-29T09:09:13.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2eb806ac90 0x7f2eb806d150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ed8160770 0x7f2ed80b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ed80d0d60 0x7f2ed80b1190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 --2- 172.21.15.99:0/263813607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ed80d3550 0x7f2ed80afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 >> 172.21.15.99:0/263813607 conn(0x7f2ed808afd0 msgr2=0x7f2ed80b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:13.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 shutdown_connections 2024-03-29T09:09:13.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.755+0000 7f2ee0000700 1 -- 172.21.15.99:0/263813607 wait complete. 2024-03-29T09:09:13.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:09:13.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.967+0000 7f2cb1a00700 1 Processor -- start 2024-03-29T09:09:13.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.968+0000 7f2cb1a00700 1 -- start start 2024-03-29T09:09:13.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2cb1a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac160770 0x7f2cac0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2cb1a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2cac0d0d60 0x7f2cac0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2cb1a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac0d3810 con 0x7f2cac0d3380 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2cb1a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac0d1520 con 0x7f2cac0d0d60 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2cb1a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0d1800 con 0x7f2cac160770 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2caa000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2cac0d0d60 0x7f2cac0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2caa000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2cac0d0d60 0x7f2cac0d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59558/0 (socket says 172.21.15.99:59558) 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2caa000700 1 -- 172.21.15.99:0/1208358482 learned_addr learned my addr 172.21.15.99:0/1208358482 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2caa000700 1 -- 172.21.15.99:0/1208358482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac160770 msgr2=0x7f2cac0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2caa000700 1 --2- 172.21.15.99:0/1208358482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac160770 0x7f2cac0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2caa000700 1 -- 172.21.15.99:0/1208358482 >> v1:172.21.15.133:6789/0 conn(0x7f2cac0d3380 legacy=0x7f2cac0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.969+0000 7f2caa000700 1 -- 172.21.15.99:0/1208358482 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0d1ae0 con 0x7f2cac0d0d60 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2caa000700 1 --2- 172.21.15.99:0/1208358482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac0d0d60 0x7f2cac0d2e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c980098f0 tx=0x7f2c98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d8d5ef3c2a2c6d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2ca9600700 1 -- 172.21.15.99:0/1208358482 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98004230 con 0x7f2cac0d0d60 2024-03-29T09:09:13.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2ca9600700 1 -- 172.21.15.99:0/1208358482 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98004390 con 0x7f2cac0d0d60 2024-03-29T09:09:13.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2ca9600700 1 -- 172.21.15.99:0/1208358482 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980036a0 con 0x7f2cac0d0d60 2024-03-29T09:09:13.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 -- 172.21.15.99:0/1208358482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac0d0d60 msgr2=0x7f2cac0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/1208358482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac0d0d60 0x7f2cac0d2e40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2c980098f0 tx=0x7f2c98009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 -- 172.21.15.99:0/1208358482 shutdown_connections 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/1208358482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac160770 0x7f2cac0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/1208358482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac0d0d60 0x7f2cac0d2e40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 -- 172.21.15.99:0/1208358482 >> 172.21.15.99:0/1208358482 conn(0x7f2cac08afd0 msgr2=0x7f2cac09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 -- 172.21.15.99:0/1208358482 shutdown_connections 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.970+0000 7f2cb1a00700 1 -- 172.21.15.99:0/1208358482 wait complete. 2024-03-29T09:09:13.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.972+0000 7f2cb1a00700 1 Processor -- start 2024-03-29T09:09:13.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.972+0000 7f2cb1a00700 1 -- start start 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.973+0000 7f2cb1a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac0d0d60 0x7f2cac0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.973+0000 7f2cb1a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cac0d3380 0x7f2cac0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.973+0000 7f2cb1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac160770 0x7f2cac154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.973+0000 7f2cb1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0cc110 con 0x7f2cac160770 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.973+0000 7f2cb1a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0cab60 con 0x7f2cac0d3380 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.973+0000 7f2cb1a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac0cae40 con 0x7f2cac0d0d60 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac160770 0x7f2cac154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2caa000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cac0d3380 0x7f2cac0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2caaa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac0d0d60 0x7f2cac0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac160770 0x7f2cac154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59574/0 (socket says 172.21.15.99:59574) 2024-03-29T09:09:13.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 -- 172.21.15.99:0/2830949011 learned_addr learned my addr 172.21.15.99:0/2830949011 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 -- 172.21.15.99:0/2830949011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac0d0d60 msgr2=0x7f2cac0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac0d0d60 0x7f2cac0ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 -- 172.21.15.99:0/2830949011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cac0d3380 msgr2=0x7f2cac0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cac0d3380 0x7f2cac0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 -- 172.21.15.99:0/2830949011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98009550 con 0x7f2cac160770 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2cab400700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac160770 0x7f2cac154820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400cc10 tx=0x7f2ca400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.974+0000 7f2ca2a00700 1 -- 172.21.15.99:0/2830949011 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400aa60 con 0x7f2cac160770 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.975+0000 7f2ca2a00700 1 -- 172.21.15.99:0/2830949011 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca400abc0 con 0x7f2cac160770 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.975+0000 7f2ca2a00700 1 -- 172.21.15.99:0/2830949011 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4012420 con 0x7f2cac160770 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.975+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac0cb120 con 0x7f2cac160770 2024-03-29T09:09:13.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.975+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac0b15c0 con 0x7f2cac160770 2024-03-29T09:09:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.976+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac0cbe70 con 0x7f2cac160770 2024-03-29T09:09:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.977+0000 7f2ca2a00700 1 -- 172.21.15.99:0/2830949011 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2ca4012580 con 0x7f2cac160770 2024-03-29T09:09:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.977+0000 7f2ca2a00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2c8c06ac90 0x7f2c8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.977+0000 7f2ca2a00700 1 -- 172.21.15.99:0/2830949011 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2ca4089940 con 0x7f2cac160770 2024-03-29T09:09:13.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.978+0000 7f2caaa00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2c8c06ac90 0x7f2c8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:13.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.978+0000 7f2caaa00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2c8c06ac90 0x7f2c8c06d150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009970 tx=0x7f2c9c009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:13.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:13.982+0000 7f2ca2a00700 1 -- 172.21.15.99:0/2830949011 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2cac0cbe70 con 0x7f2cac160770 2024-03-29T09:09:14.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.182+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cac0cc250 con 0x7f2cac160770 2024-03-29T09:09:14.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.183+0000 7f2ca2a00700 1 -- 172.21.15.99:0/2830949011 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cac0cc250 con 0x7f2cac160770 2024-03-29T09:09:14.183 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.185+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2c8c06ac90 msgr2=0x7f2c8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.185+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2c8c06ac90 0x7f2c8c06d150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009970 tx=0x7f2c9c009110 comp rx=0 tx=0).stop 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.185+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac160770 msgr2=0x7f2cac154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.185+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac160770 0x7f2cac154820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400cc10 tx=0x7f2ca400cf20 comp rx=0 tx=0).stop 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 shutdown_connections 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2c8c06ac90 0x7f2c8c06d150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cac160770 0x7f2cac154820 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2cac0d0d60 0x7f2cac0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 --2- 172.21.15.99:0/2830949011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2cac0d3380 0x7f2cac0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 >> 172.21.15.99:0/2830949011 conn(0x7f2cac08afd0 msgr2=0x7f2cac073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:14.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 shutdown_connections 2024-03-29T09:09:14.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.186+0000 7f2cb1a00700 1 -- 172.21.15.99:0/2830949011 wait complete. 2024-03-29T09:09:14.204 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:09:14.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.395+0000 7f37b8400700 1 Processor -- start 2024-03-29T09:09:14.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.395+0000 7f37b8400700 1 -- start start 2024-03-29T09:09:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b8400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b8400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f37b00d3380 0x7f37b00ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b8400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00d3810 con 0x7f37b0160770 2024-03-29T09:09:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b8400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b00b4140 con 0x7f37b00d3380 2024-03-29T09:09:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b8400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00d1520 con 0x7f37b00b3cc0 2024-03-29T09:09:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b5000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b6400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f37b00d3380 0x7f37b00ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b5000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:33720/0 (socket says 172.21.15.99:33720) 2024-03-29T09:09:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b6400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f37b00d3380 0x7f37b00ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:42088/0 (socket says 172.21.15.99:42088) 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.396+0000 7f37b5000700 1 -- 172.21.15.99:0/2343537657 learned_addr learned my addr 172.21.15.99:0/2343537657 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37b6400700 1 -- 172.21.15.99:0/2343537657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 msgr2=0x7f37b00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37b6400700 1 --2- 172.21.15.99:0/2343537657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37b6400700 1 -- 172.21.15.99:0/2343537657 >> v1:172.21.15.133:6789/0 conn(0x7f37b0160770 legacy=0x7f37b00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37b6400700 1 -- 172.21.15.99:0/2343537657 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00d1800 con 0x7f37b00d3380 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37afe00700 1 -- 172.21.15.99:0/2343537657 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203484805 0 0) 0x7f37b00d3810 con 0x7f37b0160770 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37b6400700 1 --2- 172.21.15.99:0/2343537657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b00d3380 0x7f37b00ce080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009d90 tx=0x7f37a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab449a699ce6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37afe00700 1 -- 172.21.15.99:0/2343537657 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a800e070 con 0x7f37b00d3380 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37afe00700 1 -- 172.21.15.99:0/2343537657 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a800bb20 con 0x7f37b00d3380 2024-03-29T09:09:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.397+0000 7f37afe00700 1 -- 172.21.15.99:0/2343537657 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8012430 con 0x7f37b00d3380 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 -- 172.21.15.99:0/2343537657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b00d3380 msgr2=0x7f37b00ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 --2- 172.21.15.99:0/2343537657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b00d3380 0x7f37b00ce080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009d90 tx=0x7f37a800b330 comp rx=0 tx=0).stop 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 -- 172.21.15.99:0/2343537657 shutdown_connections 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 --2- 172.21.15.99:0/2343537657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b00d3380 0x7f37b00ce080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 --2- 172.21.15.99:0/2343537657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 -- 172.21.15.99:0/2343537657 >> 172.21.15.99:0/2343537657 conn(0x7f37b008afd0 msgr2=0x7f37b009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 -- 172.21.15.99:0/2343537657 shutdown_connections 2024-03-29T09:09:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.398+0000 7f37b8400700 1 -- 172.21.15.99:0/2343537657 wait complete. 2024-03-29T09:09:14.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.400+0000 7f37b8400700 1 Processor -- start 2024-03-29T09:09:14.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.400+0000 7f37b8400700 1 -- start start 2024-03-29T09:09:14.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b8400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b8400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b00d3380 0x7f37b00b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b8400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b0160770 0x7f37b00b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b8400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00b7340 con 0x7f37b00d3380 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b8400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00b7620 con 0x7f37b0160770 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b8400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00b7900 con 0x7f37b00b3cc0 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b00d3380 0x7f37b00b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b00d3380 0x7f37b00b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59582/0 (socket says 172.21.15.99:59582) 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b5000700 1 -- 172.21.15.99:0/272434724 learned_addr learned my addr 172.21.15.99:0/272434724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b5a00700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.401+0000 7f37b6400700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b0160770 0x7f37b00b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b5000700 1 -- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 msgr2=0x7f37b00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b5000700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b5000700 1 -- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b0160770 msgr2=0x7f37b00b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b5000700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b0160770 0x7f37b00b83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b5000700 1 -- 172.21.15.99:0/272434724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a8009a70 con 0x7f37b00d3380 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b5000700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b00d3380 0x7f37b00b1080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e980 tx=0x7f379c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37ad600700 1 -- 172.21.15.99:0/272434724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00c690 con 0x7f37b00d3380 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37ad600700 1 -- 172.21.15.99:0/272434724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00c7f0 con 0x7f37b00d3380 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b00b7be0 con 0x7f37b00d3380 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37ad600700 1 -- 172.21.15.99:0/272434724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00cba0 con 0x7f37b00d3380 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.402+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b00cd530 con 0x7f37b00d3380 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.404+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b00b8100 con 0x7f37b00d3380 2024-03-29T09:09:14.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.404+0000 7f37ad600700 1 -- 172.21.15.99:0/272434724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f379c00cd00 con 0x7f37b00d3380 2024-03-29T09:09:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.406+0000 7f37ad600700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f379006ac90 0x7f379006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.406+0000 7f37ad600700 1 -- 172.21.15.99:0/272434724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f379c0896e0 con 0x7f37b00d3380 2024-03-29T09:09:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.406+0000 7f37b5a00700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f379006ac90 0x7f379006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.406+0000 7f37b5a00700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f379006ac90 0x7f379006d150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f60 tx=0x7f37a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:14.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.410+0000 7f37ad600700 1 -- 172.21.15.99:0/272434724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f37b00b8100 con 0x7f37b00d3380 2024-03-29T09:09:14.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.614+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37b00b8100 con 0x7f37b00d3380 2024-03-29T09:09:14.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.614+0000 7f37ad600700 1 -- 172.21.15.99:0/272434724 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37b00b8100 con 0x7f37b00d3380 2024-03-29T09:09:14.614 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:09:14.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f379006ac90 msgr2=0x7f379006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f379006ac90 0x7f379006d150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f60 tx=0x7f37a4009040 comp rx=0 tx=0).stop 2024-03-29T09:09:14.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b00d3380 msgr2=0x7f37b00b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b00d3380 0x7f37b00b1080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e980 tx=0x7f379c00ec90 comp rx=0 tx=0).stop 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 shutdown_connections 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f379006ac90 0x7f379006d150 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37b0160770 0x7f37b00b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f37b00b3cc0 0x7f37b00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 --2- 172.21.15.99:0/272434724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37b00d3380 0x7f37b00b1080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 >> 172.21.15.99:0/272434724 conn(0x7f37b008afd0 msgr2=0x7f37b00cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 shutdown_connections 2024-03-29T09:09:14.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.617+0000 7f37b8400700 1 -- 172.21.15.99:0/272434724 wait complete. 2024-03-29T09:09:14.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:09:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.826+0000 7f06bc400700 1 Processor -- start 2024-03-29T09:09:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.826+0000 7f06bc400700 1 -- start start 2024-03-29T09:09:14.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.827+0000 7f06bc400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b4160770 0x7f06b40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.827+0000 7f06bc400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06b40d3550 0x7f06b40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.827+0000 7f06bc400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b40d1520 con 0x7f06b40d0d60 2024-03-29T09:09:14.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.827+0000 7f06bc400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b40d1800 con 0x7f06b40d3550 2024-03-29T09:09:14.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.827+0000 7f06bc400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40d1ae0 con 0x7f06b4160770 2024-03-29T09:09:14.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b9000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f06b40d0d60 0x7f06b40d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37542/0 (socket says 172.21.15.99:37542) 2024-03-29T09:09:14.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b9000700 1 -- 172.21.15.99:0/3228565597 learned_addr learned my addr 172.21.15.99:0/3228565597 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:14.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b9a00700 1 --2- 172.21.15.99:0/3228565597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b4160770 0x7f06b40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06ba400700 1 --2- 172.21.15.99:0/3228565597 >> v2:172.21.15.133:3300/0 conn(0x7f06b40d3550 0x7f06b40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b3e00700 1 -- 172.21.15.99:0/3228565597 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560996916 0 0) 0x7f06b40d1520 con 0x7f06b40d0d60 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b3e00700 1 -- 172.21.15.99:0/3228565597 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f069c0035b0 con 0x7f06b40d0d60 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b9a00700 1 -- 172.21.15.99:0/3228565597 >> v2:172.21.15.133:3300/0 conn(0x7f06b40d3550 msgr2=0x7f06b40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b9a00700 1 --2- 172.21.15.99:0/3228565597 >> v2:172.21.15.133:3300/0 conn(0x7f06b40d3550 0x7f06b40ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b9a00700 1 -- 172.21.15.99:0/3228565597 >> v1:172.21.15.99:6789/0 conn(0x7f06b40d0d60 legacy=0x7f06b40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06b9a00700 1 -- 172.21.15.99:0/3228565597 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b40d1dc0 con 0x7f06b4160770 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.828+0000 7f06ba400700 1 --2- 172.21.15.99:0/3228565597 >> v2:172.21.15.133:3300/0 conn(0x7f06b40d3550 0x7f06b40ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.829+0000 7f06b9a00700 1 --2- 172.21.15.99:0/3228565597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b4160770 0x7f06b40b3cc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009d90 tx=0x7f06a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a5bf7f52e79b0c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.829+0000 7f06b3e00700 1 -- 172.21.15.99:0/3228565597 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800e070 con 0x7f06b4160770 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.829+0000 7f06b3e00700 1 -- 172.21.15.99:0/3228565597 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800bb20 con 0x7f06b4160770 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.829+0000 7f06b3e00700 1 -- 172.21.15.99:0/3228565597 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8012430 con 0x7f06b4160770 2024-03-29T09:09:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.829+0000 7f06bc400700 1 -- 172.21.15.99:0/3228565597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b4160770 msgr2=0x7f06b40b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.829+0000 7f06bc400700 1 --2- 172.21.15.99:0/3228565597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b4160770 0x7f06b40b3cc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009d90 tx=0x7f06a800b330 comp rx=0 tx=0).stop 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.830+0000 7f06bc400700 1 -- 172.21.15.99:0/3228565597 shutdown_connections 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.830+0000 7f06bc400700 1 --2- 172.21.15.99:0/3228565597 >> v2:172.21.15.133:3300/0 conn(0x7f06b40d3550 0x7f06b40ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.830+0000 7f06bc400700 1 --2- 172.21.15.99:0/3228565597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b4160770 0x7f06b40b3cc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.830+0000 7f06bc400700 1 -- 172.21.15.99:0/3228565597 >> 172.21.15.99:0/3228565597 conn(0x7f06b408afd0 msgr2=0x7f06b409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.830+0000 7f06bc400700 1 -- 172.21.15.99:0/3228565597 shutdown_connections 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.830+0000 7f06bc400700 1 -- 172.21.15.99:0/3228565597 wait complete. 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.831+0000 7f06bc400700 1 Processor -- start 2024-03-29T09:09:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.831+0000 7f06bc400700 1 -- start start 2024-03-29T09:09:14.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.832+0000 7f06bc400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06b40d0d60 0x7f06b40b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.832+0000 7f06bc400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b40d3550 0x7f06b40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.832+0000 7f06bc400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06b4160770 0x7f06b40ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.832+0000 7f06bc400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40b6df0 con 0x7f06b40d0d60 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.832+0000 7f06bc400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40b61e0 con 0x7f06b4160770 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.832+0000 7f06bc400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40b64c0 con 0x7f06b40d3550 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06b40d0d60 0x7f06b40b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06b40d0d60 0x7f06b40b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59586/0 (socket says 172.21.15.99:59586) 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 -- 172.21.15.99:0/1798144447 learned_addr learned my addr 172.21.15.99:0/1798144447 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9000700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b40d3550 0x7f06b40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06ba400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06b4160770 0x7f06b40ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 -- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b40d3550 msgr2=0x7f06b40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b40d3550 0x7f06b40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 -- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06b4160770 msgr2=0x7f06b40ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06b4160770 0x7f06b40ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 -- 172.21.15.99:0/1798144447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0094a0 con 0x7f06b40d0d60 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06ba400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06b4160770 0x7f06b40ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.833+0000 7f06b9a00700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06b40d0d60 0x7f06b40b7280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06a8014040 tx=0x7f06a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.834+0000 7f06b1600700 1 -- 172.21.15.99:0/1798144447 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8012430 con 0x7f06b40d0d60 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.834+0000 7f06b1600700 1 -- 172.21.15.99:0/1798144447 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8020070 con 0x7f06b40d0d60 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.834+0000 7f06b1600700 1 -- 172.21.15.99:0/1798144447 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8004280 con 0x7f06b40d0d60 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.834+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a8009a70 con 0x7f06b40d0d60 2024-03-29T09:09:14.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.834+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b40b6a80 con 0x7f06b40d0d60 2024-03-29T09:09:14.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.835+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b40678b0 con 0x7f06b40d0d60 2024-03-29T09:09:14.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.836+0000 7f06b1600700 1 -- 172.21.15.99:0/1798144447 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f06a80043e0 con 0x7f06b40d0d60 2024-03-29T09:09:14.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.836+0000 7f06b1600700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f069406ac90 0x7f069406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:14.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.836+0000 7f06b1600700 1 -- 172.21.15.99:0/1798144447 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f06a8022070 con 0x7f06b40d0d60 2024-03-29T09:09:14.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.837+0000 7f06b9000700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f069406ac90 0x7f069406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:14.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.837+0000 7f06b9000700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f069406ac90 0x7f069406d150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06b40a1050 tx=0x7f06a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:14.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:14.841+0000 7f06b1600700 1 -- 172.21.15.99:0/1798144447 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f06a8056260 con 0x7f06b40d0d60 2024-03-29T09:09:15.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.043+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f06b40b4430 con 0x7f06b40d0d60 2024-03-29T09:09:15.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.044+0000 7f06b1600700 1 -- 172.21.15.99:0/1798144447 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f06a8059880 con 0x7f06b40d0d60 2024-03-29T09:09:15.044 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f069406ac90 msgr2=0x7f069406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f069406ac90 0x7f069406d150 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06b40a1050 tx=0x7f06a0009040 comp rx=0 tx=0).stop 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06b40d0d60 msgr2=0x7f06b40b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06b40d0d60 0x7f06b40b7280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06a8014040 tx=0x7f06a8004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 shutdown_connections 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f069406ac90 0x7f069406d150 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06b4160770 0x7f06b40ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f06b40d0d60 0x7f06b40b7280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 --2- 172.21.15.99:0/1798144447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06b40d3550 0x7f06b40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 >> 172.21.15.99:0/1798144447 conn(0x7f06b408afd0 msgr2=0x7f06b406c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 shutdown_connections 2024-03-29T09:09:15.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.046+0000 7f06bc400700 1 -- 172.21.15.99:0/1798144447 wait complete. 2024-03-29T09:09:15.064 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:09:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.253+0000 7fdbd5000700 1 Processor -- start 2024-03-29T09:09:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.254+0000 7fdbd5000700 1 -- start start 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.255+0000 7fdbd5000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdbd015d980 0x7fdbd00cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.255+0000 7fdbd5000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00d1a50 con 0x7fdbd00b1780 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbd5000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00d2260 con 0x7fdbd00cfd70 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbd5000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00d1ca0 con 0x7fdbd015d980 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbc7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdbd015d980 0x7fdbd00cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbce800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdbd00b1780 0x7fdbd00ce270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37544/0 (socket says 172.21.15.99:37544) 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbc7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdbd015d980 0x7fdbd00cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59596/0 (socket says 172.21.15.99:59596) 2024-03-29T09:09:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbce800700 1 -- 172.21.15.99:0/1244727750 learned_addr learned my addr 172.21.15.99:0/1244727750 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:15.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbc7e00700 1 -- 172.21.15.99:0/1244727750 >> v1:172.21.15.133:6789/0 conn(0x7fdbd00cfd70 legacy=0x7fdbd015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbc7e00700 1 -- 172.21.15.99:0/1244727750 >> v1:172.21.15.99:6789/0 conn(0x7fdbd00b1780 legacy=0x7fdbd00ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbc7e00700 1 -- 172.21.15.99:0/1244727750 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00d1f80 con 0x7fdbd015d980 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbcd400700 1 -- 172.21.15.99:0/1244727750 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183173515 0 0) 0x7fdbd00d1a50 con 0x7fdbd00b1780 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.256+0000 7fdbc7e00700 1 --2- 172.21.15.99:0/1244727750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00cdd30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009840 tx=0x7fdbb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe1b8d7b0d978e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbcd400700 1 -- 172.21.15.99:0/1244727750 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8011830 con 0x7fdbd015d980 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbcd400700 1 -- 172.21.15.99:0/1244727750 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb8011990 con 0x7fdbd015d980 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbcd400700 1 -- 172.21.15.99:0/1244727750 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8011d20 con 0x7fdbd015d980 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbd5000700 1 -- 172.21.15.99:0/1244727750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 msgr2=0x7fdbd00cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbd5000700 1 --2- 172.21.15.99:0/1244727750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00cdd30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009840 tx=0x7fdbb8009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbd5000700 1 -- 172.21.15.99:0/1244727750 shutdown_connections 2024-03-29T09:09:15.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbd5000700 1 --2- 172.21.15.99:0/1244727750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00cdd30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbd5000700 1 -- 172.21.15.99:0/1244727750 >> 172.21.15.99:0/1244727750 conn(0x7fdbd009c470 msgr2=0x7fdbd00d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbd5000700 1 -- 172.21.15.99:0/1244727750 shutdown_connections 2024-03-29T09:09:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.257+0000 7fdbd5000700 1 -- 172.21.15.99:0/1244727750 wait complete. 2024-03-29T09:09:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.260+0000 7fdbd5000700 1 Processor -- start 2024-03-29T09:09:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.260+0000 7fdbd5000700 1 -- start start 2024-03-29T09:09:15.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbd5000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdbd00b1780 0x7fdbd00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbd5000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbd00cfd70 0x7fdbd015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbd5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbd5000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00cb4f0 con 0x7fdbd015d980 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbd5000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00c6c40 con 0x7fdbd00cfd70 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbd5000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00c6e50 con 0x7fdbd00b1780 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbce800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbce800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59602/0 (socket says 172.21.15.99:59602) 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbce800700 1 -- 172.21.15.99:0/4086360911 learned_addr learned my addr 172.21.15.99:0/4086360911 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbc7e00700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbd00cfd70 0x7fdbd015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.261+0000 7fdbcde00700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdbd00b1780 0x7fdbd00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbce800700 1 -- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdbd00b1780 msgr2=0x7fdbd00caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbce800700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdbd00b1780 0x7fdbd00caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbce800700 1 -- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbd00cfd70 msgr2=0x7fdbd015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbce800700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbd00cfd70 0x7fdbd015a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbce800700 1 -- 172.21.15.99:0/4086360911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb80094a0 con 0x7fdbd015d980 2024-03-29T09:09:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbce800700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00b7d70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8002230 tx=0x7fdbc800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:15.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbc6000700 1 -- 172.21.15.99:0/4086360911 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800aac0 con 0x7fdbd015d980 2024-03-29T09:09:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbc6000700 1 -- 172.21.15.99:0/4086360911 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc800ac40 con 0x7fdbd015d980 2024-03-29T09:09:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd00c7130 con 0x7fdbd015d980 2024-03-29T09:09:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbc6000700 1 -- 172.21.15.99:0/4086360911 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc80070d0 con 0x7fdbd015d980 2024-03-29T09:09:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.262+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd00c76f0 con 0x7fdbd015d980 2024-03-29T09:09:15.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.263+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd00b4a30 con 0x7fdbd015d980 2024-03-29T09:09:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.264+0000 7fdbc6000700 1 -- 172.21.15.99:0/4086360911 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdbc8007250 con 0x7fdbd015d980 2024-03-29T09:09:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.265+0000 7fdbc6000700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdbac06ac90 0x7fdbac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.265+0000 7fdbc6000700 1 -- 172.21.15.99:0/4086360911 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdbc8089110 con 0x7fdbd015d980 2024-03-29T09:09:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.265+0000 7fdbcde00700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdbac06ac90 0x7fdbac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.266+0000 7fdbcde00700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdbac06ac90 0x7fdbac06d150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:15.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.269+0000 7fdbc6000700 1 -- 172.21.15.99:0/4086360911 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdbc8053f60 con 0x7fdbd015d980 2024-03-29T09:09:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.489+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdbd00c78e0 con 0x7fdbd015d980 2024-03-29T09:09:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.490+0000 7fdbc6000700 1 -- 172.21.15.99:0/4086360911 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdbc8057580 con 0x7fdbd015d980 2024-03-29T09:09:15.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdbac06ac90 msgr2=0x7fdbac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdbac06ac90 0x7fdbac06d150 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0009040 comp rx=0 tx=0).stop 2024-03-29T09:09:15.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 msgr2=0x7fdbd00b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:15.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00b7d70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8002230 tx=0x7fdbc800cf50 comp rx=0 tx=0).stop 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 shutdown_connections 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdbac06ac90 0x7fdbac06d150 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbd015d980 0x7fdbd00b7d70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdbd00b1780 0x7fdbd00caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 --2- 172.21.15.99:0/4086360911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbd00cfd70 0x7fdbd015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 >> 172.21.15.99:0/4086360911 conn(0x7fdbd009c470 msgr2=0x7fdbd00cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 shutdown_connections 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:15.492+0000 7fdbd5000700 1 -- 172.21.15.99:0/4086360911 wait complete. 2024-03-29T09:09:15.493 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:09:15.510 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:09:15.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:15.510 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-29T09:09:15.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi099.stderr:ceph-fuse[49336]: fuse finished with error 0 and tester_r 0 2024-03-29T09:09:15.576 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:09:21.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:09:21.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:21.678 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-29T09:09:21.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:21.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.933+0000 7f45f1600700 1 Processor -- start 2024-03-29T09:09:21.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.933+0000 7f45f1600700 1 -- start start 2024-03-29T09:09:21.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.934+0000 7f45f1600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f45ec1608e0 0x7f45ec0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:21.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.934+0000 7f45f1600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ec0cddc0 con 0x7f45ec160cc0 2024-03-29T09:09:21.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.934+0000 7f45f1600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ec0ce0a0 con 0x7f45ec0d1b20 2024-03-29T09:09:21.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.934+0000 7f45f1600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45ec0ce380 con 0x7f45ec1608e0 2024-03-29T09:09:21.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45ea000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f45ec160cc0 0x7f45ec0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50388/0 (socket says 172.21.15.99:50388) 2024-03-29T09:09:21.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45ea000700 1 -- 172.21.15.99:0/4249908069 learned_addr learned my addr 172.21.15.99:0/4249908069 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:21.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45eaa00700 1 --2- 172.21.15.99:0/4249908069 >> v2:172.21.15.133:3300/0 conn(0x7f45ec1608e0 0x7f45ec0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:21.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45e9600700 1 -- 172.21.15.99:0/4249908069 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612175266 0 0) 0x7f45ec0cddc0 con 0x7f45ec160cc0 2024-03-29T09:09:21.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45e9600700 1 -- 172.21.15.99:0/4249908069 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d00035b0 con 0x7f45ec160cc0 2024-03-29T09:09:21.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45eaa00700 1 -- 172.21.15.99:0/4249908069 >> v1:172.21.15.133:6789/0 conn(0x7f45ec0d1b20 legacy=0x7f45ec0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:21.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45eaa00700 1 -- 172.21.15.99:0/4249908069 >> v1:172.21.15.99:6789/0 conn(0x7f45ec160cc0 legacy=0x7f45ec0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:21.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45e9600700 1 -- 172.21.15.99:0/4249908069 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699840405 0 0) 0x7f45d00035b0 con 0x7f45ec160cc0 2024-03-29T09:09:21.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.935+0000 7f45eaa00700 1 -- 172.21.15.99:0/4249908069 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec0ce660 con 0x7f45ec1608e0 2024-03-29T09:09:21.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.936+0000 7f45eaa00700 1 --2- 172.21.15.99:0/4249908069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 0x7f45ec0d0150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009840 tx=0x7f45dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c6d8857a47ee7c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:21.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.936+0000 7f45e9600700 1 -- 172.21.15.99:0/4249908069 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8001670 con 0x7f45ec1608e0 2024-03-29T09:09:21.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.936+0000 7f45e9600700 1 -- 172.21.15.99:0/4249908069 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc004030 con 0x7f45ec1608e0 2024-03-29T09:09:21.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.936+0000 7f45e9600700 1 -- 172.21.15.99:0/4249908069 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0043a0 con 0x7f45ec1608e0 2024-03-29T09:09:21.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.937+0000 7f45f1600700 1 -- 172.21.15.99:0/4249908069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 msgr2=0x7f45ec0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:21.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.937+0000 7f45f1600700 1 --2- 172.21.15.99:0/4249908069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 0x7f45ec0d0150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009840 tx=0x7f45dc009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:21.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.937+0000 7f45f1600700 1 -- 172.21.15.99:0/4249908069 shutdown_connections 2024-03-29T09:09:21.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.937+0000 7f45f1600700 1 --2- 172.21.15.99:0/4249908069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 0x7f45ec0d0150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:21.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.937+0000 7f45f1600700 1 -- 172.21.15.99:0/4249908069 >> 172.21.15.99:0/4249908069 conn(0x7f45ec09c490 msgr2=0x7f45ec0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:21.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.937+0000 7f45f1600700 1 -- 172.21.15.99:0/4249908069 shutdown_connections 2024-03-29T09:09:21.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.937+0000 7f45f1600700 1 -- 172.21.15.99:0/4249908069 wait complete. 2024-03-29T09:09:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.938+0000 7f45f1600700 1 Processor -- start 2024-03-29T09:09:21.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.939+0000 7f45f1600700 1 -- start start 2024-03-29T09:09:21.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.939+0000 7f45f1600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45ec0d1b20 0x7f45ec0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.939+0000 7f45f1600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 0x7f45ec1f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45f1600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ec160cc0 0x7f45ec1f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45f1600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec0ccf40 con 0x7f45ec0d1b20 2024-03-29T09:09:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45f1600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec0cd220 con 0x7f45ec1608e0 2024-03-29T09:09:21.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45f1600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec157400 con 0x7f45ec160cc0 2024-03-29T09:09:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45ec0d1b20 0x7f45ec0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45ec0d1b20 0x7f45ec0ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48326/0 (socket says 172.21.15.99:48326) 2024-03-29T09:09:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 -- 172.21.15.99:0/3240983485 learned_addr learned my addr 172.21.15.99:0/3240983485 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 -- 172.21.15.99:0/3240983485 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ec160cc0 msgr2=0x7f45ec1f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ec160cc0 0x7f45ec1f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:21.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 -- 172.21.15.99:0/3240983485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 msgr2=0x7f45ec1f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 0x7f45ec1f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 -- 172.21.15.99:0/3240983485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc0094a0 con 0x7f45ec0d1b20 2024-03-29T09:09:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.940+0000 7f45eaa00700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45ec0d1b20 0x7f45ec0ca320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0090b0 tx=0x7f45dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.941+0000 7f45e2a00700 1 -- 172.21.15.99:0/3240983485 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc011630 con 0x7f45ec0d1b20 2024-03-29T09:09:21.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.941+0000 7f45e2a00700 1 -- 172.21.15.99:0/3240983485 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc01f070 con 0x7f45ec0d1b20 2024-03-29T09:09:21.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.941+0000 7f45e2a00700 1 -- 172.21.15.99:0/3240983485 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0119d0 con 0x7f45ec0d1b20 2024-03-29T09:09:21.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.941+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ec1576e0 con 0x7f45ec0d1b20 2024-03-29T09:09:21.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.941+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ec146230 con 0x7f45ec0d1b20 2024-03-29T09:09:21.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.942+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ec0b1ab0 con 0x7f45ec0d1b20 2024-03-29T09:09:21.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.942+0000 7f45e2a00700 1 -- 172.21.15.99:0/3240983485 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f45dc011b30 con 0x7f45ec0d1b20 2024-03-29T09:09:21.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.943+0000 7f45e2a00700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f45cc06ac90 0x7f45cc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.943+0000 7f45e2a00700 1 -- 172.21.15.99:0/3240983485 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f45dc021070 con 0x7f45ec0d1b20 2024-03-29T09:09:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.944+0000 7f45ea000700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f45cc06ac90 0x7f45cc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.944+0000 7f45ea000700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f45cc06ac90 0x7f45cc06d150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000c00 tx=0x7f45d80010e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:21.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:21.949+0000 7f45e2a00700 1 -- 172.21.15.99:0/3240983485 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f45dc055ef0 con 0x7f45ec0d1b20 2024-03-29T09:09:22.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.155+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45ec0677a0 con 0x7f45ec0d1b20 2024-03-29T09:09:22.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.156+0000 7f45e2a00700 1 -- 172.21.15.99:0/3240983485 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f45dc02b020 con 0x7f45ec0d1b20 2024-03-29T09:09:22.159 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:22.159 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:22.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f45cc06ac90 msgr2=0x7f45cc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f45cc06ac90 0x7f45cc06d150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f45d8000c00 tx=0x7f45d80010e0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45ec0d1b20 msgr2=0x7f45ec0ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45ec0d1b20 0x7f45ec0ca320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0090b0 tx=0x7f45dc004750 comp rx=0 tx=0).stop 2024-03-29T09:09:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 shutdown_connections 2024-03-29T09:09:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f45cc06ac90 0x7f45cc06d150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ec160cc0 0x7f45ec1f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45ec0d1b20 0x7f45ec0ca320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 --2- 172.21.15.99:0/3240983485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ec1608e0 0x7f45ec1f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.158+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 >> 172.21.15.99:0/3240983485 conn(0x7f45ec09c490 msgr2=0x7f45ec0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.159+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 shutdown_connections 2024-03-29T09:09:22.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.159+0000 7f45f1600700 1 -- 172.21.15.99:0/3240983485 wait complete. 2024-03-29T09:09:22.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:22.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.366+0000 7f436cc00700 1 Processor -- start 2024-03-29T09:09:22.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.366+0000 7f436cc00700 1 -- start start 2024-03-29T09:09:22.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.367+0000 7f436cc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43680d1b20 0x7f43680d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.367+0000 7f436cc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.367+0000 7f436cc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43680cddc0 con 0x7f43681608e0 2024-03-29T09:09:22.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.367+0000 7f436cc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43680ce0a0 con 0x7f43680d1b20 2024-03-29T09:09:22.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.367+0000 7f436cc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43680ce380 con 0x7f4368160cc0 2024-03-29T09:09:22.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43680d1b20 0x7f43680d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43680d1b20 0x7f43680d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48332/0 (socket says 172.21.15.99:48332) 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 -- 172.21.15.99:0/1960965938 learned_addr learned my addr 172.21.15.99:0/1960965938 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365e00700 1 --1- 172.21.15.99:0/1960965938 >> v1:172.21.15.99:6789/0 conn(0x7f43681608e0 0x7f43680d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50398/0 (socket says 172.21.15.99:50398) 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4366800700 1 --2- 172.21.15.99:0/1960965938 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 -- 172.21.15.99:0/1960965938 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 msgr2=0x7f43680ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 --2- 172.21.15.99:0/1960965938 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 -- 172.21.15.99:0/1960965938 >> v1:172.21.15.99:6789/0 conn(0x7f43681608e0 legacy=0x7f43680d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 -- 172.21.15.99:0/1960965938 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680ce660 con 0x7f43680d1b20 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4365400700 1 --2- 172.21.15.99:0/1960965938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f43680d0150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43540098f0 tx=0x7f4354009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc0ddc01f3c6d07 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.368+0000 7f4364a00700 1 -- 172.21.15.99:0/1960965938 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400b8a0 con 0x7f43680d1b20 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f4364a00700 1 -- 172.21.15.99:0/1960965938 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435400ba00 con 0x7f43680d1b20 2024-03-29T09:09:22.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f4364a00700 1 -- 172.21.15.99:0/1960965938 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435400bd90 con 0x7f43680d1b20 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 -- 172.21.15.99:0/1960965938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 msgr2=0x7f43680d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 --2- 172.21.15.99:0/1960965938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f43680d0150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43540098f0 tx=0x7f4354009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 -- 172.21.15.99:0/1960965938 shutdown_connections 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 --2- 172.21.15.99:0/1960965938 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 --2- 172.21.15.99:0/1960965938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f43680d0150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 -- 172.21.15.99:0/1960965938 >> 172.21.15.99:0/1960965938 conn(0x7f436809c490 msgr2=0x7f43680d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 -- 172.21.15.99:0/1960965938 shutdown_connections 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.369+0000 7f436cc00700 1 -- 172.21.15.99:0/1960965938 wait complete. 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.371+0000 7f436cc00700 1 Processor -- start 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.371+0000 7f436cc00700 1 -- start start 2024-03-29T09:09:22.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f436cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f436814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f436cc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43681608e0 0x7f436814e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f436cc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f436cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436814d510 con 0x7f43680d1b20 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f436cc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436814d720 con 0x7f43681608e0 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f436cc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436814da00 con 0x7f4368160cc0 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f4365e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f436814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f4365e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f436814cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48344/0 (socket says 172.21.15.99:48344) 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f4365e00700 1 -- 172.21.15.99:0/3951203118 learned_addr learned my addr 172.21.15.99:0/3951203118 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f4366800700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f4365400700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43681608e0 0x7f436814e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f4365e00700 1 -- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 msgr2=0x7f43680b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.372+0000 7f4365e00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f4365e00700 1 -- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43681608e0 msgr2=0x7f436814e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f4365e00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43681608e0 0x7f436814e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f4365e00700 1 -- 172.21.15.99:0/3951203118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4354009550 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f4365400700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43681608e0 0x7f436814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f4365e00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f436814cfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f435800c520 tx=0x7f435800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f435e000700 1 -- 172.21.15.99:0/3951203118 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358005cb0 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f435e000700 1 -- 172.21.15.99:0/3951203118 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358005e10 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436814dce0 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f435e000700 1 -- 172.21.15.99:0/3951203118 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580072d0 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.373+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436814e2a0 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.374+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43680677a0 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.375+0000 7f435e000700 1 -- 172.21.15.99:0/3951203118 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4358007430 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.376+0000 7f435e000700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f434806ac90 0x7f434806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.376+0000 7f435e000700 1 -- 172.21.15.99:0/3951203118 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4358088aa0 con 0x7f43680d1b20 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.376+0000 7f4365400700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f434806ac90 0x7f434806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.377+0000 7f4365400700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f434806ac90 0x7f434806d150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f436814cea0 tx=0x7f4354008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:22.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.380+0000 7f435e000700 1 -- 172.21.15.99:0/3951203118 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f43580539a0 con 0x7f43680d1b20 2024-03-29T09:09:22.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:09:22.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:09:22.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.591+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43680d2c40 con 0x7f43680d1b20 2024-03-29T09:09:22.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.593+0000 7f435e000700 1 -- 172.21.15.99:0/3951203118 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f4358017020 con 0x7f43680d1b20 2024-03-29T09:09:22.593 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:22.593 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:22.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f434806ac90 msgr2=0x7f434806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f434806ac90 0x7f434806d150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f436814cea0 tx=0x7f4354008030 comp rx=0 tx=0).stop 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 msgr2=0x7f436814cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f436814cfd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f435800c520 tx=0x7f435800c8e0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 shutdown_connections 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f434806ac90 0x7f434806d150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4368160cc0 0x7f43680b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43680d1b20 0x7f436814cfd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 --2- 172.21.15.99:0/3951203118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43681608e0 0x7f436814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.595+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 >> 172.21.15.99:0/3951203118 conn(0x7f436809c490 msgr2=0x7f43680cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.596+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 shutdown_connections 2024-03-29T09:09:22.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.596+0000 7f436cc00700 1 -- 172.21.15.99:0/3951203118 wait complete. 2024-03-29T09:09:22.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T09:09:22.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.808+0000 7fea1e600700 1 Processor -- start 2024-03-29T09:09:22.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.808+0000 7fea1e600700 1 -- start start 2024-03-29T09:09:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.809+0000 7fea1e600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fea18160770 0x7fea180b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.809+0000 7fea1e600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 0x7fea180d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea1e600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea180d3810 con 0x7fea180d3380 2024-03-29T09:09:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea1e600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea180d1520 con 0x7fea18160770 2024-03-29T09:09:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea1e600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea180d1800 con 0x7fea180d0d60 2024-03-29T09:09:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea17e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fea180d3380 0x7fea180cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50406/0 (socket says 172.21.15.99:50406) 2024-03-29T09:09:22.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea17e00700 1 -- 172.21.15.99:0/436152545 learned_addr learned my addr 172.21.15.99:0/436152545 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:22.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea16a00700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 0x7fea180d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea17400700 1 --2- 172.21.15.99:0/436152545 >> v2:172.21.15.133:3300/0 conn(0x7fea18160770 0x7fea180b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847932894 0 0) 0x7fea180d3810 con 0x7fea180d3380 2024-03-29T09:09:22.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.810+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9fc0035b0 con 0x7fea180d3380 2024-03-29T09:09:22.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665067510 0 0) 0x7fe9fc0035b0 con 0x7fea180d3380 2024-03-29T09:09:22.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 msgr2=0x7fea180d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:22.814 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 0x7fea180d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 >> v2:172.21.15.133:3300/0 conn(0x7fea18160770 msgr2=0x7fea180b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 --2- 172.21.15.99:0/436152545 >> v2:172.21.15.133:3300/0 conn(0x7fea18160770 0x7fea180b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea180d1ae0 con 0x7fea180d3380 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fea100019f0 con 0x7fea180d3380 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 >> v1:172.21.15.99:6789/0 conn(0x7fea180d3380 legacy=0x7fea180cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9fc003a00 0x7fe9fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 0x7fe9fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc004d90 0x7fe9fc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea180d1520 con 0x7fe9fc003a00 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea180d1800 con 0x7fe9fc004d90 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc00ba10 con 0x7fea180d0d60 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea17400700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9fc003a00 0x7fe9fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16a00700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 0x7fe9fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea17e00700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc004d90 0x7fe9fc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea17400700 1 -- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 msgr2=0x7fe9fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea17400700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 0x7fe9fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea17400700 1 -- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc004d90 msgr2=0x7fe9fc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea17400700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc004d90 0x7fe9fc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea17400700 1 -- 172.21.15.99:0/436152545 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0035b0 con 0x7fe9fc003a00 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.811+0000 7fea16a00700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180d0d60 0x7fe9fc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea17400700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9fc003a00 0x7fe9fc003e20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fea08009b60 tx=0x7fe9fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcefe05ddd3447c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800e6c0 con 0x7fe9fc003a00 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0800e820 con 0x7fe9fc003a00 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea16000700 1 -- 172.21.15.99:0/436152545 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0800eb90 con 0x7fe9fc003a00 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea1e600700 1 -- 172.21.15.99:0/436152545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9fc003a00 msgr2=0x7fe9fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea1e600700 1 --2- 172.21.15.99:0/436152545 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9fc003a00 0x7fe9fc003e20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fea08009b60 tx=0x7fe9fc005880 comp rx=0 tx=0).stop 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea17e00700 1 -- 172.21.15.99:0/436152545 reap_dead start 2024-03-29T09:09:22.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea1e600700 1 -- 172.21.15.99:0/436152545 shutdown_connections 2024-03-29T09:09:22.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea1e600700 1 -- 172.21.15.99:0/436152545 >> 172.21.15.99:0/436152545 conn(0x7fea1808afd0 msgr2=0x7fea1809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:22.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea1e600700 1 -- 172.21.15.99:0/436152545 shutdown_connections 2024-03-29T09:09:22.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.812+0000 7fea1e600700 1 -- 172.21.15.99:0/436152545 wait complete. 2024-03-29T09:09:22.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.814+0000 7fea1e600700 1 Processor -- start 2024-03-29T09:09:22.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.814+0000 7fea1e600700 1 -- start start 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.815+0000 7fea1e600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc003a00 0x7fea181f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.815+0000 7fea1e600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea1814f730 0x7fea181f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.815+0000 7fea1e600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180cded0 0x7fea180ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.815+0000 7fea1e600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea180ce8c0 con 0x7fea1814f730 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.815+0000 7fea1e600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1815c5c0 con 0x7fe9fc003a00 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.815+0000 7fea1e600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1815cdd0 con 0x7fea180cded0 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea1814f730 0x7fea181f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea1814f730 0x7fea181f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48352/0 (socket says 172.21.15.99:48352) 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 -- 172.21.15.99:0/1674388891 learned_addr learned my addr 172.21.15.99:0/1674388891 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea17400700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc003a00 0x7fea181f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea17e00700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180cded0 0x7fea180ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 -- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180cded0 msgr2=0x7fea180ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180cded0 0x7fea180ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 -- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc003a00 msgr2=0x7fea181f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc003a00 0x7fea181f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 -- 172.21.15.99:0/1674388891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea181f6f90 con 0x7fea1814f730 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea16a00700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea1814f730 0x7fea181f2230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea04010960 tx=0x7fea04010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea0f400700 1 -- 172.21.15.99:0/1674388891 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04004020 con 0x7fea1814f730 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea0f400700 1 -- 172.21.15.99:0/1674388891 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04004180 con 0x7fea1814f730 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.816+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1815c810 con 0x7fea1814f730 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.817+0000 7fea0f400700 1 -- 172.21.15.99:0/1674388891 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04004530 con 0x7fea1814f730 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.817+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1814d140 con 0x7fea1814f730 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.818+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea180678b0 con 0x7fea1814f730 2024-03-29T09:09:22.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.818+0000 7fea0f400700 1 -- 172.21.15.99:0/1674388891 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fea0400e870 con 0x7fea1814f730 2024-03-29T09:09:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.819+0000 7fea0f400700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe9f806ac90 0x7fe9f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:22.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.819+0000 7fea0f400700 1 -- 172.21.15.99:0/1674388891 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fea04089980 con 0x7fea1814f730 2024-03-29T09:09:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.820+0000 7fea17400700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe9f806ac90 0x7fe9f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:22.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.820+0000 7fea17400700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe9f806ac90 0x7fe9f806d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea080090b0 tx=0x7fea08008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:22.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:22.824+0000 7fea0f400700 1 -- 172.21.15.99:0/1674388891 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fea04054860 con 0x7fea1814f730 2024-03-29T09:09:23.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.075+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fea180b20b0 con 0x7fea1814f730 2024-03-29T09:09:23.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.917+0000 7fea0f400700 1 -- 172.21.15.99:0/1674388891 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fea04057e80 con 0x7fea1814f730 2024-03-29T09:09:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe9f806ac90 msgr2=0x7fe9f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe9f806ac90 0x7fe9f806d150 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fea080090b0 tx=0x7fea08008030 comp rx=0 tx=0).stop 2024-03-29T09:09:23.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea1814f730 msgr2=0x7fea181f2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea1814f730 0x7fea181f2230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea04010960 tx=0x7fea04010d20 comp rx=0 tx=0).stop 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 shutdown_connections 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe9f806ac90 0x7fe9f806d150 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fea180cded0 0x7fea180ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9fc003a00 0x7fea181f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 --2- 172.21.15.99:0/1674388891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea1814f730 0x7fea181f2230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 >> 172.21.15.99:0/1674388891 conn(0x7fea1808afd0 msgr2=0x7fea180b25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 shutdown_connections 2024-03-29T09:09:23.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:23.920+0000 7fea1e600700 1 -- 172.21.15.99:0/1674388891 wait complete. 2024-03-29T09:09:23.943 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-29T09:09:23.943 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T09:09:24.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.141+0000 7f6291a00700 1 Processor -- start 2024-03-29T09:09:24.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.142+0000 7f6291a00700 1 -- start start 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f6291a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f628c0b3cc0 0x7f628c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f6291a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f628c0d3380 0x7f628c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f6291a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0d3810 con 0x7f628c160770 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f6291a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0b4140 con 0x7f628c0d3380 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f6291a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0d1520 con 0x7f628c0b3cc0 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f628b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f628c0d3380 0x7f628c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f628b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f628c0d3380 0x7f628c0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48362/0 (socket says 172.21.15.99:48362) 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f628b400700 1 -- 172.21.15.99:0/1534313090 learned_addr learned my addr 172.21.15.99:0/1534313090 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.143+0000 7f628a000700 1 --2- 172.21.15.99:0/1534313090 >> v2:172.21.15.133:3300/0 conn(0x7f628c0b3cc0 0x7f628c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f628b400700 1 -- 172.21.15.99:0/1534313090 >> v2:172.21.15.133:3300/0 conn(0x7f628c0b3cc0 msgr2=0x7f628c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:24.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f628b400700 1 --2- 172.21.15.99:0/1534313090 >> v2:172.21.15.133:3300/0 conn(0x7f628c0b3cc0 0x7f628c0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f628b400700 1 -- 172.21.15.99:0/1534313090 >> v1:172.21.15.133:6789/0 conn(0x7f628c160770 legacy=0x7f628c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f628b400700 1 -- 172.21.15.99:0/1534313090 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0d1800 con 0x7f628c0d3380 2024-03-29T09:09:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f628a000700 1 --2- 172.21.15.99:0/1534313090 >> v2:172.21.15.133:3300/0 conn(0x7f628c0b3cc0 0x7f628c0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f628b400700 1 --2- 172.21.15.99:0/1534313090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0d3380 0x7f628c0ce080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b5cf0 tx=0x7f628400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384e50a759e9c3f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f6289600700 1 -- 172.21.15.99:0/1534313090 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400e070 con 0x7f628c0d3380 2024-03-29T09:09:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f6289600700 1 -- 172.21.15.99:0/1534313090 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628400b9c0 con 0x7f628c0d3380 2024-03-29T09:09:24.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.144+0000 7f6289600700 1 -- 172.21.15.99:0/1534313090 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628400bd70 con 0x7f628c0d3380 2024-03-29T09:09:24.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 -- 172.21.15.99:0/1534313090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0d3380 msgr2=0x7f628c0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:24.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 --2- 172.21.15.99:0/1534313090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0d3380 0x7f628c0ce080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b5cf0 tx=0x7f628400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:24.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 -- 172.21.15.99:0/1534313090 shutdown_connections 2024-03-29T09:09:24.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 --2- 172.21.15.99:0/1534313090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0d3380 0x7f628c0ce080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 --2- 172.21.15.99:0/1534313090 >> v2:172.21.15.133:3300/0 conn(0x7f628c0b3cc0 0x7f628c0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 -- 172.21.15.99:0/1534313090 >> 172.21.15.99:0/1534313090 conn(0x7f628c08afd0 msgr2=0x7f628c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:24.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 -- 172.21.15.99:0/1534313090 shutdown_connections 2024-03-29T09:09:24.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.145+0000 7f6291a00700 1 -- 172.21.15.99:0/1534313090 wait complete. 2024-03-29T09:09:24.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.146+0000 7f6291a00700 1 Processor -- start 2024-03-29T09:09:24.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.147+0000 7f6291a00700 1 -- start start 2024-03-29T09:09:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.147+0000 7f6291a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0b3cc0 0x7f628c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f6291a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c0d3380 0x7f628c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f6291a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c160770 0x7f628c0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f6291a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b78e0 con 0x7f628c0b3cc0 2024-03-29T09:09:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f6291a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c15ab50 con 0x7f628c0d3380 2024-03-29T09:09:24.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f6291a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b6c60 con 0x7f628c160770 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0b3cc0 0x7f628c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0b3cc0 0x7f628c0cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48364/0 (socket says 172.21.15.99:48364) 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 -- 172.21.15.99:0/2446053688 learned_addr learned my addr 172.21.15.99:0/2446053688 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628a000700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c0d3380 0x7f628c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628b400700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c160770 0x7f628c0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 -- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c160770 msgr2=0x7f628c0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c160770 0x7f628c0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 -- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c0d3380 msgr2=0x7f628c15a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c0d3380 0x7f628c15a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 -- 172.21.15.99:0/2446053688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6284009b30 con 0x7f628c0b3cc0 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628a000700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c0d3380 0x7f628c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.148+0000 7f628aa00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0b3cc0 0x7f628c0cd020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c450 tx=0x7f627c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:24.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.149+0000 7f6282a00700 1 -- 172.21.15.99:0/2446053688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c0052d0 con 0x7f628c0b3cc0 2024-03-29T09:09:24.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.149+0000 7f6282a00700 1 -- 172.21.15.99:0/2446053688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c005430 con 0x7f628c0b3cc0 2024-03-29T09:09:24.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.149+0000 7f6282a00700 1 -- 172.21.15.99:0/2446053688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c0038c0 con 0x7f628c0b3cc0 2024-03-29T09:09:24.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.149+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0b6f40 con 0x7f628c0b3cc0 2024-03-29T09:09:24.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.149+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0b7500 con 0x7f628c0b3cc0 2024-03-29T09:09:24.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.150+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c0678b0 con 0x7f628c0b3cc0 2024-03-29T09:09:24.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.151+0000 7f6282a00700 1 -- 172.21.15.99:0/2446053688 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f627c003a20 con 0x7f628c0b3cc0 2024-03-29T09:09:24.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.152+0000 7f6282a00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f626c06ac90 0x7f626c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:24.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.152+0000 7f6282a00700 1 -- 172.21.15.99:0/2446053688 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f627c088a90 con 0x7f628c0b3cc0 2024-03-29T09:09:24.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.152+0000 7f628a000700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f626c06ac90 0x7f626c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:24.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.152+0000 7f628a000700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f626c06ac90 0x7f626c06d150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b4630 tx=0x7f6278008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:24.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.156+0000 7f6282a00700 1 -- 172.21.15.99:0/2446053688 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f627c0538e0 con 0x7f628c0b3cc0 2024-03-29T09:09:24.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.403+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f628c0b76c0 con 0x7f628c0b3cc0 2024-03-29T09:09:24.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.925+0000 7f6282a00700 1 -- 172.21.15.99:0/2446053688 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f627c0170c0 con 0x7f628c0b3cc0 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.927+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f626c06ac90 msgr2=0x7f626c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.927+0000 7f6291a00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f626c06ac90 0x7f626c06d150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b4630 tx=0x7f6278008040 comp rx=0 tx=0).stop 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.927+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0b3cc0 msgr2=0x7f628c0cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.927+0000 7f6291a00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0b3cc0 0x7f628c0cd020 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f627c00c450 tx=0x7f627c00c760 comp rx=0 tx=0).stop 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 shutdown_connections 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f626c06ac90 0x7f626c06d150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c160770 0x7f628c0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f628c0b3cc0 0x7f628c0cd020 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 --2- 172.21.15.99:0/2446053688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c0d3380 0x7f628c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:24.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 >> 172.21.15.99:0/2446053688 conn(0x7f628c08afd0 msgr2=0x7f628c15f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:24.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 shutdown_connections 2024-03-29T09:09:24.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:24.928+0000 7f6291a00700 1 -- 172.21.15.99:0/2446053688 wait complete. 2024-03-29T09:09:24.947 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T09:09:24.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:09:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.138+0000 7f8c6f400700 1 Processor -- start 2024-03-29T09:09:25.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.138+0000 7f8c6f400700 1 -- start start 2024-03-29T09:09:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.140+0000 7f8c6f400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c681608e0 0x7f8c680d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.140+0000 7f8c6f400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c68160cc0 0x7f8c680ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.140+0000 7f8c6f400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c680cddc0 con 0x7f8c680d3a70 2024-03-29T09:09:25.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.140+0000 7f8c6f400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c680ce0a0 con 0x7f8c681608e0 2024-03-29T09:09:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.140+0000 7f8c6f400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680ce380 con 0x7f8c68160cc0 2024-03-29T09:09:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.140+0000 7f8c67e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c68160cc0 0x7f8c680ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.140+0000 7f8c6ca00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c681608e0 0x7f8c680d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c6d400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8c680d3a70 0x7f8c680d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:51498/0 (socket says 172.21.15.99:51498) 2024-03-29T09:09:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c6d400700 1 -- 172.21.15.99:0/1231115364 learned_addr learned my addr 172.21.15.99:0/1231115364 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:25.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c68160cc0 0x7f8c680ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:45470/0 (socket says 172.21.15.99:45470) 2024-03-29T09:09:25.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67400700 1 -- 172.21.15.99:0/1231115364 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648118360 0 0) 0x7f8c680cddc0 con 0x7f8c680d3a70 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67400700 1 -- 172.21.15.99:0/1231115364 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c500035b0 con 0x7f8c680d3a70 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67e00700 1 -- 172.21.15.99:0/1231115364 >> v2:172.21.15.133:3300/0 conn(0x7f8c681608e0 msgr2=0x7f8c680d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67e00700 1 --2- 172.21.15.99:0/1231115364 >> v2:172.21.15.133:3300/0 conn(0x7f8c681608e0 0x7f8c680d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67e00700 1 -- 172.21.15.99:0/1231115364 >> v1:172.21.15.133:6789/0 conn(0x7f8c680d3a70 legacy=0x7f8c680d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67e00700 1 -- 172.21.15.99:0/1231115364 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680ce660 con 0x7f8c68160cc0 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.141+0000 7f8c67e00700 1 --2- 172.21.15.99:0/1231115364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c68160cc0 0x7f8c680ceb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009840 tx=0x7f8c54009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=399e9c5e51de4d38 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c67400700 1 -- 172.21.15.99:0/1231115364 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54011830 con 0x7f8c68160cc0 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c67400700 1 -- 172.21.15.99:0/1231115364 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c54011990 con 0x7f8c68160cc0 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c67400700 1 -- 172.21.15.99:0/1231115364 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54011d20 con 0x7f8c68160cc0 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 -- 172.21.15.99:0/1231115364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c68160cc0 msgr2=0x7f8c680ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 --2- 172.21.15.99:0/1231115364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c68160cc0 0x7f8c680ceb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009840 tx=0x7f8c54009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 -- 172.21.15.99:0/1231115364 shutdown_connections 2024-03-29T09:09:25.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 --2- 172.21.15.99:0/1231115364 >> v2:172.21.15.133:3300/0 conn(0x7f8c681608e0 0x7f8c680d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 --2- 172.21.15.99:0/1231115364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c68160cc0 0x7f8c680ceb80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 -- 172.21.15.99:0/1231115364 >> 172.21.15.99:0/1231115364 conn(0x7f8c6809c490 msgr2=0x7f8c680d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 -- 172.21.15.99:0/1231115364 shutdown_connections 2024-03-29T09:09:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.142+0000 7f8c6f400700 1 -- 172.21.15.99:0/1231115364 wait complete. 2024-03-29T09:09:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.144+0000 7f8c6f400700 1 Processor -- start 2024-03-29T09:09:25.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.144+0000 7f8c6f400700 1 -- start start 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c680d3a70 0x7f8c680cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6f400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c681608e0 0x7f8c680b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c68160cc0 0x7f8c681e7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6f400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680b2960 con 0x7f8c68160cc0 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6f400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680b2b90 con 0x7f8c680d3a70 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6f400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680b2e70 con 0x7f8c681608e0 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c68160cc0 0x7f8c681e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c68160cc0 0x7f8c681e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48376/0 (socket says 172.21.15.99:48376) 2024-03-29T09:09:25.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.145+0000 7f8c6d400700 1 -- 172.21.15.99:0/3738433332 learned_addr learned my addr 172.21.15.99:0/3738433332 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6ca00700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c680d3a70 0x7f8c680cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c67e00700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c681608e0 0x7f8c680b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6d400700 1 -- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c681608e0 msgr2=0x7f8c680b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6d400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c681608e0 0x7f8c680b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6d400700 1 -- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c680d3a70 msgr2=0x7f8c680cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6d400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c680d3a70 0x7f8c680cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6d400700 1 -- 172.21.15.99:0/3738433332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540094a0 con 0x7f8c68160cc0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6ca00700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c680d3a70 0x7f8c680cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6d400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c68160cc0 0x7f8c681e7090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000cb40 tx=0x7f8c6000ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c64c00700 1 -- 172.21.15.99:0/3738433332 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000aaa0 con 0x7f8c68160cc0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c64c00700 1 -- 172.21.15.99:0/3738433332 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6000ac00 con 0x7f8c68160cc0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c64c00700 1 -- 172.21.15.99:0/3738433332 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60012400 con 0x7f8c68160cc0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.146+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c680b3150 con 0x7f8c68160cc0 2024-03-29T09:09:25.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.147+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c68154880 con 0x7f8c68160cc0 2024-03-29T09:09:25.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.147+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c68154e10 con 0x7f8c68160cc0 2024-03-29T09:09:25.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.148+0000 7f8c64c00700 1 -- 172.21.15.99:0/3738433332 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8c6001b3b0 con 0x7f8c68160cc0 2024-03-29T09:09:25.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.149+0000 7f8c64c00700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8c4806ac90 0x7f8c4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.149+0000 7f8c64c00700 1 -- 172.21.15.99:0/3738433332 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8c60089380 con 0x7f8c68160cc0 2024-03-29T09:09:25.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.150+0000 7f8c6ca00700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8c4806ac90 0x7f8c4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.150+0000 7f8c6ca00700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8c4806ac90 0x7f8c4806d150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009c30 tx=0x7f8c5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:25.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.153+0000 7f8c64c00700 1 -- 172.21.15.99:0/3738433332 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8c68154e10 con 0x7f8c68160cc0 2024-03-29T09:09:25.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.400+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c68154e10 con 0x7f8c68160cc0 2024-03-29T09:09:25.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.400+0000 7f8c64c00700 1 -- 172.21.15.99:0/3738433332 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f8c68154e10 con 0x7f8c68160cc0 2024-03-29T09:09:25.401 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:25.401 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:09:25.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.402+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8c4806ac90 msgr2=0x7f8c4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.402+0000 7f8c6f400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8c4806ac90 0x7f8c4806d150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009c30 tx=0x7f8c5c008040 comp rx=0 tx=0).stop 2024-03-29T09:09:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.402+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c68160cc0 msgr2=0x7f8c681e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.402+0000 7f8c6f400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c68160cc0 0x7f8c681e7090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000cb40 tx=0x7f8c6000ce50 comp rx=0 tx=0).stop 2024-03-29T09:09:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 shutdown_connections 2024-03-29T09:09:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8c4806ac90 0x7f8c4806d150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c68160cc0 0x7f8c681e7090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c680d3a70 0x7f8c680cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 --2- 172.21.15.99:0/3738433332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c681608e0 0x7f8c680b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 >> 172.21.15.99:0/3738433332 conn(0x7f8c6809c490 msgr2=0x7f8c6815d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 shutdown_connections 2024-03-29T09:09:25.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.403+0000 7f8c6f400700 1 -- 172.21.15.99:0/3738433332 wait complete. 2024-03-29T09:09:25.406 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:09:25.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.613+0000 7f0418a00700 1 Processor -- start 2024-03-29T09:09:25.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.614+0000 7f0418a00700 1 -- start start 2024-03-29T09:09:25.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0418a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f04141608e0 0x7f04140d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0418a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0414160cc0 0x7f04140ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0418a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140d3a70 0x7f04140d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0418a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140cddc0 con 0x7f0414160cc0 2024-03-29T09:09:25.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0418a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140ce0a0 con 0x7f04141608e0 2024-03-29T09:09:25.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0418a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140ce380 con 0x7f04140d3a70 2024-03-29T09:09:25.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0411400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0414160cc0 0x7f04140ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0411400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0414160cc0 0x7f04140ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48378/0 (socket says 172.21.15.99:48378) 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0411400700 1 -- 172.21.15.99:0/946300028 learned_addr learned my addr 172.21.15.99:0/946300028 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0411e00700 1 --2- 172.21.15.99:0/946300028 >> v2:172.21.15.133:3300/0 conn(0x7f04141608e0 0x7f04140d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.615+0000 7f0412800700 1 --2- 172.21.15.99:0/946300028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140d3a70 0x7f04140d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0411400700 1 -- 172.21.15.99:0/946300028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140d3a70 msgr2=0x7f04140d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0411400700 1 --2- 172.21.15.99:0/946300028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140d3a70 0x7f04140d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0411400700 1 -- 172.21.15.99:0/946300028 >> v2:172.21.15.133:3300/0 conn(0x7f04141608e0 msgr2=0x7f04140d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0411400700 1 --2- 172.21.15.99:0/946300028 >> v2:172.21.15.133:3300/0 conn(0x7f04141608e0 0x7f04140d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0411400700 1 -- 172.21.15.99:0/946300028 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140ce660 con 0x7f0414160cc0 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0411e00700 1 --2- 172.21.15.99:0/946300028 >> v2:172.21.15.133:3300/0 conn(0x7f04141608e0 0x7f04140d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0411400700 1 --2- 172.21.15.99:0/946300028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04140ceb80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f041408d790 tx=0x7f0400009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68b76da2224b173 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:25.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0410a00700 1 -- 172.21.15.99:0/946300028 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040000b870 con 0x7f0414160cc0 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0410a00700 1 -- 172.21.15.99:0/946300028 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040000b9d0 con 0x7f0414160cc0 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.616+0000 7f0410a00700 1 -- 172.21.15.99:0/946300028 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040000bd60 con 0x7f0414160cc0 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 -- 172.21.15.99:0/946300028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 msgr2=0x7f04140ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 --2- 172.21.15.99:0/946300028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04140ceb80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f041408d790 tx=0x7f0400009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 -- 172.21.15.99:0/946300028 shutdown_connections 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 --2- 172.21.15.99:0/946300028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140d3a70 0x7f04140d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 --2- 172.21.15.99:0/946300028 >> v2:172.21.15.133:3300/0 conn(0x7f04141608e0 0x7f04140d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 --2- 172.21.15.99:0/946300028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04140ceb80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 -- 172.21.15.99:0/946300028 >> 172.21.15.99:0/946300028 conn(0x7f041409c490 msgr2=0x7f04140d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 -- 172.21.15.99:0/946300028 shutdown_connections 2024-03-29T09:09:25.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.617+0000 7f0418a00700 1 -- 172.21.15.99:0/946300028 wait complete. 2024-03-29T09:09:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.619+0000 7f0418a00700 1 Processor -- start 2024-03-29T09:09:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.619+0000 7f0418a00700 1 -- start start 2024-03-29T09:09:25.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0418a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04140d3a70 0x7f041415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0418a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04141608e0 0x7f04141f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0418a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04141536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0418a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04141f6c70 con 0x7f0414160cc0 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0418a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0414153bf0 con 0x7f04140d3a70 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0418a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04141f30f0 con 0x7f04141608e0 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04141536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04141536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48384/0 (socket says 172.21.15.99:48384) 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 -- 172.21.15.99:0/2231017168 learned_addr learned my addr 172.21.15.99:0/2231017168 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0411e00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04140d3a70 0x7f041415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0411400700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04141608e0 0x7f04141f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 -- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04141608e0 msgr2=0x7f04141f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04141608e0 0x7f04141f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 -- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04140d3a70 msgr2=0x7f041415a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04140d3a70 0x7f041415a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0412800700 1 -- 172.21.15.99:0/2231017168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0400009550 con 0x7f0414160cc0 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.620+0000 7f0411e00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04140d3a70 0x7f041415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.621+0000 7f0412800700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04141536b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f040c007e60 tx=0x7f040c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.621+0000 7f040a000700 1 -- 172.21.15.99:0/2231017168 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00ede0 con 0x7f0414160cc0 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.621+0000 7f040a000700 1 -- 172.21.15.99:0/2231017168 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c014420 con 0x7f0414160cc0 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.621+0000 7f040a000700 1 -- 172.21.15.99:0/2231017168 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c0147d0 con 0x7f0414160cc0 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.621+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04141f6f10 con 0x7f0414160cc0 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.621+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04141f74d0 con 0x7f0414160cc0 2024-03-29T09:09:25.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.622+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04141f76c0 con 0x7f0414160cc0 2024-03-29T09:09:25.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.623+0000 7f040a000700 1 -- 172.21.15.99:0/2231017168 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f040c014930 con 0x7f0414160cc0 2024-03-29T09:09:25.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.624+0000 7f040a000700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03f406ac90 0x7f03f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:25.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.624+0000 7f040a000700 1 -- 172.21.15.99:0/2231017168 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f040c089c40 con 0x7f0414160cc0 2024-03-29T09:09:25.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.624+0000 7f0411e00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03f406ac90 0x7f03f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:25.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.625+0000 7f0411e00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03f406ac90 0x7f03f406d150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0404009a50 tx=0x7f0404008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:25.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.628+0000 7f040a000700 1 -- 172.21.15.99:0/2231017168 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f04141f76c0 con 0x7f0414160cc0 2024-03-29T09:09:25.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.832+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04141f76c0 con 0x7f0414160cc0 2024-03-29T09:09:25.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.833+0000 7f040a000700 1 -- 172.21.15.99:0/2231017168 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f04141f76c0 con 0x7f0414160cc0 2024-03-29T09:09:25.833 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:25.833 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:25.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03f406ac90 msgr2=0x7f03f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03f406ac90 0x7f03f406d150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0404009a50 tx=0x7f0404008040 comp rx=0 tx=0).stop 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 msgr2=0x7f04141536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04141536b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f040c007e60 tx=0x7f040c00e3c0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 shutdown_connections 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03f406ac90 0x7f03f406d150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0414160cc0 0x7f04141536b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04140d3a70 0x7f041415a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.835+0000 7f0418a00700 1 --2- 172.21.15.99:0/2231017168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04141608e0 0x7f04141f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.836+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 >> 172.21.15.99:0/2231017168 conn(0x7f041409c490 msgr2=0x7f041415f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.836+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 shutdown_connections 2024-03-29T09:09:25.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:25.836+0000 7f0418a00700 1 -- 172.21.15.99:0/2231017168 wait complete. 2024-03-29T09:09:25.857 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T09:09:25.857 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:09:26.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.047+0000 7ff40c600700 1 Processor -- start 2024-03-29T09:09:26.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.047+0000 7ff40c600700 1 -- start start 2024-03-29T09:09:26.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.048+0000 7ff40c600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 0x7ff4040d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.048+0000 7ff40c600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040cddc0 con 0x7ff4040d1b20 2024-03-29T09:09:26.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.048+0000 7ff40c600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040ce0a0 con 0x7ff4041608e0 2024-03-29T09:09:26.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.048+0000 7ff40c600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040ce380 con 0x7ff404160cc0 2024-03-29T09:09:26.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.048+0000 7ff409200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff4040d1b20 0x7ff4040afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50416/0 (socket says 172.21.15.99:50416) 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff409200700 1 -- 172.21.15.99:0/2317959139 learned_addr learned my addr 172.21.15.99:0/2317959139 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff40a600700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 0x7ff4040d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218535052 0 0) 0x7ff4040cddc0 con 0x7ff4040d1b20 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ec0035b0 con 0x7ff4040d1b20 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833788879 0 0) 0x7ff4040ce0a0 con 0x7ff4041608e0 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4040cddc0 con 0x7ff4041608e0 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2535658891 0 0) 0x7ff3ec0035b0 con 0x7ff4040d1b20 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 msgr2=0x7ff4040d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 0x7ff4040d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 >> v1:172.21.15.133:6789/0 conn(0x7ff4041608e0 legacy=0x7ff4040d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040ce660 con 0x7ff4040d1b20 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7ff3f0001670 con 0x7ff4040d1b20 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 >> v1:172.21.15.99:6789/0 conn(0x7ff4040d1b20 legacy=0x7ff4040afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec003bc0 0x7ff3ec003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 0x7ff3ec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec004d90 0x7ff3ec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040ce380 con 0x7ff3ec003bc0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec00bac0 con 0x7ff3ec004d90 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.049+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec00bc30 con 0x7ff404160cc0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409c00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec003bc0 0x7ff3ec003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff40a600700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec004d90 0x7ff3ec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409200700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 0x7ff3ec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409c00700 1 -- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 msgr2=0x7ff3ec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409c00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff404160cc0 0x7ff3ec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409c00700 1 -- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec004d90 msgr2=0x7ff3ec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409c00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec004d90 0x7ff3ec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409c00700 1 -- 172.21.15.99:0/2317959139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0035b0 con 0x7ff3ec003bc0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff409c00700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec003bc0 0x7ff3ec003fc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8001980 tx=0x7ff3ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e0c9202de88eb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8001470 con 0x7ff3ec003bc0 2024-03-29T09:09:26.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f80052d0 con 0x7ff3ec003bc0 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.050+0000 7ff403e00700 1 -- 172.21.15.99:0/2317959139 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8003680 con 0x7ff3ec003bc0 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.051+0000 7ff40c600700 1 -- 172.21.15.99:0/2317959139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec003bc0 msgr2=0x7ff3ec003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.051+0000 7ff40c600700 1 --2- 172.21.15.99:0/2317959139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3ec003bc0 0x7ff3ec003fc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8001980 tx=0x7ff3ec005880 comp rx=0 tx=0).stop 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.051+0000 7ff40a600700 1 -- 172.21.15.99:0/2317959139 reap_dead start 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.051+0000 7ff40c600700 1 -- 172.21.15.99:0/2317959139 shutdown_connections 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.051+0000 7ff40c600700 1 -- 172.21.15.99:0/2317959139 >> 172.21.15.99:0/2317959139 conn(0x7ff40409c490 msgr2=0x7ff4040d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.051+0000 7ff40c600700 1 -- 172.21.15.99:0/2317959139 shutdown_connections 2024-03-29T09:09:26.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.051+0000 7ff40c600700 1 -- 172.21.15.99:0/2317959139 wait complete. 2024-03-29T09:09:26.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.053+0000 7ff40c600700 1 Processor -- start 2024-03-29T09:09:26.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.053+0000 7ff40c600700 1 -- start start 2024-03-29T09:09:26.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff40c600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec003bc0 0x7ff4041f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff40c600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4041f3080 0x7ff4041f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff40c600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff40414fa10 0x7ff40414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff40c600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404151c10 con 0x7ff4041f3080 2024-03-29T09:09:26.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff40c600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404151000 con 0x7ff3ec003bc0 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff40c600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4041512e0 con 0x7ff40414fa10 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff409200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4041f3080 0x7ff4041f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff409200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4041f3080 0x7ff4041f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48392/0 (socket says 172.21.15.99:48392) 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff409c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec003bc0 0x7ff4041f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff409200700 1 -- 172.21.15.99:0/3800572661 learned_addr learned my addr 172.21.15.99:0/3800572661 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.054+0000 7ff40a600700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff40414fa10 0x7ff40414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff409200700 1 -- 172.21.15.99:0/3800572661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff40414fa10 msgr2=0x7ff40414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff409200700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff40414fa10 0x7ff40414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff409200700 1 -- 172.21.15.99:0/3800572661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec003bc0 msgr2=0x7ff4041f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff409200700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec003bc0 0x7ff4041f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff409200700 1 -- 172.21.15.99:0/3800572661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4041515c0 con 0x7ff4041f3080 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff409200700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4041f3080 0x7ff4041f3500 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e800 tx=0x7ff3f000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:26.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff401600700 1 -- 172.21.15.99:0/3800572661 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0012590 con 0x7ff4041f3080 2024-03-29T09:09:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff401600700 1 -- 172.21.15.99:0/3800572661 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0012710 con 0x7ff4041f3080 2024-03-29T09:09:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff401600700 1 -- 172.21.15.99:0/3800572661 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0012ac0 con 0x7ff4041f3080 2024-03-29T09:09:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff40c600700 1 -- 172.21.15.99:0/3800572661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4041518a0 con 0x7ff4041f3080 2024-03-29T09:09:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.055+0000 7ff40c600700 1 -- 172.21.15.99:0/3800572661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4041520a0 con 0x7ff4041f3080 2024-03-29T09:09:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.056+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4040677a0 con 0x7ff4041f3080 2024-03-29T09:09:26.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.057+0000 7ff401600700 1 -- 172.21.15.99:0/3800572661 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff3f0012c20 con 0x7ff4041f3080 2024-03-29T09:09:26.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.058+0000 7ff401600700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff3e406acc0 0x7ff3e406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.059+0000 7ff401600700 1 -- 172.21.15.99:0/3800572661 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff3f00893e0 con 0x7ff4041f3080 2024-03-29T09:09:26.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.059+0000 7ff409c00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff3e406acc0 0x7ff3e406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.059+0000 7ff409c00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff3e406acc0 0x7ff3e406d180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8015040 tx=0x7ff3f800a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:26.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.063+0000 7ff401600700 1 -- 172.21.15.99:0/3800572661 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff3f00541a0 con 0x7ff4041f3080 2024-03-29T09:09:26.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.262+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4040b22a0 con 0x7ff3e406acc0 2024-03-29T09:09:26.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.274+0000 7ff401600700 1 -- 172.21.15.99:0/3800572661 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7ff3f80050d0 con 0x7ff3e406acc0 2024-03-29T09:09:26.279 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff3e406acc0 msgr2=0x7ff3e406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff3e406acc0 0x7ff3e406d180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8015040 tx=0x7ff3f800a000 comp rx=0 tx=0).stop 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4041f3080 msgr2=0x7ff4041f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4041f3080 0x7ff4041f3500 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000e800 tx=0x7ff3f000eb10 comp rx=0 tx=0).stop 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 shutdown_connections 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff3e406acc0 0x7ff3e406d180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff40414fa10 0x7ff40414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3ec003bc0 0x7ff4041f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 --2- 172.21.15.99:0/3800572661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4041f3080 0x7ff4041f3500 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 >> 172.21.15.99:0/3800572661 conn(0x7ff40409c490 msgr2=0x7ff40415e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:26.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 shutdown_connections 2024-03-29T09:09:26.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.279+0000 7ff3f6a00700 1 -- 172.21.15.99:0/3800572661 wait complete. 2024-03-29T09:09:26.283 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:09:26.298 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-29T09:09:24.455127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219392,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755328,"statfs":{"total":773094113280,"available":772869455872,"internally_reserved":0,"allocated":3284992,"data_stored":1988228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534081+0000","last_change":"2024-03-29T09:08:57.419769+0000","last_active":"2024-03-29T09:09:01.534081+0000","last_peered":"2024-03-29T09:09:01.534081+0000","last_clean":"2024-03-29T09:09:01.534081+0000","last_became_active":"2024-03-29T09:08:57.419455+0000","last_became_peered":"2024-03-29T09:08:57.419455+0000","last_unstale":"2024-03-29T09:09:01.534081+0000","last_undegraded":"2024-03-29T09:09:01.534081+0000","last_fullsized":"2024-03-29T09:09:01.534081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530834+0000","last_change":"2024-03-29T09:08:57.427159+0000","last_active":"2024-03-29T09:09:01.530834+0000","last_peered":"2024-03-29T09:09:01.530834+0000","last_clean":"2024-03-29T09:09:01.530834+0000","last_became_active":"2024-03-29T09:08:57.426926+0000","last_became_peered":"2024-03-29T09:08:57.426926+0000","last_unstale":"2024-03-29T09:09:01.530834+0000","last_undegraded":"2024-03-29T09:09:01.530834+0000","last_fullsized":"2024-03-29T09:09:01.530834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:08:57.419169+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:08:57.418824+0000","last_became_peered":"2024-03-29T09:08:57.418824+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530961+0000","last_change":"2024-03-29T09:08:57.409236+0000","last_active":"2024-03-29T09:09:01.530961+0000","last_peered":"2024-03-29T09:09:01.530961+0000","last_clean":"2024-03-29T09:09:01.530961+0000","last_became_active":"2024-03-29T09:08:57.408981+0000","last_became_peered":"2024-03-29T09:08:57.408981+0000","last_unstale":"2024-03-29T09:09:01.530961+0000","last_undegraded":"2024-03-29T09:09:01.530961+0000","last_fullsized":"2024-03-29T09:09:01.530961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533948+0000","last_change":"2024-03-29T09:08:57.423360+0000","last_active":"2024-03-29T09:09:01.533948+0000","last_peered":"2024-03-29T09:09:01.533948+0000","last_clean":"2024-03-29T09:09:01.533948+0000","last_became_active":"2024-03-29T09:08:57.423088+0000","last_became_peered":"2024-03-29T09:08:57.423088+0000","last_unstale":"2024-03-29T09:09:01.533948+0000","last_undegraded":"2024-03-29T09:09:01.533948+0000","last_fullsized":"2024-03-29T09:09:01.533948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530887+0000","last_change":"2024-03-29T09:08:57.415603+0000","last_active":"2024-03-29T09:09:01.530887+0000","last_peered":"2024-03-29T09:09:01.530887+0000","last_clean":"2024-03-29T09:09:01.530887+0000","last_became_active":"2024-03-29T09:08:57.413227+0000","last_became_peered":"2024-03-29T09:08:57.413227+0000","last_unstale":"2024-03-29T09:09:01.530887+0000","last_undegraded":"2024-03-29T09:09:01.530887+0000","last_fullsized":"2024-03-29T09:09:01.530887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533929+0000","last_change":"2024-03-29T09:08:57.421465+0000","last_active":"2024-03-29T09:09:01.533929+0000","last_peered":"2024-03-29T09:09:01.533929+0000","last_clean":"2024-03-29T09:09:01.533929+0000","last_became_active":"2024-03-29T09:08:57.421189+0000","last_became_peered":"2024-03-29T09:08:57.421189+0000","last_unstale":"2024-03-29T09:09:01.533929+0000","last_undegraded":"2024-03-29T09:09:01.533929+0000","last_fullsized":"2024-03-29T09:09:01.533929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530711+0000","last_change":"2024-03-29T09:08:57.430900+0000","last_active":"2024-03-29T09:09:01.530711+0000","last_peered":"2024-03-29T09:09:01.530711+0000","last_clean":"2024-03-29T09:09:01.530711+0000","last_became_active":"2024-03-29T09:08:57.430636+0000","last_became_peered":"2024-03-29T09:08:57.430636+0000","last_unstale":"2024-03-29T09:09:01.530711+0000","last_undegraded":"2024-03-29T09:09:01.530711+0000","last_fullsized":"2024-03-29T09:09:01.530711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528748+0000","last_change":"2024-03-29T09:08:57.413603+0000","last_active":"2024-03-29T09:09:01.528748+0000","last_peered":"2024-03-29T09:09:01.528748+0000","last_clean":"2024-03-29T09:09:01.528748+0000","last_became_active":"2024-03-29T09:08:57.413373+0000","last_became_peered":"2024-03-29T09:08:57.413373+0000","last_unstale":"2024-03-29T09:09:01.528748+0000","last_undegraded":"2024-03-29T09:09:01.528748+0000","last_fullsized":"2024-03-29T09:09:01.528748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534071+0000","last_change":"2024-03-29T09:08:57.417036+0000","last_active":"2024-03-29T09:09:01.534071+0000","last_peered":"2024-03-29T09:09:01.534071+0000","last_clean":"2024-03-29T09:09:01.534071+0000","last_became_active":"2024-03-29T09:08:57.416762+0000","last_became_peered":"2024-03-29T09:08:57.416762+0000","last_unstale":"2024-03-29T09:09:01.534071+0000","last_undegraded":"2024-03-29T09:09:01.534071+0000","last_fullsized":"2024-03-29T09:09:01.534071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534055+0000","last_change":"2024-03-29T09:08:57.417500+0000","last_active":"2024-03-29T09:09:01.534055+0000","last_peered":"2024-03-29T09:09:01.534055+0000","last_clean":"2024-03-29T09:09:01.534055+0000","last_became_active":"2024-03-29T09:08:57.417187+0000","last_became_peered":"2024-03-29T09:08:57.417187+0000","last_unstale":"2024-03-29T09:09:01.534055+0000","last_undegraded":"2024-03-29T09:09:01.534055+0000","last_fullsized":"2024-03-29T09:09:01.534055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534022+0000","last_change":"2024-03-29T09:08:57.416133+0000","last_active":"2024-03-29T09:09:01.534022+0000","last_peered":"2024-03-29T09:09:01.534022+0000","last_clean":"2024-03-29T09:09:01.534022+0000","last_became_active":"2024-03-29T09:08:57.415878+0000","last_became_peered":"2024-03-29T09:08:57.415878+0000","last_unstale":"2024-03-29T09:09:01.534022+0000","last_undegraded":"2024-03-29T09:09:01.534022+0000","last_fullsized":"2024-03-29T09:09:01.534022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525479+0000","last_change":"2024-03-29T09:08:57.418824+0000","last_active":"2024-03-29T09:09:01.525479+0000","last_peered":"2024-03-29T09:09:01.525479+0000","last_clean":"2024-03-29T09:09:01.525479+0000","last_became_active":"2024-03-29T09:08:57.418534+0000","last_became_peered":"2024-03-29T09:08:57.418534+0000","last_unstale":"2024-03-29T09:09:01.525479+0000","last_undegraded":"2024-03-29T09:09:01.525479+0000","last_fullsized":"2024-03-29T09:09:01.525479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:17:11.207261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525458+0000","last_change":"2024-03-29T09:08:57.412768+0000","last_active":"2024-03-29T09:09:01.525458+0000","last_peered":"2024-03-29T09:09:01.525458+0000","last_clean":"2024-03-29T09:09:01.525458+0000","last_became_active":"2024-03-29T09:08:57.412077+0000","last_became_peered":"2024-03-29T09:08:57.412077+0000","last_unstale":"2024-03-29T09:09:01.525458+0000","last_undegraded":"2024-03-29T09:09:01.525458+0000","last_fullsized":"2024-03-29T09:09:01.525458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530745+0000","last_change":"2024-03-29T09:08:57.430532+0000","last_active":"2024-03-29T09:09:01.530745+0000","last_peered":"2024-03-29T09:09:01.530745+0000","last_clean":"2024-03-29T09:09:01.530745+0000","last_became_active":"2024-03-29T09:08:57.430163+0000","last_became_peered":"2024-03-29T09:08:57.430163+0000","last_unstale":"2024-03-29T09:09:01.530745+0000","last_undegraded":"2024-03-29T09:09:01.530745+0000","last_fullsized":"2024-03-29T09:09:01.530745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527616+0000","last_change":"2024-03-29T09:08:57.413899+0000","last_active":"2024-03-29T09:09:01.527616+0000","last_peered":"2024-03-29T09:09:01.527616+0000","last_clean":"2024-03-29T09:09:01.527616+0000","last_became_active":"2024-03-29T09:08:57.413637+0000","last_became_peered":"2024-03-29T09:08:57.413637+0000","last_unstale":"2024-03-29T09:09:01.527616+0000","last_undegraded":"2024-03-29T09:09:01.527616+0000","last_fullsized":"2024-03-29T09:09:01.527616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:32.639943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527925+0000","last_change":"2024-03-29T09:08:57.418000+0000","last_active":"2024-03-29T09:09:01.527925+0000","last_peered":"2024-03-29T09:09:01.527925+0000","last_clean":"2024-03-29T09:09:01.527925+0000","last_became_active":"2024-03-29T09:08:57.417649+0000","last_became_peered":"2024-03-29T09:08:57.417649+0000","last_unstale":"2024-03-29T09:09:01.527925+0000","last_undegraded":"2024-03-29T09:09:01.527925+0000","last_fullsized":"2024-03-29T09:09:01.527925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:06.479973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527377+0000","last_change":"2024-03-29T09:08:57.430547+0000","last_active":"2024-03-29T09:09:01.527377+0000","last_peered":"2024-03-29T09:09:01.527377+0000","last_clean":"2024-03-29T09:09:01.527377+0000","last_became_active":"2024-03-29T09:08:57.430013+0000","last_became_peered":"2024-03-29T09:08:57.430013+0000","last_unstale":"2024-03-29T09:09:01.527377+0000","last_undegraded":"2024-03-29T09:09:01.527377+0000","last_fullsized":"2024-03-29T09:09:01.527377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527878+0000","last_change":"2024-03-29T09:08:57.419008+0000","last_active":"2024-03-29T09:09:01.527878+0000","last_peered":"2024-03-29T09:09:01.527878+0000","last_clean":"2024-03-29T09:09:01.527878+0000","last_became_active":"2024-03-29T09:08:57.418723+0000","last_became_peered":"2024-03-29T09:08:57.418723+0000","last_unstale":"2024-03-29T09:09:01.527878+0000","last_undegraded":"2024-03-29T09:09:01.527878+0000","last_fullsized":"2024-03-29T09:09:01.527878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:06.268267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527289+0000","last_change":"2024-03-29T09:09:01.448984+0000","last_active":"2024-03-29T09:09:01.527289+0000","last_peered":"2024-03-29T09:09:01.527289+0000","last_clean":"2024-03-29T09:09:01.527289+0000","last_became_active":"2024-03-29T09:09:01.446825+0000","last_became_peered":"2024-03-29T09:09:01.446825+0000","last_unstale":"2024-03-29T09:09:01.527289+0000","last_undegraded":"2024-03-29T09:09:01.527289+0000","last_fullsized":"2024-03-29T09:09:01.527289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:55.385731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.425281+0000","last_change":"2024-03-29T09:08:56.898980+0000","last_active":"2024-03-29T09:09:01.425281+0000","last_peered":"2024-03-29T09:09:01.425281+0000","last_clean":"2024-03-29T09:09:01.425281+0000","last_became_active":"2024-03-29T09:08:56.898646+0000","last_became_peered":"2024-03-29T09:08:56.898646+0000","last_unstale":"2024-03-29T09:09:01.425281+0000","last_undegraded":"2024-03-29T09:09:01.425281+0000","last_fullsized":"2024-03-29T09:09:01.425281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530854+0000","last_change":"2024-03-29T09:08:57.420574+0000","last_active":"2024-03-29T09:09:01.530854+0000","last_peered":"2024-03-29T09:09:01.530854+0000","last_clean":"2024-03-29T09:09:01.530854+0000","last_became_active":"2024-03-29T09:08:57.420318+0000","last_became_peered":"2024-03-29T09:08:57.420318+0000","last_unstale":"2024-03-29T09:09:01.530854+0000","last_undegraded":"2024-03-29T09:09:01.530854+0000","last_fullsized":"2024-03-29T09:09:01.530854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530783+0000","last_change":"2024-03-29T09:09:01.444440+0000","last_active":"2024-03-29T09:09:01.530783+0000","last_peered":"2024-03-29T09:09:01.530783+0000","last_clean":"2024-03-29T09:09:01.530783+0000","last_became_active":"2024-03-29T09:09:01.444026+0000","last_became_peered":"2024-03-29T09:09:01.444026+0000","last_unstale":"2024-03-29T09:09:01.530783+0000","last_undegraded":"2024-03-29T09:09:01.530783+0000","last_fullsized":"2024-03-29T09:09:01.530783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.421739+0000","last_change":"2024-03-29T09:08:56.908349+0000","last_active":"2024-03-29T09:09:01.421739+0000","last_peered":"2024-03-29T09:09:01.421739+0000","last_clean":"2024-03-29T09:09:01.421739+0000","last_became_active":"2024-03-29T09:08:56.908070+0000","last_became_peered":"2024-03-29T09:08:56.908070+0000","last_unstale":"2024-03-29T09:09:01.421739+0000","last_undegraded":"2024-03-29T09:09:01.421739+0000","last_fullsized":"2024-03-29T09:09:01.421739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530920+0000","last_change":"2024-03-29T09:08:57.422633+0000","last_active":"2024-03-29T09:09:01.530920+0000","last_peered":"2024-03-29T09:09:01.530920+0000","last_clean":"2024-03-29T09:09:01.530920+0000","last_became_active":"2024-03-29T09:08:57.422298+0000","last_became_peered":"2024-03-29T09:08:57.422298+0000","last_unstale":"2024-03-29T09:09:01.530920+0000","last_undegraded":"2024-03-29T09:09:01.530920+0000","last_fullsized":"2024-03-29T09:09:01.530920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530925+0000","last_change":"2024-03-29T09:09:01.435391+0000","last_active":"2024-03-29T09:09:01.530925+0000","last_peered":"2024-03-29T09:09:01.530925+0000","last_clean":"2024-03-29T09:09:01.530925+0000","last_became_active":"2024-03-29T09:09:01.435087+0000","last_became_peered":"2024-03-29T09:09:01.435087+0000","last_unstale":"2024-03-29T09:09:01.530925+0000","last_undegraded":"2024-03-29T09:09:01.530925+0000","last_fullsized":"2024-03-29T09:09:01.530925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424816+0000","last_change":"2024-03-29T09:08:56.910805+0000","last_active":"2024-03-29T09:09:01.424816+0000","last_peered":"2024-03-29T09:09:01.424816+0000","last_clean":"2024-03-29T09:09:01.424816+0000","last_became_active":"2024-03-29T09:08:56.908213+0000","last_became_peered":"2024-03-29T09:08:56.908213+0000","last_unstale":"2024-03-29T09:09:01.424816+0000","last_undegraded":"2024-03-29T09:09:01.424816+0000","last_fullsized":"2024-03-29T09:09:01.424816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528733+0000","last_change":"2024-03-29T09:08:57.420307+0000","last_active":"2024-03-29T09:09:01.528733+0000","last_peered":"2024-03-29T09:09:01.528733+0000","last_clean":"2024-03-29T09:09:01.528733+0000","last_became_active":"2024-03-29T09:08:57.420033+0000","last_became_peered":"2024-03-29T09:08:57.420033+0000","last_unstale":"2024-03-29T09:09:01.528733+0000","last_undegraded":"2024-03-29T09:09:01.528733+0000","last_fullsized":"2024-03-29T09:09:01.528733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530866+0000","last_change":"2024-03-29T09:09:01.441108+0000","last_active":"2024-03-29T09:09:01.530866+0000","last_peered":"2024-03-29T09:09:01.530866+0000","last_clean":"2024-03-29T09:09:01.530866+0000","last_became_active":"2024-03-29T09:09:01.440455+0000","last_became_peered":"2024-03-29T09:09:01.440455+0000","last_unstale":"2024-03-29T09:09:01.530866+0000","last_undegraded":"2024-03-29T09:09:01.530866+0000","last_fullsized":"2024-03-29T09:09:01.530866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424962+0000","last_change":"2024-03-29T09:08:56.908246+0000","last_active":"2024-03-29T09:09:01.424962+0000","last_peered":"2024-03-29T09:09:01.424962+0000","last_clean":"2024-03-29T09:09:01.424962+0000","last_became_active":"2024-03-29T09:08:56.907815+0000","last_became_peered":"2024-03-29T09:08:56.907815+0000","last_unstale":"2024-03-29T09:09:01.424962+0000","last_undegraded":"2024-03-29T09:09:01.424962+0000","last_fullsized":"2024-03-29T09:09:01.424962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527840+0000","last_change":"2024-03-29T09:08:57.420229+0000","last_active":"2024-03-29T09:09:01.527840+0000","last_peered":"2024-03-29T09:09:01.527840+0000","last_clean":"2024-03-29T09:09:01.527840+0000","last_became_active":"2024-03-29T09:08:57.419940+0000","last_became_peered":"2024-03-29T09:08:57.419940+0000","last_unstale":"2024-03-29T09:09:01.527840+0000","last_undegraded":"2024-03-29T09:09:01.527840+0000","last_fullsized":"2024-03-29T09:09:01.527840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:19.867399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527346+0000","last_change":"2024-03-29T09:09:01.447861+0000","last_active":"2024-03-29T09:09:01.527346+0000","last_peered":"2024-03-29T09:09:01.527346+0000","last_clean":"2024-03-29T09:09:01.527346+0000","last_became_active":"2024-03-29T09:09:01.447337+0000","last_became_peered":"2024-03-29T09:09:01.447337+0000","last_unstale":"2024-03-29T09:09:01.527346+0000","last_undegraded":"2024-03-29T09:09:01.527346+0000","last_fullsized":"2024-03-29T09:09:01.527346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:19.879877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419991+0000","last_change":"2024-03-29T09:08:56.908710+0000","last_active":"2024-03-29T09:09:01.419991+0000","last_peered":"2024-03-29T09:09:01.419991+0000","last_clean":"2024-03-29T09:09:01.419991+0000","last_became_active":"2024-03-29T09:08:56.906213+0000","last_became_peered":"2024-03-29T09:08:56.906213+0000","last_unstale":"2024-03-29T09:09:01.419991+0000","last_undegraded":"2024-03-29T09:09:01.419991+0000","last_fullsized":"2024-03-29T09:09:01.419991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528714+0000","last_change":"2024-03-29T09:08:57.423981+0000","last_active":"2024-03-29T09:09:01.528714+0000","last_peered":"2024-03-29T09:09:01.528714+0000","last_clean":"2024-03-29T09:09:01.528714+0000","last_became_active":"2024-03-29T09:08:57.423667+0000","last_became_peered":"2024-03-29T09:08:57.423667+0000","last_unstale":"2024-03-29T09:09:01.528714+0000","last_undegraded":"2024-03-29T09:09:01.528714+0000","last_fullsized":"2024-03-29T09:09:01.528714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527487+0000","last_change":"2024-03-29T09:09:01.455365+0000","last_active":"2024-03-29T09:09:01.527487+0000","last_peered":"2024-03-29T09:09:01.527487+0000","last_clean":"2024-03-29T09:09:01.527487+0000","last_became_active":"2024-03-29T09:09:01.453949+0000","last_became_peered":"2024-03-29T09:09:01.453949+0000","last_unstale":"2024-03-29T09:09:01.527487+0000","last_undegraded":"2024-03-29T09:09:01.527487+0000","last_fullsized":"2024-03-29T09:09:01.527487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:00.449212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414280+0000","last_change":"2024-03-29T09:08:56.912593+0000","last_active":"2024-03-29T09:09:01.414280+0000","last_peered":"2024-03-29T09:09:01.414280+0000","last_clean":"2024-03-29T09:09:01.414280+0000","last_became_active":"2024-03-29T09:08:56.912401+0000","last_became_peered":"2024-03-29T09:08:56.912401+0000","last_unstale":"2024-03-29T09:09:01.414280+0000","last_undegraded":"2024-03-29T09:09:01.414280+0000","last_fullsized":"2024-03-29T09:09:01.414280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:27.805156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527468+0000","last_change":"2024-03-29T09:08:57.421182+0000","last_active":"2024-03-29T09:09:01.527468+0000","last_peered":"2024-03-29T09:09:01.527468+0000","last_clean":"2024-03-29T09:09:01.527468+0000","last_became_active":"2024-03-29T09:08:57.420980+0000","last_became_peered":"2024-03-29T09:08:57.420980+0000","last_unstale":"2024-03-29T09:09:01.527468+0000","last_undegraded":"2024-03-29T09:09:01.527468+0000","last_fullsized":"2024-03-29T09:09:01.527468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527895+0000","last_change":"2024-03-29T09:09:01.433710+0000","last_active":"2024-03-29T09:09:01.527895+0000","last_peered":"2024-03-29T09:09:01.527895+0000","last_clean":"2024-03-29T09:09:01.527895+0000","last_became_active":"2024-03-29T09:09:01.433161+0000","last_became_peered":"2024-03-29T09:09:01.433161+0000","last_unstale":"2024-03-29T09:09:01.527895+0000","last_undegraded":"2024-03-29T09:09:01.527895+0000","last_fullsized":"2024-03-29T09:09:01.527895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:07:30.745379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429199+0000","last_change":"2024-03-29T09:08:56.911770+0000","last_active":"2024-03-29T09:09:01.429199+0000","last_peered":"2024-03-29T09:09:01.429199+0000","last_clean":"2024-03-29T09:09:01.429199+0000","last_became_active":"2024-03-29T09:08:56.911493+0000","last_became_peered":"2024-03-29T09:08:56.911493+0000","last_unstale":"2024-03-29T09:09:01.429199+0000","last_undegraded":"2024-03-29T09:09:01.429199+0000","last_fullsized":"2024-03-29T09:09:01.429199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530721+0000","last_change":"2024-03-29T09:08:57.426875+0000","last_active":"2024-03-29T09:09:01.530721+0000","last_peered":"2024-03-29T09:09:01.530721+0000","last_clean":"2024-03-29T09:09:01.530721+0000","last_became_active":"2024-03-29T09:08:57.426586+0000","last_became_peered":"2024-03-29T09:08:57.426586+0000","last_unstale":"2024-03-29T09:09:01.530721+0000","last_undegraded":"2024-03-29T09:09:01.530721+0000","last_fullsized":"2024-03-29T09:09:01.530721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527529+0000","last_change":"2024-03-29T09:09:01.440649+0000","last_active":"2024-03-29T09:09:01.527529+0000","last_peered":"2024-03-29T09:09:01.527529+0000","last_clean":"2024-03-29T09:09:01.527529+0000","last_became_active":"2024-03-29T09:09:01.440252+0000","last_became_peered":"2024-03-29T09:09:01.440252+0000","last_unstale":"2024-03-29T09:09:01.527529+0000","last_undegraded":"2024-03-29T09:09:01.527529+0000","last_fullsized":"2024-03-29T09:09:01.527529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414326+0000","last_change":"2024-03-29T09:08:56.908431+0000","last_active":"2024-03-29T09:09:01.414326+0000","last_peered":"2024-03-29T09:09:01.414326+0000","last_clean":"2024-03-29T09:09:01.414326+0000","last_became_active":"2024-03-29T09:08:56.906823+0000","last_became_peered":"2024-03-29T09:08:56.906823+0000","last_unstale":"2024-03-29T09:09:01.414326+0000","last_undegraded":"2024-03-29T09:09:01.414326+0000","last_fullsized":"2024-03-29T09:09:01.414326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:24.070569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527421+0000","last_change":"2024-03-29T09:08:57.410009+0000","last_active":"2024-03-29T09:09:01.527421+0000","last_peered":"2024-03-29T09:09:01.527421+0000","last_clean":"2024-03-29T09:09:01.527421+0000","last_became_active":"2024-03-29T09:08:57.409632+0000","last_became_peered":"2024-03-29T09:08:57.409632+0000","last_unstale":"2024-03-29T09:09:01.527421+0000","last_undegraded":"2024-03-29T09:09:01.527421+0000","last_fullsized":"2024-03-29T09:09:01.527421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527438+0000","last_change":"2024-03-29T09:09:01.447901+0000","last_active":"2024-03-29T09:09:01.527438+0000","last_peered":"2024-03-29T09:09:01.527438+0000","last_clean":"2024-03-29T09:09:01.527438+0000","last_became_active":"2024-03-29T09:09:01.447482+0000","last_became_peered":"2024-03-29T09:09:01.447482+0000","last_unstale":"2024-03-29T09:09:01.527438+0000","last_undegraded":"2024-03-29T09:09:01.527438+0000","last_fullsized":"2024-03-29T09:09:01.527438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.426470+0000","last_change":"2024-03-29T09:08:56.909506+0000","last_active":"2024-03-29T09:09:01.426470+0000","last_peered":"2024-03-29T09:09:01.426470+0000","last_clean":"2024-03-29T09:09:01.426470+0000","last_became_active":"2024-03-29T09:08:56.909183+0000","last_became_peered":"2024-03-29T09:08:56.909183+0000","last_unstale":"2024-03-29T09:09:01.426470+0000","last_undegraded":"2024-03-29T09:09:01.426470+0000","last_fullsized":"2024-03-29T09:09:01.426470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:04:15.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531038+0000","last_change":"2024-03-29T09:08:57.413715+0000","last_active":"2024-03-29T09:09:01.531038+0000","last_peered":"2024-03-29T09:09:01.531038+0000","last_clean":"2024-03-29T09:09:01.531038+0000","last_became_active":"2024-03-29T09:08:57.413145+0000","last_became_peered":"2024-03-29T09:08:57.413145+0000","last_unstale":"2024-03-29T09:09:01.531038+0000","last_undegraded":"2024-03-29T09:09:01.531038+0000","last_fullsized":"2024-03-29T09:09:01.531038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527274+0000","last_change":"2024-03-29T09:09:01.444258+0000","last_active":"2024-03-29T09:09:01.527274+0000","last_peered":"2024-03-29T09:09:01.527274+0000","last_clean":"2024-03-29T09:09:01.527274+0000","last_became_active":"2024-03-29T09:09:01.443829+0000","last_became_peered":"2024-03-29T09:09:01.443829+0000","last_unstale":"2024-03-29T09:09:01.527274+0000","last_undegraded":"2024-03-29T09:09:01.527274+0000","last_fullsized":"2024-03-29T09:09:01.527274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:29:01.967027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424436+0000","last_change":"2024-03-29T09:08:56.909671+0000","last_active":"2024-03-29T09:09:01.424436+0000","last_peered":"2024-03-29T09:09:01.424436+0000","last_clean":"2024-03-29T09:09:01.424436+0000","last_became_active":"2024-03-29T09:08:56.909426+0000","last_became_peered":"2024-03-29T09:08:56.909426+0000","last_unstale":"2024-03-29T09:09:01.424436+0000","last_undegraded":"2024-03-29T09:09:01.424436+0000","last_fullsized":"2024-03-29T09:09:01.424436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530931+0000","last_change":"2024-03-29T09:08:57.422965+0000","last_active":"2024-03-29T09:09:01.530931+0000","last_peered":"2024-03-29T09:09:01.530931+0000","last_clean":"2024-03-29T09:09:01.530931+0000","last_became_active":"2024-03-29T09:08:57.422689+0000","last_became_peered":"2024-03-29T09:08:57.422689+0000","last_unstale":"2024-03-29T09:09:01.530931+0000","last_undegraded":"2024-03-29T09:09:01.530931+0000","last_fullsized":"2024-03-29T09:09:01.530931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530907+0000","last_change":"2024-03-29T09:09:01.447399+0000","last_active":"2024-03-29T09:09:01.530907+0000","last_peered":"2024-03-29T09:09:01.530907+0000","last_clean":"2024-03-29T09:09:01.530907+0000","last_became_active":"2024-03-29T09:09:01.446880+0000","last_became_peered":"2024-03-29T09:09:01.446880+0000","last_unstale":"2024-03-29T09:09:01.530907+0000","last_undegraded":"2024-03-29T09:09:01.530907+0000","last_fullsized":"2024-03-29T09:09:01.530907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:49:27.133997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.433574+0000","last_change":"2024-03-29T09:08:56.908399+0000","last_active":"2024-03-29T09:09:01.433574+0000","last_peered":"2024-03-29T09:09:01.433574+0000","last_clean":"2024-03-29T09:09:01.433574+0000","last_became_active":"2024-03-29T09:08:56.906866+0000","last_became_peered":"2024-03-29T09:08:56.906866+0000","last_unstale":"2024-03-29T09:09:01.433574+0000","last_undegraded":"2024-03-29T09:09:01.433574+0000","last_fullsized":"2024-03-29T09:09:01.433574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527738+0000","last_change":"2024-03-29T09:08:57.423735+0000","last_active":"2024-03-29T09:09:01.527738+0000","last_peered":"2024-03-29T09:09:01.527738+0000","last_clean":"2024-03-29T09:09:01.527738+0000","last_became_active":"2024-03-29T09:08:57.423405+0000","last_became_peered":"2024-03-29T09:08:57.423405+0000","last_unstale":"2024-03-29T09:09:01.527738+0000","last_undegraded":"2024-03-29T09:09:01.527738+0000","last_fullsized":"2024-03-29T09:09:01.527738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527448+0000","last_change":"2024-03-29T09:09:01.442875+0000","last_active":"2024-03-29T09:09:01.527448+0000","last_peered":"2024-03-29T09:09:01.527448+0000","last_clean":"2024-03-29T09:09:01.527448+0000","last_became_active":"2024-03-29T09:09:01.442426+0000","last_became_peered":"2024-03-29T09:09:01.442426+0000","last_unstale":"2024-03-29T09:09:01.527448+0000","last_undegraded":"2024-03-29T09:09:01.527448+0000","last_fullsized":"2024-03-29T09:09:01.527448+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T09:09:15.577021+0000","last_change":"2024-03-29T09:08:56.895791+0000","last_active":"2024-03-29T09:09:15.577021+0000","last_peered":"2024-03-29T09:09:15.577021+0000","last_clean":"2024-03-29T09:09:15.577021+0000","last_became_active":"2024-03-29T09:08:56.894997+0000","last_became_peered":"2024-03-29T09:08:56.894997+0000","last_unstale":"2024-03-29T09:09:15.577021+0000","last_undegraded":"2024-03-29T09:09:15.577021+0000","last_fullsized":"2024-03-29T09:09:15.577021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534030+0000","last_change":"2024-03-29T09:08:57.427194+0000","last_active":"2024-03-29T09:09:01.534030+0000","last_peered":"2024-03-29T09:09:01.534030+0000","last_clean":"2024-03-29T09:09:01.534030+0000","last_became_active":"2024-03-29T09:08:57.425015+0000","last_became_peered":"2024-03-29T09:08:57.425015+0000","last_unstale":"2024-03-29T09:09:01.534030+0000","last_undegraded":"2024-03-29T09:09:01.534030+0000","last_fullsized":"2024-03-29T09:09:01.534030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527755+0000","last_change":"2024-03-29T09:09:01.442768+0000","last_active":"2024-03-29T09:09:01.527755+0000","last_peered":"2024-03-29T09:09:01.527755+0000","last_clean":"2024-03-29T09:09:01.527755+0000","last_became_active":"2024-03-29T09:09:01.442373+0000","last_became_peered":"2024-03-29T09:09:01.442373+0000","last_unstale":"2024-03-29T09:09:01.527755+0000","last_undegraded":"2024-03-29T09:09:01.527755+0000","last_fullsized":"2024-03-29T09:09:01.527755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:05:41.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.432870+0000","last_change":"2024-03-29T09:08:56.912783+0000","last_active":"2024-03-29T09:09:01.432870+0000","last_peered":"2024-03-29T09:09:01.432870+0000","last_clean":"2024-03-29T09:09:01.432870+0000","last_became_active":"2024-03-29T09:08:56.912488+0000","last_became_peered":"2024-03-29T09:08:56.912488+0000","last_unstale":"2024-03-29T09:09:01.432870+0000","last_undegraded":"2024-03-29T09:09:01.432870+0000","last_fullsized":"2024-03-29T09:09:01.432870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525440+0000","last_change":"2024-03-29T09:08:57.417913+0000","last_active":"2024-03-29T09:09:01.525440+0000","last_peered":"2024-03-29T09:09:01.525440+0000","last_clean":"2024-03-29T09:09:01.525440+0000","last_became_active":"2024-03-29T09:08:57.416399+0000","last_became_peered":"2024-03-29T09:08:57.416399+0000","last_unstale":"2024-03-29T09:09:01.525440+0000","last_undegraded":"2024-03-29T09:09:01.525440+0000","last_fullsized":"2024-03-29T09:09:01.525440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525543+0000","last_change":"2024-03-29T09:09:01.448636+0000","last_active":"2024-03-29T09:09:01.525543+0000","last_peered":"2024-03-29T09:09:01.525543+0000","last_clean":"2024-03-29T09:09:01.525543+0000","last_became_active":"2024-03-29T09:09:01.448098+0000","last_became_peered":"2024-03-29T09:09:01.448098+0000","last_unstale":"2024-03-29T09:09:01.525543+0000","last_undegraded":"2024-03-29T09:09:01.525543+0000","last_fullsized":"2024-03-29T09:09:01.525543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414449+0000","last_change":"2024-03-29T09:08:56.906000+0000","last_active":"2024-03-29T09:09:01.414449+0000","last_peered":"2024-03-29T09:09:01.414449+0000","last_clean":"2024-03-29T09:09:01.414449+0000","last_became_active":"2024-03-29T09:08:56.905743+0000","last_became_peered":"2024-03-29T09:08:56.905743+0000","last_unstale":"2024-03-29T09:09:01.414449+0000","last_undegraded":"2024-03-29T09:09:01.414449+0000","last_fullsized":"2024-03-29T09:09:01.414449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527722+0000","last_change":"2024-03-29T09:08:57.429076+0000","last_active":"2024-03-29T09:09:01.527722+0000","last_peered":"2024-03-29T09:09:01.527722+0000","last_clean":"2024-03-29T09:09:01.527722+0000","last_became_active":"2024-03-29T09:08:57.426637+0000","last_became_peered":"2024-03-29T09:08:57.426637+0000","last_unstale":"2024-03-29T09:09:01.527722+0000","last_undegraded":"2024-03-29T09:09:01.527722+0000","last_fullsized":"2024-03-29T09:09:01.527722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:48.952540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530885+0000","last_change":"2024-03-29T09:09:01.437381+0000","last_active":"2024-03-29T09:09:01.530885+0000","last_peered":"2024-03-29T09:09:01.530885+0000","last_clean":"2024-03-29T09:09:01.530885+0000","last_became_active":"2024-03-29T09:09:01.436950+0000","last_became_peered":"2024-03-29T09:09:01.436950+0000","last_unstale":"2024-03-29T09:09:01.530885+0000","last_undegraded":"2024-03-29T09:09:01.530885+0000","last_fullsized":"2024-03-29T09:09:01.530885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.356624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422703+0000","last_change":"2024-03-29T09:08:56.915220+0000","last_active":"2024-03-29T09:09:01.422703+0000","last_peered":"2024-03-29T09:09:01.422703+0000","last_clean":"2024-03-29T09:09:01.422703+0000","last_became_active":"2024-03-29T09:08:56.914954+0000","last_became_peered":"2024-03-29T09:08:56.914954+0000","last_unstale":"2024-03-29T09:09:01.422703+0000","last_undegraded":"2024-03-29T09:09:01.422703+0000","last_fullsized":"2024-03-29T09:09:01.422703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:19:55.114280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528690+0000","last_change":"2024-03-29T09:08:57.423950+0000","last_active":"2024-03-29T09:09:01.528690+0000","last_peered":"2024-03-29T09:09:01.528690+0000","last_clean":"2024-03-29T09:09:01.528690+0000","last_became_active":"2024-03-29T09:08:57.423703+0000","last_became_peered":"2024-03-29T09:08:57.423703+0000","last_unstale":"2024-03-29T09:09:01.528690+0000","last_undegraded":"2024-03-29T09:09:01.528690+0000","last_fullsized":"2024-03-29T09:09:01.528690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:28.288403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530874+0000","last_change":"2024-03-29T09:09:01.440631+0000","last_active":"2024-03-29T09:09:01.530874+0000","last_peered":"2024-03-29T09:09:01.530874+0000","last_clean":"2024-03-29T09:09:01.530874+0000","last_became_active":"2024-03-29T09:09:01.440159+0000","last_became_peered":"2024-03-29T09:09:01.440159+0000","last_unstale":"2024-03-29T09:09:01.530874+0000","last_undegraded":"2024-03-29T09:09:01.530874+0000","last_fullsized":"2024-03-29T09:09:01.530874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.423297+0000","last_change":"2024-03-29T09:08:56.904028+0000","last_active":"2024-03-29T09:09:01.423297+0000","last_peered":"2024-03-29T09:09:01.423297+0000","last_clean":"2024-03-29T09:09:01.423297+0000","last_became_active":"2024-03-29T09:08:56.903620+0000","last_became_peered":"2024-03-29T09:08:56.903620+0000","last_unstale":"2024-03-29T09:09:01.423297+0000","last_undegraded":"2024-03-29T09:09:01.423297+0000","last_fullsized":"2024-03-29T09:09:01.423297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527655+0000","last_change":"2024-03-29T09:08:57.414230+0000","last_active":"2024-03-29T09:09:01.527655+0000","last_peered":"2024-03-29T09:09:01.527655+0000","last_clean":"2024-03-29T09:09:01.527655+0000","last_became_active":"2024-03-29T09:08:57.413731+0000","last_became_peered":"2024-03-29T09:08:57.413731+0000","last_unstale":"2024-03-29T09:09:01.527655+0000","last_undegraded":"2024-03-29T09:09:01.527655+0000","last_fullsized":"2024-03-29T09:09:01.527655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528703+0000","last_change":"2024-03-29T09:09:01.435579+0000","last_active":"2024-03-29T09:09:01.528703+0000","last_peered":"2024-03-29T09:09:01.528703+0000","last_clean":"2024-03-29T09:09:01.528703+0000","last_became_active":"2024-03-29T09:09:01.433461+0000","last_became_peered":"2024-03-29T09:09:01.433461+0000","last_unstale":"2024-03-29T09:09:01.528703+0000","last_undegraded":"2024-03-29T09:09:01.528703+0000","last_fullsized":"2024-03-29T09:09:01.528703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:17.513175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422681+0000","last_change":"2024-03-29T09:08:56.915230+0000","last_active":"2024-03-29T09:09:01.422681+0000","last_peered":"2024-03-29T09:09:01.422681+0000","last_clean":"2024-03-29T09:09:01.422681+0000","last_became_active":"2024-03-29T09:08:56.914943+0000","last_became_peered":"2024-03-29T09:08:56.914943+0000","last_unstale":"2024-03-29T09:09:01.422681+0000","last_undegraded":"2024-03-29T09:09:01.422681+0000","last_fullsized":"2024-03-29T09:09:01.422681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531054+0000","last_change":"2024-03-29T09:08:57.425607+0000","last_active":"2024-03-29T09:09:01.531054+0000","last_peered":"2024-03-29T09:09:01.531054+0000","last_clean":"2024-03-29T09:09:01.531054+0000","last_became_active":"2024-03-29T09:08:57.424221+0000","last_became_peered":"2024-03-29T09:08:57.424221+0000","last_unstale":"2024-03-29T09:09:01.531054+0000","last_undegraded":"2024-03-29T09:09:01.531054+0000","last_fullsized":"2024-03-29T09:09:01.531054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525430+0000","last_change":"2024-03-29T09:09:01.447292+0000","last_active":"2024-03-29T09:09:01.525430+0000","last_peered":"2024-03-29T09:09:01.525430+0000","last_clean":"2024-03-29T09:09:01.525430+0000","last_became_active":"2024-03-29T09:09:01.446779+0000","last_became_peered":"2024-03-29T09:09:01.446779+0000","last_unstale":"2024-03-29T09:09:01.525430+0000","last_undegraded":"2024-03-29T09:09:01.525430+0000","last_fullsized":"2024-03-29T09:09:01.525430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:10.002579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419894+0000","last_change":"2024-03-29T09:08:56.905021+0000","last_active":"2024-03-29T09:09:01.419894+0000","last_peered":"2024-03-29T09:09:01.419894+0000","last_clean":"2024-03-29T09:09:01.419894+0000","last_became_active":"2024-03-29T09:08:56.903363+0000","last_became_peered":"2024-03-29T09:08:56.903363+0000","last_unstale":"2024-03-29T09:09:01.419894+0000","last_undegraded":"2024-03-29T09:09:01.419894+0000","last_fullsized":"2024-03-29T09:09:01.419894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530999+0000","last_change":"2024-03-29T09:08:57.427940+0000","last_active":"2024-03-29T09:09:01.530999+0000","last_peered":"2024-03-29T09:09:01.530999+0000","last_clean":"2024-03-29T09:09:01.530999+0000","last_became_active":"2024-03-29T09:08:57.427293+0000","last_became_peered":"2024-03-29T09:08:57.427293+0000","last_unstale":"2024-03-29T09:09:01.530999+0000","last_undegraded":"2024-03-29T09:09:01.530999+0000","last_fullsized":"2024-03-29T09:09:01.530999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525532+0000","last_change":"2024-03-29T09:09:01.451319+0000","last_active":"2024-03-29T09:09:01.525532+0000","last_peered":"2024-03-29T09:09:01.525532+0000","last_clean":"2024-03-29T09:09:01.525532+0000","last_became_active":"2024-03-29T09:09:01.450880+0000","last_became_peered":"2024-03-29T09:09:01.450880+0000","last_unstale":"2024-03-29T09:09:01.525532+0000","last_undegraded":"2024-03-29T09:09:01.525532+0000","last_fullsized":"2024-03-29T09:09:01.525532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414329+0000","last_change":"2024-03-29T09:08:56.909170+0000","last_active":"2024-03-29T09:09:01.414329+0000","last_peered":"2024-03-29T09:09:01.414329+0000","last_clean":"2024-03-29T09:09:01.414329+0000","last_became_active":"2024-03-29T09:08:56.908866+0000","last_became_peered":"2024-03-29T09:08:56.908866+0000","last_unstale":"2024-03-29T09:09:01.414329+0000","last_undegraded":"2024-03-29T09:09:01.414329+0000","last_fullsized":"2024-03-29T09:09:01.414329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:58.589863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533977+0000","last_change":"2024-03-29T09:08:57.415898+0000","last_active":"2024-03-29T09:09:01.533977+0000","last_peered":"2024-03-29T09:09:01.533977+0000","last_clean":"2024-03-29T09:09:01.533977+0000","last_became_active":"2024-03-29T09:08:57.415653+0000","last_became_peered":"2024-03-29T09:08:57.415653+0000","last_unstale":"2024-03-29T09:09:01.533977+0000","last_undegraded":"2024-03-29T09:09:01.533977+0000","last_fullsized":"2024-03-29T09:09:01.533977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525566+0000","last_change":"2024-03-29T09:09:01.451066+0000","last_active":"2024-03-29T09:09:01.525566+0000","last_peered":"2024-03-29T09:09:01.525566+0000","last_clean":"2024-03-29T09:09:01.525566+0000","last_became_active":"2024-03-29T09:09:01.450444+0000","last_became_peered":"2024-03-29T09:09:01.450444+0000","last_unstale":"2024-03-29T09:09:01.525566+0000","last_undegraded":"2024-03-29T09:09:01.525566+0000","last_fullsized":"2024-03-29T09:09:01.525566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530697+0000","last_change":"2024-03-29T09:08:57.426320+0000","last_active":"2024-03-29T09:09:01.530697+0000","last_peered":"2024-03-29T09:09:01.530697+0000","last_clean":"2024-03-29T09:09:01.530697+0000","last_became_active":"2024-03-29T09:08:57.426071+0000","last_became_peered":"2024-03-29T09:08:57.426071+0000","last_unstale":"2024-03-29T09:09:01.530697+0000","last_undegraded":"2024-03-29T09:09:01.530697+0000","last_fullsized":"2024-03-29T09:09:01.530697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.428903+0000","last_change":"2024-03-29T09:08:56.906294+0000","last_active":"2024-03-29T09:09:01.428903+0000","last_peered":"2024-03-29T09:09:01.428903+0000","last_clean":"2024-03-29T09:09:01.428903+0000","last_became_active":"2024-03-29T09:08:56.905993+0000","last_became_peered":"2024-03-29T09:08:56.905993+0000","last_unstale":"2024-03-29T09:09:01.428903+0000","last_undegraded":"2024-03-29T09:09:01.428903+0000","last_fullsized":"2024-03-29T09:09:01.428903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527859+0000","last_change":"2024-03-29T09:09:01.435004+0000","last_active":"2024-03-29T09:09:01.527859+0000","last_peered":"2024-03-29T09:09:01.527859+0000","last_clean":"2024-03-29T09:09:01.527859+0000","last_became_active":"2024-03-29T09:09:01.434628+0000","last_became_peered":"2024-03-29T09:09:01.434628+0000","last_unstale":"2024-03-29T09:09:01.527859+0000","last_undegraded":"2024-03-29T09:09:01.527859+0000","last_fullsized":"2024-03-29T09:09:01.527859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:05.380910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527458+0000","last_change":"2024-03-29T09:08:57.411609+0000","last_active":"2024-03-29T09:09:01.527458+0000","last_peered":"2024-03-29T09:09:01.527458+0000","last_clean":"2024-03-29T09:09:01.527458+0000","last_became_active":"2024-03-29T09:08:57.411313+0000","last_became_peered":"2024-03-29T09:08:57.411313+0000","last_unstale":"2024-03-29T09:09:01.527458+0000","last_undegraded":"2024-03-29T09:09:01.527458+0000","last_fullsized":"2024-03-29T09:09:01.527458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414265+0000","last_change":"2024-03-29T09:08:56.900590+0000","last_active":"2024-03-29T09:09:01.414265+0000","last_peered":"2024-03-29T09:09:01.414265+0000","last_clean":"2024-03-29T09:09:01.414265+0000","last_became_active":"2024-03-29T09:08:56.900329+0000","last_became_peered":"2024-03-29T09:08:56.900329+0000","last_unstale":"2024-03-29T09:09:01.414265+0000","last_undegraded":"2024-03-29T09:09:01.414265+0000","last_fullsized":"2024-03-29T09:09:01.414265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:56.593036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420635+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:09:01.420635+0000","last_peered":"2024-03-29T09:09:01.420635+0000","last_clean":"2024-03-29T09:09:01.420635+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:09:01.420635+0000","last_undegraded":"2024-03-29T09:09:01.420635+0000","last_fullsized":"2024-03-29T09:09:01.420635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530864+0000","last_change":"2024-03-29T09:09:01.438759+0000","last_active":"2024-03-29T09:09:01.530864+0000","last_peered":"2024-03-29T09:09:01.530864+0000","last_clean":"2024-03-29T09:09:01.530864+0000","last_became_active":"2024-03-29T09:09:01.438275+0000","last_became_peered":"2024-03-29T09:09:01.438275+0000","last_unstale":"2024-03-29T09:09:01.530864+0000","last_undegraded":"2024-03-29T09:09:01.530864+0000","last_fullsized":"2024-03-29T09:09:01.530864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:02:29.227678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525418+0000","last_change":"2024-03-29T09:08:57.418469+0000","last_active":"2024-03-29T09:09:01.525418+0000","last_peered":"2024-03-29T09:09:01.525418+0000","last_clean":"2024-03-29T09:09:01.525418+0000","last_became_active":"2024-03-29T09:08:57.416011+0000","last_became_peered":"2024-03-29T09:08:57.416011+0000","last_unstale":"2024-03-29T09:09:01.525418+0000","last_undegraded":"2024-03-29T09:09:01.525418+0000","last_fullsized":"2024-03-29T09:09:01.525418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414118+0000","last_change":"2024-03-29T09:08:56.900382+0000","last_active":"2024-03-29T09:09:01.414118+0000","last_peered":"2024-03-29T09:09:01.414118+0000","last_clean":"2024-03-29T09:09:01.414118+0000","last_became_active":"2024-03-29T09:08:56.899884+0000","last_became_peered":"2024-03-29T09:08:56.899884+0000","last_unstale":"2024-03-29T09:09:01.414118+0000","last_undegraded":"2024-03-29T09:09:01.414118+0000","last_fullsized":"2024-03-29T09:09:01.414118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:51.548290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525469+0000","last_change":"2024-03-29T09:09:01.447790+0000","last_active":"2024-03-29T09:09:01.525469+0000","last_peered":"2024-03-29T09:09:01.525469+0000","last_clean":"2024-03-29T09:09:01.525469+0000","last_became_active":"2024-03-29T09:09:01.447254+0000","last_became_peered":"2024-03-29T09:09:01.447254+0000","last_unstale":"2024-03-29T09:09:01.525469+0000","last_undegraded":"2024-03-29T09:09:01.525469+0000","last_fullsized":"2024-03-29T09:09:01.525469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:27.709797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.415102+0000","last_change":"2024-03-29T09:08:56.904347+0000","last_active":"2024-03-29T09:09:01.415102+0000","last_peered":"2024-03-29T09:09:01.415102+0000","last_clean":"2024-03-29T09:09:01.415102+0000","last_became_active":"2024-03-29T09:08:56.903387+0000","last_became_peered":"2024-03-29T09:08:56.903387+0000","last_unstale":"2024-03-29T09:09:01.415102+0000","last_undegraded":"2024-03-29T09:09:01.415102+0000","last_fullsized":"2024-03-29T09:09:01.415102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525510+0000","last_change":"2024-03-29T09:08:57.419559+0000","last_active":"2024-03-29T09:09:01.525510+0000","last_peered":"2024-03-29T09:09:01.525510+0000","last_clean":"2024-03-29T09:09:01.525510+0000","last_became_active":"2024-03-29T09:08:57.419295+0000","last_became_peered":"2024-03-29T09:08:57.419295+0000","last_unstale":"2024-03-29T09:09:01.525510+0000","last_undegraded":"2024-03-29T09:09:01.525510+0000","last_fullsized":"2024-03-29T09:09:01.525510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527366+0000","last_change":"2024-03-29T09:09:01.433109+0000","last_active":"2024-03-29T09:09:01.527366+0000","last_peered":"2024-03-29T09:09:01.527366+0000","last_clean":"2024-03-29T09:09:01.527366+0000","last_became_active":"2024-03-29T09:09:01.432662+0000","last_became_peered":"2024-03-29T09:09:01.432662+0000","last_unstale":"2024-03-29T09:09:01.527366+0000","last_undegraded":"2024-03-29T09:09:01.527366+0000","last_fullsized":"2024-03-29T09:09:01.527366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422867+0000","last_change":"2024-03-29T09:08:56.912181+0000","last_active":"2024-03-29T09:09:01.422867+0000","last_peered":"2024-03-29T09:09:01.422867+0000","last_clean":"2024-03-29T09:09:01.422867+0000","last_became_active":"2024-03-29T09:08:56.911842+0000","last_became_peered":"2024-03-29T09:08:56.911842+0000","last_unstale":"2024-03-29T09:09:01.422867+0000","last_undegraded":"2024-03-29T09:09:01.422867+0000","last_fullsized":"2024-03-29T09:09:01.422867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527556+0000","last_change":"2024-03-29T09:08:57.423784+0000","last_active":"2024-03-29T09:09:01.527556+0000","last_peered":"2024-03-29T09:09:01.527556+0000","last_clean":"2024-03-29T09:09:01.527556+0000","last_became_active":"2024-03-29T09:08:57.423442+0000","last_became_peered":"2024-03-29T09:08:57.423442+0000","last_unstale":"2024-03-29T09:09:01.527556+0000","last_undegraded":"2024-03-29T09:09:01.527556+0000","last_fullsized":"2024-03-29T09:09:01.527556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:09:01.451554+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:09:01.450348+0000","last_became_peered":"2024-03-29T09:09:01.450348+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414924+0000","last_change":"2024-03-29T09:08:56.911357+0000","last_active":"2024-03-29T09:09:01.414924+0000","last_peered":"2024-03-29T09:09:01.414924+0000","last_clean":"2024-03-29T09:09:01.414924+0000","last_became_active":"2024-03-29T09:08:56.910692+0000","last_became_peered":"2024-03-29T09:08:56.910692+0000","last_unstale":"2024-03-29T09:09:01.414924+0000","last_undegraded":"2024-03-29T09:09:01.414924+0000","last_fullsized":"2024-03-29T09:09:01.414924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:32.927312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527540+0000","last_change":"2024-03-29T09:08:57.415718+0000","last_active":"2024-03-29T09:09:01.527540+0000","last_peered":"2024-03-29T09:09:01.527540+0000","last_clean":"2024-03-29T09:09:01.527540+0000","last_became_active":"2024-03-29T09:08:57.415442+0000","last_became_peered":"2024-03-29T09:08:57.415442+0000","last_unstale":"2024-03-29T09:09:01.527540+0000","last_undegraded":"2024-03-29T09:09:01.527540+0000","last_fullsized":"2024-03-29T09:09:01.527540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:44.638689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531018+0000","last_change":"2024-03-29T09:09:01.444447+0000","last_active":"2024-03-29T09:09:01.531018+0000","last_peered":"2024-03-29T09:09:01.531018+0000","last_clean":"2024-03-29T09:09:01.531018+0000","last_became_active":"2024-03-29T09:09:01.443976+0000","last_became_peered":"2024-03-29T09:09:01.443976+0000","last_unstale":"2024-03-29T09:09:01.531018+0000","last_undegraded":"2024-03-29T09:09:01.531018+0000","last_fullsized":"2024-03-29T09:09:01.531018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422915+0000","last_change":"2024-03-29T09:08:56.904777+0000","last_active":"2024-03-29T09:09:01.422915+0000","last_peered":"2024-03-29T09:09:01.422915+0000","last_clean":"2024-03-29T09:09:01.422915+0000","last_became_active":"2024-03-29T09:08:56.904503+0000","last_became_peered":"2024-03-29T09:08:56.904503+0000","last_unstale":"2024-03-29T09:09:01.422915+0000","last_undegraded":"2024-03-29T09:09:01.422915+0000","last_fullsized":"2024-03-29T09:09:01.422915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527647+0000","last_change":"2024-03-29T09:08:57.409679+0000","last_active":"2024-03-29T09:09:01.527647+0000","last_peered":"2024-03-29T09:09:01.527647+0000","last_clean":"2024-03-29T09:09:01.527647+0000","last_became_active":"2024-03-29T09:08:57.409346+0000","last_became_peered":"2024-03-29T09:08:57.409346+0000","last_unstale":"2024-03-29T09:09:01.527647+0000","last_undegraded":"2024-03-29T09:09:01.527647+0000","last_fullsized":"2024-03-29T09:09:01.527647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527392+0000","last_change":"2024-03-29T09:09:01.451808+0000","last_active":"2024-03-29T09:09:01.527392+0000","last_peered":"2024-03-29T09:09:01.527392+0000","last_clean":"2024-03-29T09:09:01.527392+0000","last_became_active":"2024-03-29T09:09:01.451262+0000","last_became_peered":"2024-03-29T09:09:01.451262+0000","last_unstale":"2024-03-29T09:09:01.527392+0000","last_undegraded":"2024-03-29T09:09:01.527392+0000","last_fullsized":"2024-03-29T09:09:01.527392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:25.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422122+0000","last_change":"2024-03-29T09:08:56.912120+0000","last_active":"2024-03-29T09:09:01.422122+0000","last_peered":"2024-03-29T09:09:01.422122+0000","last_clean":"2024-03-29T09:09:01.422122+0000","last_became_active":"2024-03-29T09:08:56.911834+0000","last_became_peered":"2024-03-29T09:08:56.911834+0000","last_unstale":"2024-03-29T09:09:01.422122+0000","last_undegraded":"2024-03-29T09:09:01.422122+0000","last_fullsized":"2024-03-29T09:09:01.422122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527406+0000","last_change":"2024-03-29T09:08:57.413619+0000","last_active":"2024-03-29T09:09:01.527406+0000","last_peered":"2024-03-29T09:09:01.527406+0000","last_clean":"2024-03-29T09:09:01.527406+0000","last_became_active":"2024-03-29T09:08:57.409612+0000","last_became_peered":"2024-03-29T09:08:57.409612+0000","last_unstale":"2024-03-29T09:09:01.527406+0000","last_undegraded":"2024-03-29T09:09:01.527406+0000","last_fullsized":"2024-03-29T09:09:01.527406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533989+0000","last_change":"2024-03-29T09:09:01.436295+0000","last_active":"2024-03-29T09:09:01.533989+0000","last_peered":"2024-03-29T09:09:01.533989+0000","last_clean":"2024-03-29T09:09:01.533989+0000","last_became_active":"2024-03-29T09:09:01.435972+0000","last_became_peered":"2024-03-29T09:09:01.435972+0000","last_unstale":"2024-03-29T09:09:01.533989+0000","last_undegraded":"2024-03-29T09:09:01.533989+0000","last_fullsized":"2024-03-29T09:09:01.533989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429481+0000","last_change":"2024-03-29T09:08:56.907160+0000","last_active":"2024-03-29T09:09:01.429481+0000","last_peered":"2024-03-29T09:09:01.429481+0000","last_clean":"2024-03-29T09:09:01.429481+0000","last_became_active":"2024-03-29T09:08:56.906801+0000","last_became_peered":"2024-03-29T09:08:56.906801+0000","last_unstale":"2024-03-29T09:09:01.429481+0000","last_undegraded":"2024-03-29T09:09:01.429481+0000","last_fullsized":"2024-03-29T09:09:01.429481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527323+0000","last_change":"2024-03-29T09:08:57.425389+0000","last_active":"2024-03-29T09:09:01.527323+0000","last_peered":"2024-03-29T09:09:01.527323+0000","last_clean":"2024-03-29T09:09:01.527323+0000","last_became_active":"2024-03-29T09:08:57.425127+0000","last_became_peered":"2024-03-29T09:08:57.425127+0000","last_unstale":"2024-03-29T09:09:01.527323+0000","last_undegraded":"2024-03-29T09:09:01.527323+0000","last_fullsized":"2024-03-29T09:09:01.527323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527793+0000","last_change":"2024-03-29T09:09:01.432980+0000","last_active":"2024-03-29T09:09:01.527793+0000","last_peered":"2024-03-29T09:09:01.527793+0000","last_clean":"2024-03-29T09:09:01.527793+0000","last_became_active":"2024-03-29T09:09:01.432597+0000","last_became_peered":"2024-03-29T09:09:01.432597+0000","last_unstale":"2024-03-29T09:09:01.527793+0000","last_undegraded":"2024-03-29T09:09:01.527793+0000","last_fullsized":"2024-03-29T09:09:01.527793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:52.143266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414584+0000","last_change":"2024-03-29T09:08:56.901765+0000","last_active":"2024-03-29T09:09:01.414584+0000","last_peered":"2024-03-29T09:09:01.414584+0000","last_clean":"2024-03-29T09:09:01.414584+0000","last_became_active":"2024-03-29T09:08:56.901513+0000","last_became_peered":"2024-03-29T09:08:56.901513+0000","last_unstale":"2024-03-29T09:09:01.414584+0000","last_undegraded":"2024-03-29T09:09:01.414584+0000","last_fullsized":"2024-03-29T09:09:01.414584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:55.597965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527664+0000","last_change":"2024-03-29T09:08:57.406785+0000","last_active":"2024-03-29T09:09:01.527664+0000","last_peered":"2024-03-29T09:09:01.527664+0000","last_clean":"2024-03-29T09:09:01.527664+0000","last_became_active":"2024-03-29T09:08:57.406261+0000","last_became_peered":"2024-03-29T09:08:57.406261+0000","last_unstale":"2024-03-29T09:09:01.527664+0000","last_undegraded":"2024-03-29T09:09:01.527664+0000","last_fullsized":"2024-03-29T09:09:01.527664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:09:43.625475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530845+0000","last_change":"2024-03-29T09:09:01.445228+0000","last_active":"2024-03-29T09:09:01.530845+0000","last_peered":"2024-03-29T09:09:01.530845+0000","last_clean":"2024-03-29T09:09:01.530845+0000","last_became_active":"2024-03-29T09:09:01.444917+0000","last_became_peered":"2024-03-29T09:09:01.444917+0000","last_unstale":"2024-03-29T09:09:01.530845+0000","last_undegraded":"2024-03-29T09:09:01.530845+0000","last_fullsized":"2024-03-29T09:09:01.530845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.431916+0000","last_change":"2024-03-29T09:08:56.914015+0000","last_active":"2024-03-29T09:09:01.431916+0000","last_peered":"2024-03-29T09:09:01.431916+0000","last_clean":"2024-03-29T09:09:01.431916+0000","last_became_active":"2024-03-29T09:08:56.913673+0000","last_became_peered":"2024-03-29T09:08:56.913673+0000","last_unstale":"2024-03-29T09:09:01.431916+0000","last_undegraded":"2024-03-29T09:09:01.431916+0000","last_fullsized":"2024-03-29T09:09:01.431916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527811+0000","last_change":"2024-03-29T09:08:57.410999+0000","last_active":"2024-03-29T09:09:01.527811+0000","last_peered":"2024-03-29T09:09:01.527811+0000","last_clean":"2024-03-29T09:09:01.527811+0000","last_became_active":"2024-03-29T09:08:57.410742+0000","last_became_peered":"2024-03-29T09:08:57.410742+0000","last_unstale":"2024-03-29T09:09:01.527811+0000","last_undegraded":"2024-03-29T09:09:01.527811+0000","last_fullsized":"2024-03-29T09:09:01.527811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:24.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530767+0000","last_change":"2024-03-29T09:09:01.434584+0000","last_active":"2024-03-29T09:09:01.530767+0000","last_peered":"2024-03-29T09:09:01.530767+0000","last_clean":"2024-03-29T09:09:01.530767+0000","last_became_active":"2024-03-29T09:09:01.434233+0000","last_became_peered":"2024-03-29T09:09:01.434233+0000","last_unstale":"2024-03-29T09:09:01.530767+0000","last_undegraded":"2024-03-29T09:09:01.530767+0000","last_fullsized":"2024-03-29T09:09:01.530767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:18:14.395090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420352+0000","last_change":"2024-03-29T09:08:56.908753+0000","last_active":"2024-03-29T09:09:01.420352+0000","last_peered":"2024-03-29T09:09:01.420352+0000","last_clean":"2024-03-29T09:09:01.420352+0000","last_became_active":"2024-03-29T09:08:56.908505+0000","last_became_peered":"2024-03-29T09:08:56.908505+0000","last_unstale":"2024-03-29T09:09:01.420352+0000","last_undegraded":"2024-03-29T09:09:01.420352+0000","last_fullsized":"2024-03-29T09:09:01.420352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533999+0000","last_change":"2024-03-29T09:08:57.418006+0000","last_active":"2024-03-29T09:09:01.533999+0000","last_peered":"2024-03-29T09:09:01.533999+0000","last_clean":"2024-03-29T09:09:01.533999+0000","last_became_active":"2024-03-29T09:08:57.417644+0000","last_became_peered":"2024-03-29T09:08:57.417644+0000","last_unstale":"2024-03-29T09:09:01.533999+0000","last_undegraded":"2024-03-29T09:09:01.533999+0000","last_fullsized":"2024-03-29T09:09:01.533999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527566+0000","last_change":"2024-03-29T09:09:01.431602+0000","last_active":"2024-03-29T09:09:01.527566+0000","last_peered":"2024-03-29T09:09:01.527566+0000","last_clean":"2024-03-29T09:09:01.527566+0000","last_became_active":"2024-03-29T09:09:01.430422+0000","last_became_peered":"2024-03-29T09:09:01.430422+0000","last_unstale":"2024-03-29T09:09:01.527566+0000","last_undegraded":"2024-03-29T09:09:01.527566+0000","last_fullsized":"2024-03-29T09:09:01.527566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:21:33.289630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429605+0000","last_change":"2024-03-29T09:08:56.907602+0000","last_active":"2024-03-29T09:09:01.429605+0000","last_peered":"2024-03-29T09:09:01.429605+0000","last_clean":"2024-03-29T09:09:01.429605+0000","last_became_active":"2024-03-29T09:08:56.907321+0000","last_became_peered":"2024-03-29T09:08:56.907321+0000","last_unstale":"2024-03-29T09:09:01.429605+0000","last_undegraded":"2024-03-29T09:09:01.429605+0000","last_fullsized":"2024-03-29T09:09:01.429605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534106+0000","last_change":"2024-03-29T09:08:57.419001+0000","last_active":"2024-03-29T09:09:01.534106+0000","last_peered":"2024-03-29T09:09:01.534106+0000","last_clean":"2024-03-29T09:09:01.534106+0000","last_became_active":"2024-03-29T09:08:57.418769+0000","last_became_peered":"2024-03-29T09:08:57.418769+0000","last_unstale":"2024-03-29T09:09:01.534106+0000","last_undegraded":"2024-03-29T09:09:01.534106+0000","last_fullsized":"2024-03-29T09:09:01.534106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530742+0000","last_change":"2024-03-29T09:08:57.408417+0000","last_active":"2024-03-29T09:09:01.530742+0000","last_peered":"2024-03-29T09:09:01.530742+0000","last_clean":"2024-03-29T09:09:01.530742+0000","last_became_active":"2024-03-29T09:08:57.408131+0000","last_became_peered":"2024-03-29T09:08:57.408131+0000","last_unstale":"2024-03-29T09:09:01.530742+0000","last_undegraded":"2024-03-29T09:09:01.530742+0000","last_fullsized":"2024-03-29T09:09:01.530742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527306+0000","last_change":"2024-03-29T09:08:57.418220+0000","last_active":"2024-03-29T09:09:01.527306+0000","last_peered":"2024-03-29T09:09:01.527306+0000","last_clean":"2024-03-29T09:09:01.527306+0000","last_became_active":"2024-03-29T09:08:57.417900+0000","last_became_peered":"2024-03-29T09:08:57.417900+0000","last_unstale":"2024-03-29T09:09:01.527306+0000","last_undegraded":"2024-03-29T09:09:01.527306+0000","last_fullsized":"2024-03-29T09:09:01.527306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527708+0000","last_change":"2024-03-29T09:08:57.424647+0000","last_active":"2024-03-29T09:09:01.527708+0000","last_peered":"2024-03-29T09:09:01.527708+0000","last_clean":"2024-03-29T09:09:01.527708+0000","last_became_active":"2024-03-29T09:08:57.424404+0000","last_became_peered":"2024-03-29T09:08:57.424404+0000","last_unstale":"2024-03-29T09:09:01.527708+0000","last_undegraded":"2024-03-29T09:09:01.527708+0000","last_fullsized":"2024-03-29T09:09:01.527708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:06:29.651626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530787+0000","last_change":"2024-03-29T09:08:57.419276+0000","last_active":"2024-03-29T09:09:01.530787+0000","last_peered":"2024-03-29T09:09:01.530787+0000","last_clean":"2024-03-29T09:09:01.530787+0000","last_became_active":"2024-03-29T09:08:57.418974+0000","last_became_peered":"2024-03-29T09:08:57.418974+0000","last_unstale":"2024-03-29T09:09:01.530787+0000","last_undegraded":"2024-03-29T09:09:01.530787+0000","last_fullsized":"2024-03-29T09:09:01.530787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525521+0000","last_change":"2024-03-29T09:08:57.429156+0000","last_active":"2024-03-29T09:09:01.525521+0000","last_peered":"2024-03-29T09:09:01.525521+0000","last_clean":"2024-03-29T09:09:01.525521+0000","last_became_active":"2024-03-29T09:08:57.428873+0000","last_became_peered":"2024-03-29T09:08:57.428873+0000","last_unstale":"2024-03-29T09:09:01.525521+0000","last_undegraded":"2024-03-29T09:09:01.525521+0000","last_fullsized":"2024-03-29T09:09:01.525521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525556+0000","last_change":"2024-03-29T09:08:57.423650+0000","last_active":"2024-03-29T09:09:01.525556+0000","last_peered":"2024-03-29T09:09:01.525556+0000","last_clean":"2024-03-29T09:09:01.525556+0000","last_became_active":"2024-03-29T09:08:57.423385+0000","last_became_peered":"2024-03-29T09:08:57.423385+0000","last_unstale":"2024-03-29T09:09:01.525556+0000","last_undegraded":"2024-03-29T09:09:01.525556+0000","last_fullsized":"2024-03-29T09:09:01.525556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531076+0000","last_change":"2024-03-29T09:08:57.413866+0000","last_active":"2024-03-29T09:09:01.531076+0000","last_peered":"2024-03-29T09:09:01.531076+0000","last_clean":"2024-03-29T09:09:01.531076+0000","last_became_active":"2024-03-29T09:08:57.413483+0000","last_became_peered":"2024-03-29T09:08:57.413483+0000","last_unstale":"2024-03-29T09:09:01.531076+0000","last_undegraded":"2024-03-29T09:09:01.531076+0000","last_fullsized":"2024-03-29T09:09:01.531076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530897+0000","last_change":"2024-03-29T09:08:57.422855+0000","last_active":"2024-03-29T09:09:01.530897+0000","last_peered":"2024-03-29T09:09:01.530897+0000","last_clean":"2024-03-29T09:09:01.530897+0000","last_became_active":"2024-03-29T09:08:57.422350+0000","last_became_peered":"2024-03-29T09:08:57.422350+0000","last_unstale":"2024-03-29T09:09:01.530897+0000","last_undegraded":"2024-03-29T09:09:01.530897+0000","last_fullsized":"2024-03-29T09:09:01.530897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527637+0000","last_change":"2024-03-29T09:08:57.427109+0000","last_active":"2024-03-29T09:09:01.527637+0000","last_peered":"2024-03-29T09:09:01.527637+0000","last_clean":"2024-03-29T09:09:01.527637+0000","last_became_active":"2024-03-29T09:08:57.424525+0000","last_became_peered":"2024-03-29T09:08:57.424525+0000","last_unstale":"2024-03-29T09:09:01.527637+0000","last_undegraded":"2024-03-29T09:09:01.527637+0000","last_fullsized":"2024-03-29T09:09:01.527637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:52.774904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530755+0000","last_change":"2024-03-29T09:08:57.416922+0000","last_active":"2024-03-29T09:09:01.530755+0000","last_peered":"2024-03-29T09:09:01.530755+0000","last_clean":"2024-03-29T09:09:01.530755+0000","last_became_active":"2024-03-29T09:08:57.415900+0000","last_became_peered":"2024-03-29T09:08:57.415900+0000","last_unstale":"2024-03-29T09:09:01.530755+0000","last_undegraded":"2024-03-29T09:09:01.530755+0000","last_fullsized":"2024-03-29T09:09:01.530755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528724+0000","last_change":"2024-03-29T09:08:57.416177+0000","last_active":"2024-03-29T09:09:01.528724+0000","last_peered":"2024-03-29T09:09:01.528724+0000","last_clean":"2024-03-29T09:09:01.528724+0000","last_became_active":"2024-03-29T09:08:57.415844+0000","last_became_peered":"2024-03-29T09:08:57.415844+0000","last_unstale":"2024-03-29T09:09:01.528724+0000","last_undegraded":"2024-03-29T09:09:01.528724+0000","last_fullsized":"2024-03-29T09:09:01.528724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527479+0000","last_change":"2024-03-29T09:08:57.411579+0000","last_active":"2024-03-29T09:09:01.527479+0000","last_peered":"2024-03-29T09:09:01.527479+0000","last_clean":"2024-03-29T09:09:01.527479+0000","last_became_active":"2024-03-29T09:08:57.411297+0000","last_became_peered":"2024-03-29T09:08:57.411297+0000","last_unstale":"2024-03-29T09:09:01.527479+0000","last_undegraded":"2024-03-29T09:09:01.527479+0000","last_fullsized":"2024-03-29T09:09:01.527479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:13.990570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534013+0000","last_change":"2024-03-29T09:08:57.417300+0000","last_active":"2024-03-29T09:09:01.534013+0000","last_peered":"2024-03-29T09:09:01.534013+0000","last_clean":"2024-03-29T09:09:01.534013+0000","last_became_active":"2024-03-29T09:08:57.416733+0000","last_became_peered":"2024-03-29T09:08:57.416733+0000","last_unstale":"2024-03-29T09:09:01.534013+0000","last_undegraded":"2024-03-29T09:09:01.534013+0000","last_fullsized":"2024-03-29T09:09:01.534013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:51.863125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":690226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:09:26.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:09:26.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.529+0000 7f35a4400700 1 Processor -- start 2024-03-29T09:09:26.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.530+0000 7f35a4400700 1 -- start start 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a4400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f359c160cc0 0x7f359c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a4400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0cddc0 con 0x7f359c0d1b20 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a4400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0ce0a0 con 0x7f359c1608e0 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a4400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c0ce380 con 0x7f359c160cc0 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a2400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f359c160cc0 0x7f359c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a1000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f359c0d1b20 0x7f359c0afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:50428/0 (socket says 172.21.15.99:50428) 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a2400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f359c160cc0 0x7f359c0d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48400/0 (socket says 172.21.15.99:48400) 2024-03-29T09:09:26.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.531+0000 7f35a1000700 1 -- 172.21.15.99:0/784864350 learned_addr learned my addr 172.21.15.99:0/784864350 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.532+0000 7f35a2400700 1 -- 172.21.15.99:0/784864350 >> v1:172.21.15.133:6789/0 conn(0x7f359c1608e0 legacy=0x7f359c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.532+0000 7f35a2400700 1 -- 172.21.15.99:0/784864350 >> v1:172.21.15.99:6789/0 conn(0x7f359c0d1b20 legacy=0x7f359c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.532+0000 7f35a2400700 1 -- 172.21.15.99:0/784864350 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0ce660 con 0x7f359c160cc0 2024-03-29T09:09:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.532+0000 7f35a2400700 1 --2- 172.21.15.99:0/784864350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c160cc0 0x7f359c0d0150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3594009d90 tx=0x7f359400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18da4cb3e45aa32d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.532+0000 7f359be00700 1 -- 172.21.15.99:0/784864350 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359400e070 con 0x7f359c160cc0 2024-03-29T09:09:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.532+0000 7f359be00700 1 -- 172.21.15.99:0/784864350 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359400bbd0 con 0x7f359c160cc0 2024-03-29T09:09:26.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.532+0000 7f359be00700 1 -- 172.21.15.99:0/784864350 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3594012430 con 0x7f359c160cc0 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.533+0000 7f35a4400700 1 -- 172.21.15.99:0/784864350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c160cc0 msgr2=0x7f359c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.533+0000 7f35a4400700 1 --2- 172.21.15.99:0/784864350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c160cc0 0x7f359c0d0150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3594009d90 tx=0x7f359400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.533+0000 7f35a4400700 1 -- 172.21.15.99:0/784864350 shutdown_connections 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.533+0000 7f35a4400700 1 --2- 172.21.15.99:0/784864350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c160cc0 0x7f359c0d0150 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.533+0000 7f35a4400700 1 -- 172.21.15.99:0/784864350 >> 172.21.15.99:0/784864350 conn(0x7f359c09c490 msgr2=0x7f359c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.533+0000 7f35a4400700 1 -- 172.21.15.99:0/784864350 shutdown_connections 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.533+0000 7f35a4400700 1 -- 172.21.15.99:0/784864350 wait complete. 2024-03-29T09:09:26.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.535+0000 7f35a4400700 1 Processor -- start 2024-03-29T09:09:26.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.535+0000 7f35a4400700 1 -- start start 2024-03-29T09:09:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a4400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c0d1b20 0x7f359c0c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a4400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1608e0 0x7f359c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a4400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f359c160cc0 0x7f359c0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a4400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0b6370 con 0x7f359c0d1b20 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a4400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c14cda0 con 0x7f359c1608e0 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a4400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c09d850 con 0x7f359c160cc0 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c0d1b20 0x7f359c0c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c0d1b20 0x7f359c0c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48406/0 (socket says 172.21.15.99:48406) 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 -- 172.21.15.99:0/1076510620 learned_addr learned my addr 172.21.15.99:0/1076510620 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1000700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1608e0 0x7f359c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a2400700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f359c160cc0 0x7f359c0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 -- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f359c160cc0 msgr2=0x7f359c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f359c160cc0 0x7f359c0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 -- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1608e0 msgr2=0x7f359c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1608e0 0x7f359c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.536+0000 7f35a1a00700 1 -- 172.21.15.99:0/1076510620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594009a70 con 0x7f359c0d1b20 2024-03-29T09:09:26.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.537+0000 7f35a1a00700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c0d1b20 0x7f359c0c9b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f359000c4e0 tx=0x7f359000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.537+0000 7f3599600700 1 -- 172.21.15.99:0/1076510620 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900052d0 con 0x7f359c0d1b20 2024-03-29T09:09:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.537+0000 7f3599600700 1 -- 172.21.15.99:0/1076510620 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3590005430 con 0x7f359c0d1b20 2024-03-29T09:09:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.537+0000 7f3599600700 1 -- 172.21.15.99:0/1076510620 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900038c0 con 0x7f359c0d1b20 2024-03-29T09:09:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.537+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c09db30 con 0x7f359c0d1b20 2024-03-29T09:09:26.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.537+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c14cff0 con 0x7f359c0d1b20 2024-03-29T09:09:26.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.538+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c0677a0 con 0x7f359c0d1b20 2024-03-29T09:09:26.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.539+0000 7f3599600700 1 -- 172.21.15.99:0/1076510620 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3590003a20 con 0x7f359c0d1b20 2024-03-29T09:09:26.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.540+0000 7f3599600700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f357c06ac90 0x7f357c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:26.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.540+0000 7f3599600700 1 -- 172.21.15.99:0/1076510620 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3590088ae0 con 0x7f359c0d1b20 2024-03-29T09:09:26.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.540+0000 7f35a1000700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f357c06ac90 0x7f357c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:26.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.541+0000 7f35a1000700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f357c06ac90 0x7f357c06d150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f359c1f72a0 tx=0x7f3588009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:26.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.545+0000 7f3599600700 1 -- 172.21.15.99:0/1076510620 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f35900539c0 con 0x7f359c0d1b20 2024-03-29T09:09:26.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.748+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f359c0b54b0 con 0x7f357c06ac90 2024-03-29T09:09:26.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.757+0000 7f3599600700 1 -- 172.21.15.99:0/1076510620 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345285 (secure 0 0 0) 0x7f359c0b54b0 con 0x7f357c06ac90 2024-03-29T09:09:26.762 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:26.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f357c06ac90 msgr2=0x7f357c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f357c06ac90 0x7f357c06d150 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f359c1f72a0 tx=0x7f3588009040 comp rx=0 tx=0).stop 2024-03-29T09:09:26.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c0d1b20 msgr2=0x7f359c0c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:26.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c0d1b20 0x7f359c0c9b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f359000c4e0 tx=0x7f359000c8a0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 shutdown_connections 2024-03-29T09:09:26.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f357c06ac90 0x7f357c06d150 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f359c160cc0 0x7f359c0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f359c0d1b20 0x7f359c0c9b20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 --2- 172.21.15.99:0/1076510620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1608e0 0x7f359c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:26.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 >> 172.21.15.99:0/1076510620 conn(0x7f359c09c490 msgr2=0x7f359c0cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:26.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 shutdown_connections 2024-03-29T09:09:26.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:26.762+0000 7f35a4400700 1 -- 172.21.15.99:0/1076510620 wait complete. 2024-03-29T09:09:26.766 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:09:26.781 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-29T09:09:26.455506+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219392,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755328,"statfs":{"total":773094113280,"available":772869455872,"internally_reserved":0,"allocated":3284992,"data_stored":1988228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002813"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534081+0000","last_change":"2024-03-29T09:08:57.419769+0000","last_active":"2024-03-29T09:09:01.534081+0000","last_peered":"2024-03-29T09:09:01.534081+0000","last_clean":"2024-03-29T09:09:01.534081+0000","last_became_active":"2024-03-29T09:08:57.419455+0000","last_became_peered":"2024-03-29T09:08:57.419455+0000","last_unstale":"2024-03-29T09:09:01.534081+0000","last_undegraded":"2024-03-29T09:09:01.534081+0000","last_fullsized":"2024-03-29T09:09:01.534081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530834+0000","last_change":"2024-03-29T09:08:57.427159+0000","last_active":"2024-03-29T09:09:01.530834+0000","last_peered":"2024-03-29T09:09:01.530834+0000","last_clean":"2024-03-29T09:09:01.530834+0000","last_became_active":"2024-03-29T09:08:57.426926+0000","last_became_peered":"2024-03-29T09:08:57.426926+0000","last_unstale":"2024-03-29T09:09:01.530834+0000","last_undegraded":"2024-03-29T09:09:01.530834+0000","last_fullsized":"2024-03-29T09:09:01.530834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:08:57.419169+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:08:57.418824+0000","last_became_peered":"2024-03-29T09:08:57.418824+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530961+0000","last_change":"2024-03-29T09:08:57.409236+0000","last_active":"2024-03-29T09:09:01.530961+0000","last_peered":"2024-03-29T09:09:01.530961+0000","last_clean":"2024-03-29T09:09:01.530961+0000","last_became_active":"2024-03-29T09:08:57.408981+0000","last_became_peered":"2024-03-29T09:08:57.408981+0000","last_unstale":"2024-03-29T09:09:01.530961+0000","last_undegraded":"2024-03-29T09:09:01.530961+0000","last_fullsized":"2024-03-29T09:09:01.530961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533948+0000","last_change":"2024-03-29T09:08:57.423360+0000","last_active":"2024-03-29T09:09:01.533948+0000","last_peered":"2024-03-29T09:09:01.533948+0000","last_clean":"2024-03-29T09:09:01.533948+0000","last_became_active":"2024-03-29T09:08:57.423088+0000","last_became_peered":"2024-03-29T09:08:57.423088+0000","last_unstale":"2024-03-29T09:09:01.533948+0000","last_undegraded":"2024-03-29T09:09:01.533948+0000","last_fullsized":"2024-03-29T09:09:01.533948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530887+0000","last_change":"2024-03-29T09:08:57.415603+0000","last_active":"2024-03-29T09:09:01.530887+0000","last_peered":"2024-03-29T09:09:01.530887+0000","last_clean":"2024-03-29T09:09:01.530887+0000","last_became_active":"2024-03-29T09:08:57.413227+0000","last_became_peered":"2024-03-29T09:08:57.413227+0000","last_unstale":"2024-03-29T09:09:01.530887+0000","last_undegraded":"2024-03-29T09:09:01.530887+0000","last_fullsized":"2024-03-29T09:09:01.530887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533929+0000","last_change":"2024-03-29T09:08:57.421465+0000","last_active":"2024-03-29T09:09:01.533929+0000","last_peered":"2024-03-29T09:09:01.533929+0000","last_clean":"2024-03-29T09:09:01.533929+0000","last_became_active":"2024-03-29T09:08:57.421189+0000","last_became_peered":"2024-03-29T09:08:57.421189+0000","last_unstale":"2024-03-29T09:09:01.533929+0000","last_undegraded":"2024-03-29T09:09:01.533929+0000","last_fullsized":"2024-03-29T09:09:01.533929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530711+0000","last_change":"2024-03-29T09:08:57.430900+0000","last_active":"2024-03-29T09:09:01.530711+0000","last_peered":"2024-03-29T09:09:01.530711+0000","last_clean":"2024-03-29T09:09:01.530711+0000","last_became_active":"2024-03-29T09:08:57.430636+0000","last_became_peered":"2024-03-29T09:08:57.430636+0000","last_unstale":"2024-03-29T09:09:01.530711+0000","last_undegraded":"2024-03-29T09:09:01.530711+0000","last_fullsized":"2024-03-29T09:09:01.530711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528748+0000","last_change":"2024-03-29T09:08:57.413603+0000","last_active":"2024-03-29T09:09:01.528748+0000","last_peered":"2024-03-29T09:09:01.528748+0000","last_clean":"2024-03-29T09:09:01.528748+0000","last_became_active":"2024-03-29T09:08:57.413373+0000","last_became_peered":"2024-03-29T09:08:57.413373+0000","last_unstale":"2024-03-29T09:09:01.528748+0000","last_undegraded":"2024-03-29T09:09:01.528748+0000","last_fullsized":"2024-03-29T09:09:01.528748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534071+0000","last_change":"2024-03-29T09:08:57.417036+0000","last_active":"2024-03-29T09:09:01.534071+0000","last_peered":"2024-03-29T09:09:01.534071+0000","last_clean":"2024-03-29T09:09:01.534071+0000","last_became_active":"2024-03-29T09:08:57.416762+0000","last_became_peered":"2024-03-29T09:08:57.416762+0000","last_unstale":"2024-03-29T09:09:01.534071+0000","last_undegraded":"2024-03-29T09:09:01.534071+0000","last_fullsized":"2024-03-29T09:09:01.534071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534055+0000","last_change":"2024-03-29T09:08:57.417500+0000","last_active":"2024-03-29T09:09:01.534055+0000","last_peered":"2024-03-29T09:09:01.534055+0000","last_clean":"2024-03-29T09:09:01.534055+0000","last_became_active":"2024-03-29T09:08:57.417187+0000","last_became_peered":"2024-03-29T09:08:57.417187+0000","last_unstale":"2024-03-29T09:09:01.534055+0000","last_undegraded":"2024-03-29T09:09:01.534055+0000","last_fullsized":"2024-03-29T09:09:01.534055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534022+0000","last_change":"2024-03-29T09:08:57.416133+0000","last_active":"2024-03-29T09:09:01.534022+0000","last_peered":"2024-03-29T09:09:01.534022+0000","last_clean":"2024-03-29T09:09:01.534022+0000","last_became_active":"2024-03-29T09:08:57.415878+0000","last_became_peered":"2024-03-29T09:08:57.415878+0000","last_unstale":"2024-03-29T09:09:01.534022+0000","last_undegraded":"2024-03-29T09:09:01.534022+0000","last_fullsized":"2024-03-29T09:09:01.534022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525479+0000","last_change":"2024-03-29T09:08:57.418824+0000","last_active":"2024-03-29T09:09:01.525479+0000","last_peered":"2024-03-29T09:09:01.525479+0000","last_clean":"2024-03-29T09:09:01.525479+0000","last_became_active":"2024-03-29T09:08:57.418534+0000","last_became_peered":"2024-03-29T09:08:57.418534+0000","last_unstale":"2024-03-29T09:09:01.525479+0000","last_undegraded":"2024-03-29T09:09:01.525479+0000","last_fullsized":"2024-03-29T09:09:01.525479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:17:11.207261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525458+0000","last_change":"2024-03-29T09:08:57.412768+0000","last_active":"2024-03-29T09:09:01.525458+0000","last_peered":"2024-03-29T09:09:01.525458+0000","last_clean":"2024-03-29T09:09:01.525458+0000","last_became_active":"2024-03-29T09:08:57.412077+0000","last_became_peered":"2024-03-29T09:08:57.412077+0000","last_unstale":"2024-03-29T09:09:01.525458+0000","last_undegraded":"2024-03-29T09:09:01.525458+0000","last_fullsized":"2024-03-29T09:09:01.525458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530745+0000","last_change":"2024-03-29T09:08:57.430532+0000","last_active":"2024-03-29T09:09:01.530745+0000","last_peered":"2024-03-29T09:09:01.530745+0000","last_clean":"2024-03-29T09:09:01.530745+0000","last_became_active":"2024-03-29T09:08:57.430163+0000","last_became_peered":"2024-03-29T09:08:57.430163+0000","last_unstale":"2024-03-29T09:09:01.530745+0000","last_undegraded":"2024-03-29T09:09:01.530745+0000","last_fullsized":"2024-03-29T09:09:01.530745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527616+0000","last_change":"2024-03-29T09:08:57.413899+0000","last_active":"2024-03-29T09:09:01.527616+0000","last_peered":"2024-03-29T09:09:01.527616+0000","last_clean":"2024-03-29T09:09:01.527616+0000","last_became_active":"2024-03-29T09:08:57.413637+0000","last_became_peered":"2024-03-29T09:08:57.413637+0000","last_unstale":"2024-03-29T09:09:01.527616+0000","last_undegraded":"2024-03-29T09:09:01.527616+0000","last_fullsized":"2024-03-29T09:09:01.527616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:32.639943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527925+0000","last_change":"2024-03-29T09:08:57.418000+0000","last_active":"2024-03-29T09:09:01.527925+0000","last_peered":"2024-03-29T09:09:01.527925+0000","last_clean":"2024-03-29T09:09:01.527925+0000","last_became_active":"2024-03-29T09:08:57.417649+0000","last_became_peered":"2024-03-29T09:08:57.417649+0000","last_unstale":"2024-03-29T09:09:01.527925+0000","last_undegraded":"2024-03-29T09:09:01.527925+0000","last_fullsized":"2024-03-29T09:09:01.527925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:06.479973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527377+0000","last_change":"2024-03-29T09:08:57.430547+0000","last_active":"2024-03-29T09:09:01.527377+0000","last_peered":"2024-03-29T09:09:01.527377+0000","last_clean":"2024-03-29T09:09:01.527377+0000","last_became_active":"2024-03-29T09:08:57.430013+0000","last_became_peered":"2024-03-29T09:08:57.430013+0000","last_unstale":"2024-03-29T09:09:01.527377+0000","last_undegraded":"2024-03-29T09:09:01.527377+0000","last_fullsized":"2024-03-29T09:09:01.527377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527878+0000","last_change":"2024-03-29T09:08:57.419008+0000","last_active":"2024-03-29T09:09:01.527878+0000","last_peered":"2024-03-29T09:09:01.527878+0000","last_clean":"2024-03-29T09:09:01.527878+0000","last_became_active":"2024-03-29T09:08:57.418723+0000","last_became_peered":"2024-03-29T09:08:57.418723+0000","last_unstale":"2024-03-29T09:09:01.527878+0000","last_undegraded":"2024-03-29T09:09:01.527878+0000","last_fullsized":"2024-03-29T09:09:01.527878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:06.268267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527289+0000","last_change":"2024-03-29T09:09:01.448984+0000","last_active":"2024-03-29T09:09:01.527289+0000","last_peered":"2024-03-29T09:09:01.527289+0000","last_clean":"2024-03-29T09:09:01.527289+0000","last_became_active":"2024-03-29T09:09:01.446825+0000","last_became_peered":"2024-03-29T09:09:01.446825+0000","last_unstale":"2024-03-29T09:09:01.527289+0000","last_undegraded":"2024-03-29T09:09:01.527289+0000","last_fullsized":"2024-03-29T09:09:01.527289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:55.385731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.425281+0000","last_change":"2024-03-29T09:08:56.898980+0000","last_active":"2024-03-29T09:09:01.425281+0000","last_peered":"2024-03-29T09:09:01.425281+0000","last_clean":"2024-03-29T09:09:01.425281+0000","last_became_active":"2024-03-29T09:08:56.898646+0000","last_became_peered":"2024-03-29T09:08:56.898646+0000","last_unstale":"2024-03-29T09:09:01.425281+0000","last_undegraded":"2024-03-29T09:09:01.425281+0000","last_fullsized":"2024-03-29T09:09:01.425281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530854+0000","last_change":"2024-03-29T09:08:57.420574+0000","last_active":"2024-03-29T09:09:01.530854+0000","last_peered":"2024-03-29T09:09:01.530854+0000","last_clean":"2024-03-29T09:09:01.530854+0000","last_became_active":"2024-03-29T09:08:57.420318+0000","last_became_peered":"2024-03-29T09:08:57.420318+0000","last_unstale":"2024-03-29T09:09:01.530854+0000","last_undegraded":"2024-03-29T09:09:01.530854+0000","last_fullsized":"2024-03-29T09:09:01.530854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530783+0000","last_change":"2024-03-29T09:09:01.444440+0000","last_active":"2024-03-29T09:09:01.530783+0000","last_peered":"2024-03-29T09:09:01.530783+0000","last_clean":"2024-03-29T09:09:01.530783+0000","last_became_active":"2024-03-29T09:09:01.444026+0000","last_became_peered":"2024-03-29T09:09:01.444026+0000","last_unstale":"2024-03-29T09:09:01.530783+0000","last_undegraded":"2024-03-29T09:09:01.530783+0000","last_fullsized":"2024-03-29T09:09:01.530783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.421739+0000","last_change":"2024-03-29T09:08:56.908349+0000","last_active":"2024-03-29T09:09:01.421739+0000","last_peered":"2024-03-29T09:09:01.421739+0000","last_clean":"2024-03-29T09:09:01.421739+0000","last_became_active":"2024-03-29T09:08:56.908070+0000","last_became_peered":"2024-03-29T09:08:56.908070+0000","last_unstale":"2024-03-29T09:09:01.421739+0000","last_undegraded":"2024-03-29T09:09:01.421739+0000","last_fullsized":"2024-03-29T09:09:01.421739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530920+0000","last_change":"2024-03-29T09:08:57.422633+0000","last_active":"2024-03-29T09:09:01.530920+0000","last_peered":"2024-03-29T09:09:01.530920+0000","last_clean":"2024-03-29T09:09:01.530920+0000","last_became_active":"2024-03-29T09:08:57.422298+0000","last_became_peered":"2024-03-29T09:08:57.422298+0000","last_unstale":"2024-03-29T09:09:01.530920+0000","last_undegraded":"2024-03-29T09:09:01.530920+0000","last_fullsized":"2024-03-29T09:09:01.530920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530925+0000","last_change":"2024-03-29T09:09:01.435391+0000","last_active":"2024-03-29T09:09:01.530925+0000","last_peered":"2024-03-29T09:09:01.530925+0000","last_clean":"2024-03-29T09:09:01.530925+0000","last_became_active":"2024-03-29T09:09:01.435087+0000","last_became_peered":"2024-03-29T09:09:01.435087+0000","last_unstale":"2024-03-29T09:09:01.530925+0000","last_undegraded":"2024-03-29T09:09:01.530925+0000","last_fullsized":"2024-03-29T09:09:01.530925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424816+0000","last_change":"2024-03-29T09:08:56.910805+0000","last_active":"2024-03-29T09:09:01.424816+0000","last_peered":"2024-03-29T09:09:01.424816+0000","last_clean":"2024-03-29T09:09:01.424816+0000","last_became_active":"2024-03-29T09:08:56.908213+0000","last_became_peered":"2024-03-29T09:08:56.908213+0000","last_unstale":"2024-03-29T09:09:01.424816+0000","last_undegraded":"2024-03-29T09:09:01.424816+0000","last_fullsized":"2024-03-29T09:09:01.424816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528733+0000","last_change":"2024-03-29T09:08:57.420307+0000","last_active":"2024-03-29T09:09:01.528733+0000","last_peered":"2024-03-29T09:09:01.528733+0000","last_clean":"2024-03-29T09:09:01.528733+0000","last_became_active":"2024-03-29T09:08:57.420033+0000","last_became_peered":"2024-03-29T09:08:57.420033+0000","last_unstale":"2024-03-29T09:09:01.528733+0000","last_undegraded":"2024-03-29T09:09:01.528733+0000","last_fullsized":"2024-03-29T09:09:01.528733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530866+0000","last_change":"2024-03-29T09:09:01.441108+0000","last_active":"2024-03-29T09:09:01.530866+0000","last_peered":"2024-03-29T09:09:01.530866+0000","last_clean":"2024-03-29T09:09:01.530866+0000","last_became_active":"2024-03-29T09:09:01.440455+0000","last_became_peered":"2024-03-29T09:09:01.440455+0000","last_unstale":"2024-03-29T09:09:01.530866+0000","last_undegraded":"2024-03-29T09:09:01.530866+0000","last_fullsized":"2024-03-29T09:09:01.530866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424962+0000","last_change":"2024-03-29T09:08:56.908246+0000","last_active":"2024-03-29T09:09:01.424962+0000","last_peered":"2024-03-29T09:09:01.424962+0000","last_clean":"2024-03-29T09:09:01.424962+0000","last_became_active":"2024-03-29T09:08:56.907815+0000","last_became_peered":"2024-03-29T09:08:56.907815+0000","last_unstale":"2024-03-29T09:09:01.424962+0000","last_undegraded":"2024-03-29T09:09:01.424962+0000","last_fullsized":"2024-03-29T09:09:01.424962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527840+0000","last_change":"2024-03-29T09:08:57.420229+0000","last_active":"2024-03-29T09:09:01.527840+0000","last_peered":"2024-03-29T09:09:01.527840+0000","last_clean":"2024-03-29T09:09:01.527840+0000","last_became_active":"2024-03-29T09:08:57.419940+0000","last_became_peered":"2024-03-29T09:08:57.419940+0000","last_unstale":"2024-03-29T09:09:01.527840+0000","last_undegraded":"2024-03-29T09:09:01.527840+0000","last_fullsized":"2024-03-29T09:09:01.527840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:19.867399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527346+0000","last_change":"2024-03-29T09:09:01.447861+0000","last_active":"2024-03-29T09:09:01.527346+0000","last_peered":"2024-03-29T09:09:01.527346+0000","last_clean":"2024-03-29T09:09:01.527346+0000","last_became_active":"2024-03-29T09:09:01.447337+0000","last_became_peered":"2024-03-29T09:09:01.447337+0000","last_unstale":"2024-03-29T09:09:01.527346+0000","last_undegraded":"2024-03-29T09:09:01.527346+0000","last_fullsized":"2024-03-29T09:09:01.527346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:19.879877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419991+0000","last_change":"2024-03-29T09:08:56.908710+0000","last_active":"2024-03-29T09:09:01.419991+0000","last_peered":"2024-03-29T09:09:01.419991+0000","last_clean":"2024-03-29T09:09:01.419991+0000","last_became_active":"2024-03-29T09:08:56.906213+0000","last_became_peered":"2024-03-29T09:08:56.906213+0000","last_unstale":"2024-03-29T09:09:01.419991+0000","last_undegraded":"2024-03-29T09:09:01.419991+0000","last_fullsized":"2024-03-29T09:09:01.419991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528714+0000","last_change":"2024-03-29T09:08:57.423981+0000","last_active":"2024-03-29T09:09:01.528714+0000","last_peered":"2024-03-29T09:09:01.528714+0000","last_clean":"2024-03-29T09:09:01.528714+0000","last_became_active":"2024-03-29T09:08:57.423667+0000","last_became_peered":"2024-03-29T09:08:57.423667+0000","last_unstale":"2024-03-29T09:09:01.528714+0000","last_undegraded":"2024-03-29T09:09:01.528714+0000","last_fullsized":"2024-03-29T09:09:01.528714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527487+0000","last_change":"2024-03-29T09:09:01.455365+0000","last_active":"2024-03-29T09:09:01.527487+0000","last_peered":"2024-03-29T09:09:01.527487+0000","last_clean":"2024-03-29T09:09:01.527487+0000","last_became_active":"2024-03-29T09:09:01.453949+0000","last_became_peered":"2024-03-29T09:09:01.453949+0000","last_unstale":"2024-03-29T09:09:01.527487+0000","last_undegraded":"2024-03-29T09:09:01.527487+0000","last_fullsized":"2024-03-29T09:09:01.527487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:00.449212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414280+0000","last_change":"2024-03-29T09:08:56.912593+0000","last_active":"2024-03-29T09:09:01.414280+0000","last_peered":"2024-03-29T09:09:01.414280+0000","last_clean":"2024-03-29T09:09:01.414280+0000","last_became_active":"2024-03-29T09:08:56.912401+0000","last_became_peered":"2024-03-29T09:08:56.912401+0000","last_unstale":"2024-03-29T09:09:01.414280+0000","last_undegraded":"2024-03-29T09:09:01.414280+0000","last_fullsized":"2024-03-29T09:09:01.414280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:27.805156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527468+0000","last_change":"2024-03-29T09:08:57.421182+0000","last_active":"2024-03-29T09:09:01.527468+0000","last_peered":"2024-03-29T09:09:01.527468+0000","last_clean":"2024-03-29T09:09:01.527468+0000","last_became_active":"2024-03-29T09:08:57.420980+0000","last_became_peered":"2024-03-29T09:08:57.420980+0000","last_unstale":"2024-03-29T09:09:01.527468+0000","last_undegraded":"2024-03-29T09:09:01.527468+0000","last_fullsized":"2024-03-29T09:09:01.527468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527895+0000","last_change":"2024-03-29T09:09:01.433710+0000","last_active":"2024-03-29T09:09:01.527895+0000","last_peered":"2024-03-29T09:09:01.527895+0000","last_clean":"2024-03-29T09:09:01.527895+0000","last_became_active":"2024-03-29T09:09:01.433161+0000","last_became_peered":"2024-03-29T09:09:01.433161+0000","last_unstale":"2024-03-29T09:09:01.527895+0000","last_undegraded":"2024-03-29T09:09:01.527895+0000","last_fullsized":"2024-03-29T09:09:01.527895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:07:30.745379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429199+0000","last_change":"2024-03-29T09:08:56.911770+0000","last_active":"2024-03-29T09:09:01.429199+0000","last_peered":"2024-03-29T09:09:01.429199+0000","last_clean":"2024-03-29T09:09:01.429199+0000","last_became_active":"2024-03-29T09:08:56.911493+0000","last_became_peered":"2024-03-29T09:08:56.911493+0000","last_unstale":"2024-03-29T09:09:01.429199+0000","last_undegraded":"2024-03-29T09:09:01.429199+0000","last_fullsized":"2024-03-29T09:09:01.429199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530721+0000","last_change":"2024-03-29T09:08:57.426875+0000","last_active":"2024-03-29T09:09:01.530721+0000","last_peered":"2024-03-29T09:09:01.530721+0000","last_clean":"2024-03-29T09:09:01.530721+0000","last_became_active":"2024-03-29T09:08:57.426586+0000","last_became_peered":"2024-03-29T09:08:57.426586+0000","last_unstale":"2024-03-29T09:09:01.530721+0000","last_undegraded":"2024-03-29T09:09:01.530721+0000","last_fullsized":"2024-03-29T09:09:01.530721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527529+0000","last_change":"2024-03-29T09:09:01.440649+0000","last_active":"2024-03-29T09:09:01.527529+0000","last_peered":"2024-03-29T09:09:01.527529+0000","last_clean":"2024-03-29T09:09:01.527529+0000","last_became_active":"2024-03-29T09:09:01.440252+0000","last_became_peered":"2024-03-29T09:09:01.440252+0000","last_unstale":"2024-03-29T09:09:01.527529+0000","last_undegraded":"2024-03-29T09:09:01.527529+0000","last_fullsized":"2024-03-29T09:09:01.527529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414326+0000","last_change":"2024-03-29T09:08:56.908431+0000","last_active":"2024-03-29T09:09:01.414326+0000","last_peered":"2024-03-29T09:09:01.414326+0000","last_clean":"2024-03-29T09:09:01.414326+0000","last_became_active":"2024-03-29T09:08:56.906823+0000","last_became_peered":"2024-03-29T09:08:56.906823+0000","last_unstale":"2024-03-29T09:09:01.414326+0000","last_undegraded":"2024-03-29T09:09:01.414326+0000","last_fullsized":"2024-03-29T09:09:01.414326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:24.070569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527421+0000","last_change":"2024-03-29T09:08:57.410009+0000","last_active":"2024-03-29T09:09:01.527421+0000","last_peered":"2024-03-29T09:09:01.527421+0000","last_clean":"2024-03-29T09:09:01.527421+0000","last_became_active":"2024-03-29T09:08:57.409632+0000","last_became_peered":"2024-03-29T09:08:57.409632+0000","last_unstale":"2024-03-29T09:09:01.527421+0000","last_undegraded":"2024-03-29T09:09:01.527421+0000","last_fullsized":"2024-03-29T09:09:01.527421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527438+0000","last_change":"2024-03-29T09:09:01.447901+0000","last_active":"2024-03-29T09:09:01.527438+0000","last_peered":"2024-03-29T09:09:01.527438+0000","last_clean":"2024-03-29T09:09:01.527438+0000","last_became_active":"2024-03-29T09:09:01.447482+0000","last_became_peered":"2024-03-29T09:09:01.447482+0000","last_unstale":"2024-03-29T09:09:01.527438+0000","last_undegraded":"2024-03-29T09:09:01.527438+0000","last_fullsized":"2024-03-29T09:09:01.527438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.426470+0000","last_change":"2024-03-29T09:08:56.909506+0000","last_active":"2024-03-29T09:09:01.426470+0000","last_peered":"2024-03-29T09:09:01.426470+0000","last_clean":"2024-03-29T09:09:01.426470+0000","last_became_active":"2024-03-29T09:08:56.909183+0000","last_became_peered":"2024-03-29T09:08:56.909183+0000","last_unstale":"2024-03-29T09:09:01.426470+0000","last_undegraded":"2024-03-29T09:09:01.426470+0000","last_fullsized":"2024-03-29T09:09:01.426470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:04:15.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531038+0000","last_change":"2024-03-29T09:08:57.413715+0000","last_active":"2024-03-29T09:09:01.531038+0000","last_peered":"2024-03-29T09:09:01.531038+0000","last_clean":"2024-03-29T09:09:01.531038+0000","last_became_active":"2024-03-29T09:08:57.413145+0000","last_became_peered":"2024-03-29T09:08:57.413145+0000","last_unstale":"2024-03-29T09:09:01.531038+0000","last_undegraded":"2024-03-29T09:09:01.531038+0000","last_fullsized":"2024-03-29T09:09:01.531038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527274+0000","last_change":"2024-03-29T09:09:01.444258+0000","last_active":"2024-03-29T09:09:01.527274+0000","last_peered":"2024-03-29T09:09:01.527274+0000","last_clean":"2024-03-29T09:09:01.527274+0000","last_became_active":"2024-03-29T09:09:01.443829+0000","last_became_peered":"2024-03-29T09:09:01.443829+0000","last_unstale":"2024-03-29T09:09:01.527274+0000","last_undegraded":"2024-03-29T09:09:01.527274+0000","last_fullsized":"2024-03-29T09:09:01.527274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:29:01.967027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424436+0000","last_change":"2024-03-29T09:08:56.909671+0000","last_active":"2024-03-29T09:09:01.424436+0000","last_peered":"2024-03-29T09:09:01.424436+0000","last_clean":"2024-03-29T09:09:01.424436+0000","last_became_active":"2024-03-29T09:08:56.909426+0000","last_became_peered":"2024-03-29T09:08:56.909426+0000","last_unstale":"2024-03-29T09:09:01.424436+0000","last_undegraded":"2024-03-29T09:09:01.424436+0000","last_fullsized":"2024-03-29T09:09:01.424436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530931+0000","last_change":"2024-03-29T09:08:57.422965+0000","last_active":"2024-03-29T09:09:01.530931+0000","last_peered":"2024-03-29T09:09:01.530931+0000","last_clean":"2024-03-29T09:09:01.530931+0000","last_became_active":"2024-03-29T09:08:57.422689+0000","last_became_peered":"2024-03-29T09:08:57.422689+0000","last_unstale":"2024-03-29T09:09:01.530931+0000","last_undegraded":"2024-03-29T09:09:01.530931+0000","last_fullsized":"2024-03-29T09:09:01.530931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530907+0000","last_change":"2024-03-29T09:09:01.447399+0000","last_active":"2024-03-29T09:09:01.530907+0000","last_peered":"2024-03-29T09:09:01.530907+0000","last_clean":"2024-03-29T09:09:01.530907+0000","last_became_active":"2024-03-29T09:09:01.446880+0000","last_became_peered":"2024-03-29T09:09:01.446880+0000","last_unstale":"2024-03-29T09:09:01.530907+0000","last_undegraded":"2024-03-29T09:09:01.530907+0000","last_fullsized":"2024-03-29T09:09:01.530907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:49:27.133997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.433574+0000","last_change":"2024-03-29T09:08:56.908399+0000","last_active":"2024-03-29T09:09:01.433574+0000","last_peered":"2024-03-29T09:09:01.433574+0000","last_clean":"2024-03-29T09:09:01.433574+0000","last_became_active":"2024-03-29T09:08:56.906866+0000","last_became_peered":"2024-03-29T09:08:56.906866+0000","last_unstale":"2024-03-29T09:09:01.433574+0000","last_undegraded":"2024-03-29T09:09:01.433574+0000","last_fullsized":"2024-03-29T09:09:01.433574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527738+0000","last_change":"2024-03-29T09:08:57.423735+0000","last_active":"2024-03-29T09:09:01.527738+0000","last_peered":"2024-03-29T09:09:01.527738+0000","last_clean":"2024-03-29T09:09:01.527738+0000","last_became_active":"2024-03-29T09:08:57.423405+0000","last_became_peered":"2024-03-29T09:08:57.423405+0000","last_unstale":"2024-03-29T09:09:01.527738+0000","last_undegraded":"2024-03-29T09:09:01.527738+0000","last_fullsized":"2024-03-29T09:09:01.527738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527448+0000","last_change":"2024-03-29T09:09:01.442875+0000","last_active":"2024-03-29T09:09:01.527448+0000","last_peered":"2024-03-29T09:09:01.527448+0000","last_clean":"2024-03-29T09:09:01.527448+0000","last_became_active":"2024-03-29T09:09:01.442426+0000","last_became_peered":"2024-03-29T09:09:01.442426+0000","last_unstale":"2024-03-29T09:09:01.527448+0000","last_undegraded":"2024-03-29T09:09:01.527448+0000","last_fullsized":"2024-03-29T09:09:01.527448+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T09:09:15.577021+0000","last_change":"2024-03-29T09:08:56.895791+0000","last_active":"2024-03-29T09:09:15.577021+0000","last_peered":"2024-03-29T09:09:15.577021+0000","last_clean":"2024-03-29T09:09:15.577021+0000","last_became_active":"2024-03-29T09:08:56.894997+0000","last_became_peered":"2024-03-29T09:08:56.894997+0000","last_unstale":"2024-03-29T09:09:15.577021+0000","last_undegraded":"2024-03-29T09:09:15.577021+0000","last_fullsized":"2024-03-29T09:09:15.577021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534030+0000","last_change":"2024-03-29T09:08:57.427194+0000","last_active":"2024-03-29T09:09:01.534030+0000","last_peered":"2024-03-29T09:09:01.534030+0000","last_clean":"2024-03-29T09:09:01.534030+0000","last_became_active":"2024-03-29T09:08:57.425015+0000","last_became_peered":"2024-03-29T09:08:57.425015+0000","last_unstale":"2024-03-29T09:09:01.534030+0000","last_undegraded":"2024-03-29T09:09:01.534030+0000","last_fullsized":"2024-03-29T09:09:01.534030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527755+0000","last_change":"2024-03-29T09:09:01.442768+0000","last_active":"2024-03-29T09:09:01.527755+0000","last_peered":"2024-03-29T09:09:01.527755+0000","last_clean":"2024-03-29T09:09:01.527755+0000","last_became_active":"2024-03-29T09:09:01.442373+0000","last_became_peered":"2024-03-29T09:09:01.442373+0000","last_unstale":"2024-03-29T09:09:01.527755+0000","last_undegraded":"2024-03-29T09:09:01.527755+0000","last_fullsized":"2024-03-29T09:09:01.527755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:05:41.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.432870+0000","last_change":"2024-03-29T09:08:56.912783+0000","last_active":"2024-03-29T09:09:01.432870+0000","last_peered":"2024-03-29T09:09:01.432870+0000","last_clean":"2024-03-29T09:09:01.432870+0000","last_became_active":"2024-03-29T09:08:56.912488+0000","last_became_peered":"2024-03-29T09:08:56.912488+0000","last_unstale":"2024-03-29T09:09:01.432870+0000","last_undegraded":"2024-03-29T09:09:01.432870+0000","last_fullsized":"2024-03-29T09:09:01.432870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525440+0000","last_change":"2024-03-29T09:08:57.417913+0000","last_active":"2024-03-29T09:09:01.525440+0000","last_peered":"2024-03-29T09:09:01.525440+0000","last_clean":"2024-03-29T09:09:01.525440+0000","last_became_active":"2024-03-29T09:08:57.416399+0000","last_became_peered":"2024-03-29T09:08:57.416399+0000","last_unstale":"2024-03-29T09:09:01.525440+0000","last_undegraded":"2024-03-29T09:09:01.525440+0000","last_fullsized":"2024-03-29T09:09:01.525440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525543+0000","last_change":"2024-03-29T09:09:01.448636+0000","last_active":"2024-03-29T09:09:01.525543+0000","last_peered":"2024-03-29T09:09:01.525543+0000","last_clean":"2024-03-29T09:09:01.525543+0000","last_became_active":"2024-03-29T09:09:01.448098+0000","last_became_peered":"2024-03-29T09:09:01.448098+0000","last_unstale":"2024-03-29T09:09:01.525543+0000","last_undegraded":"2024-03-29T09:09:01.525543+0000","last_fullsized":"2024-03-29T09:09:01.525543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414449+0000","last_change":"2024-03-29T09:08:56.906000+0000","last_active":"2024-03-29T09:09:01.414449+0000","last_peered":"2024-03-29T09:09:01.414449+0000","last_clean":"2024-03-29T09:09:01.414449+0000","last_became_active":"2024-03-29T09:08:56.905743+0000","last_became_peered":"2024-03-29T09:08:56.905743+0000","last_unstale":"2024-03-29T09:09:01.414449+0000","last_undegraded":"2024-03-29T09:09:01.414449+0000","last_fullsized":"2024-03-29T09:09:01.414449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527722+0000","last_change":"2024-03-29T09:08:57.429076+0000","last_active":"2024-03-29T09:09:01.527722+0000","last_peered":"2024-03-29T09:09:01.527722+0000","last_clean":"2024-03-29T09:09:01.527722+0000","last_became_active":"2024-03-29T09:08:57.426637+0000","last_became_peered":"2024-03-29T09:08:57.426637+0000","last_unstale":"2024-03-29T09:09:01.527722+0000","last_undegraded":"2024-03-29T09:09:01.527722+0000","last_fullsized":"2024-03-29T09:09:01.527722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:48.952540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530885+0000","last_change":"2024-03-29T09:09:01.437381+0000","last_active":"2024-03-29T09:09:01.530885+0000","last_peered":"2024-03-29T09:09:01.530885+0000","last_clean":"2024-03-29T09:09:01.530885+0000","last_became_active":"2024-03-29T09:09:01.436950+0000","last_became_peered":"2024-03-29T09:09:01.436950+0000","last_unstale":"2024-03-29T09:09:01.530885+0000","last_undegraded":"2024-03-29T09:09:01.530885+0000","last_fullsized":"2024-03-29T09:09:01.530885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.356624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422703+0000","last_change":"2024-03-29T09:08:56.915220+0000","last_active":"2024-03-29T09:09:01.422703+0000","last_peered":"2024-03-29T09:09:01.422703+0000","last_clean":"2024-03-29T09:09:01.422703+0000","last_became_active":"2024-03-29T09:08:56.914954+0000","last_became_peered":"2024-03-29T09:08:56.914954+0000","last_unstale":"2024-03-29T09:09:01.422703+0000","last_undegraded":"2024-03-29T09:09:01.422703+0000","last_fullsized":"2024-03-29T09:09:01.422703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:19:55.114280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528690+0000","last_change":"2024-03-29T09:08:57.423950+0000","last_active":"2024-03-29T09:09:01.528690+0000","last_peered":"2024-03-29T09:09:01.528690+0000","last_clean":"2024-03-29T09:09:01.528690+0000","last_became_active":"2024-03-29T09:08:57.423703+0000","last_became_peered":"2024-03-29T09:08:57.423703+0000","last_unstale":"2024-03-29T09:09:01.528690+0000","last_undegraded":"2024-03-29T09:09:01.528690+0000","last_fullsized":"2024-03-29T09:09:01.528690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:28.288403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530874+0000","last_change":"2024-03-29T09:09:01.440631+0000","last_active":"2024-03-29T09:09:01.530874+0000","last_peered":"2024-03-29T09:09:01.530874+0000","last_clean":"2024-03-29T09:09:01.530874+0000","last_became_active":"2024-03-29T09:09:01.440159+0000","last_became_peered":"2024-03-29T09:09:01.440159+0000","last_unstale":"2024-03-29T09:09:01.530874+0000","last_undegraded":"2024-03-29T09:09:01.530874+0000","last_fullsized":"2024-03-29T09:09:01.530874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.423297+0000","last_change":"2024-03-29T09:08:56.904028+0000","last_active":"2024-03-29T09:09:01.423297+0000","last_peered":"2024-03-29T09:09:01.423297+0000","last_clean":"2024-03-29T09:09:01.423297+0000","last_became_active":"2024-03-29T09:08:56.903620+0000","last_became_peered":"2024-03-29T09:08:56.903620+0000","last_unstale":"2024-03-29T09:09:01.423297+0000","last_undegraded":"2024-03-29T09:09:01.423297+0000","last_fullsized":"2024-03-29T09:09:01.423297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527655+0000","last_change":"2024-03-29T09:08:57.414230+0000","last_active":"2024-03-29T09:09:01.527655+0000","last_peered":"2024-03-29T09:09:01.527655+0000","last_clean":"2024-03-29T09:09:01.527655+0000","last_became_active":"2024-03-29T09:08:57.413731+0000","last_became_peered":"2024-03-29T09:08:57.413731+0000","last_unstale":"2024-03-29T09:09:01.527655+0000","last_undegraded":"2024-03-29T09:09:01.527655+0000","last_fullsized":"2024-03-29T09:09:01.527655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528703+0000","last_change":"2024-03-29T09:09:01.435579+0000","last_active":"2024-03-29T09:09:01.528703+0000","last_peered":"2024-03-29T09:09:01.528703+0000","last_clean":"2024-03-29T09:09:01.528703+0000","last_became_active":"2024-03-29T09:09:01.433461+0000","last_became_peered":"2024-03-29T09:09:01.433461+0000","last_unstale":"2024-03-29T09:09:01.528703+0000","last_undegraded":"2024-03-29T09:09:01.528703+0000","last_fullsized":"2024-03-29T09:09:01.528703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:17.513175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422681+0000","last_change":"2024-03-29T09:08:56.915230+0000","last_active":"2024-03-29T09:09:01.422681+0000","last_peered":"2024-03-29T09:09:01.422681+0000","last_clean":"2024-03-29T09:09:01.422681+0000","last_became_active":"2024-03-29T09:08:56.914943+0000","last_became_peered":"2024-03-29T09:08:56.914943+0000","last_unstale":"2024-03-29T09:09:01.422681+0000","last_undegraded":"2024-03-29T09:09:01.422681+0000","last_fullsized":"2024-03-29T09:09:01.422681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531054+0000","last_change":"2024-03-29T09:08:57.425607+0000","last_active":"2024-03-29T09:09:01.531054+0000","last_peered":"2024-03-29T09:09:01.531054+0000","last_clean":"2024-03-29T09:09:01.531054+0000","last_became_active":"2024-03-29T09:08:57.424221+0000","last_became_peered":"2024-03-29T09:08:57.424221+0000","last_unstale":"2024-03-29T09:09:01.531054+0000","last_undegraded":"2024-03-29T09:09:01.531054+0000","last_fullsized":"2024-03-29T09:09:01.531054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525430+0000","last_change":"2024-03-29T09:09:01.447292+0000","last_active":"2024-03-29T09:09:01.525430+0000","last_peered":"2024-03-29T09:09:01.525430+0000","last_clean":"2024-03-29T09:09:01.525430+0000","last_became_active":"2024-03-29T09:09:01.446779+0000","last_became_peered":"2024-03-29T09:09:01.446779+0000","last_unstale":"2024-03-29T09:09:01.525430+0000","last_undegraded":"2024-03-29T09:09:01.525430+0000","last_fullsized":"2024-03-29T09:09:01.525430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:10.002579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419894+0000","last_change":"2024-03-29T09:08:56.905021+0000","last_active":"2024-03-29T09:09:01.419894+0000","last_peered":"2024-03-29T09:09:01.419894+0000","last_clean":"2024-03-29T09:09:01.419894+0000","last_became_active":"2024-03-29T09:08:56.903363+0000","last_became_peered":"2024-03-29T09:08:56.903363+0000","last_unstale":"2024-03-29T09:09:01.419894+0000","last_undegraded":"2024-03-29T09:09:01.419894+0000","last_fullsized":"2024-03-29T09:09:01.419894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530999+0000","last_change":"2024-03-29T09:08:57.427940+0000","last_active":"2024-03-29T09:09:01.530999+0000","last_peered":"2024-03-29T09:09:01.530999+0000","last_clean":"2024-03-29T09:09:01.530999+0000","last_became_active":"2024-03-29T09:08:57.427293+0000","last_became_peered":"2024-03-29T09:08:57.427293+0000","last_unstale":"2024-03-29T09:09:01.530999+0000","last_undegraded":"2024-03-29T09:09:01.530999+0000","last_fullsized":"2024-03-29T09:09:01.530999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525532+0000","last_change":"2024-03-29T09:09:01.451319+0000","last_active":"2024-03-29T09:09:01.525532+0000","last_peered":"2024-03-29T09:09:01.525532+0000","last_clean":"2024-03-29T09:09:01.525532+0000","last_became_active":"2024-03-29T09:09:01.450880+0000","last_became_peered":"2024-03-29T09:09:01.450880+0000","last_unstale":"2024-03-29T09:09:01.525532+0000","last_undegraded":"2024-03-29T09:09:01.525532+0000","last_fullsized":"2024-03-29T09:09:01.525532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414329+0000","last_change":"2024-03-29T09:08:56.909170+0000","last_active":"2024-03-29T09:09:01.414329+0000","last_peered":"2024-03-29T09:09:01.414329+0000","last_clean":"2024-03-29T09:09:01.414329+0000","last_became_active":"2024-03-29T09:08:56.908866+0000","last_became_peered":"2024-03-29T09:08:56.908866+0000","last_unstale":"2024-03-29T09:09:01.414329+0000","last_undegraded":"2024-03-29T09:09:01.414329+0000","last_fullsized":"2024-03-29T09:09:01.414329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:58.589863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533977+0000","last_change":"2024-03-29T09:08:57.415898+0000","last_active":"2024-03-29T09:09:01.533977+0000","last_peered":"2024-03-29T09:09:01.533977+0000","last_clean":"2024-03-29T09:09:01.533977+0000","last_became_active":"2024-03-29T09:08:57.415653+0000","last_became_peered":"2024-03-29T09:08:57.415653+0000","last_unstale":"2024-03-29T09:09:01.533977+0000","last_undegraded":"2024-03-29T09:09:01.533977+0000","last_fullsized":"2024-03-29T09:09:01.533977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525566+0000","last_change":"2024-03-29T09:09:01.451066+0000","last_active":"2024-03-29T09:09:01.525566+0000","last_peered":"2024-03-29T09:09:01.525566+0000","last_clean":"2024-03-29T09:09:01.525566+0000","last_became_active":"2024-03-29T09:09:01.450444+0000","last_became_peered":"2024-03-29T09:09:01.450444+0000","last_unstale":"2024-03-29T09:09:01.525566+0000","last_undegraded":"2024-03-29T09:09:01.525566+0000","last_fullsized":"2024-03-29T09:09:01.525566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530697+0000","last_change":"2024-03-29T09:08:57.426320+0000","last_active":"2024-03-29T09:09:01.530697+0000","last_peered":"2024-03-29T09:09:01.530697+0000","last_clean":"2024-03-29T09:09:01.530697+0000","last_became_active":"2024-03-29T09:08:57.426071+0000","last_became_peered":"2024-03-29T09:08:57.426071+0000","last_unstale":"2024-03-29T09:09:01.530697+0000","last_undegraded":"2024-03-29T09:09:01.530697+0000","last_fullsized":"2024-03-29T09:09:01.530697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.428903+0000","last_change":"2024-03-29T09:08:56.906294+0000","last_active":"2024-03-29T09:09:01.428903+0000","last_peered":"2024-03-29T09:09:01.428903+0000","last_clean":"2024-03-29T09:09:01.428903+0000","last_became_active":"2024-03-29T09:08:56.905993+0000","last_became_peered":"2024-03-29T09:08:56.905993+0000","last_unstale":"2024-03-29T09:09:01.428903+0000","last_undegraded":"2024-03-29T09:09:01.428903+0000","last_fullsized":"2024-03-29T09:09:01.428903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527859+0000","last_change":"2024-03-29T09:09:01.435004+0000","last_active":"2024-03-29T09:09:01.527859+0000","last_peered":"2024-03-29T09:09:01.527859+0000","last_clean":"2024-03-29T09:09:01.527859+0000","last_became_active":"2024-03-29T09:09:01.434628+0000","last_became_peered":"2024-03-29T09:09:01.434628+0000","last_unstale":"2024-03-29T09:09:01.527859+0000","last_undegraded":"2024-03-29T09:09:01.527859+0000","last_fullsized":"2024-03-29T09:09:01.527859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:05.380910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527458+0000","last_change":"2024-03-29T09:08:57.411609+0000","last_active":"2024-03-29T09:09:01.527458+0000","last_peered":"2024-03-29T09:09:01.527458+0000","last_clean":"2024-03-29T09:09:01.527458+0000","last_became_active":"2024-03-29T09:08:57.411313+0000","last_became_peered":"2024-03-29T09:08:57.411313+0000","last_unstale":"2024-03-29T09:09:01.527458+0000","last_undegraded":"2024-03-29T09:09:01.527458+0000","last_fullsized":"2024-03-29T09:09:01.527458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414265+0000","last_change":"2024-03-29T09:08:56.900590+0000","last_active":"2024-03-29T09:09:01.414265+0000","last_peered":"2024-03-29T09:09:01.414265+0000","last_clean":"2024-03-29T09:09:01.414265+0000","last_became_active":"2024-03-29T09:08:56.900329+0000","last_became_peered":"2024-03-29T09:08:56.900329+0000","last_unstale":"2024-03-29T09:09:01.414265+0000","last_undegraded":"2024-03-29T09:09:01.414265+0000","last_fullsized":"2024-03-29T09:09:01.414265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:56.593036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420635+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:09:01.420635+0000","last_peered":"2024-03-29T09:09:01.420635+0000","last_clean":"2024-03-29T09:09:01.420635+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:09:01.420635+0000","last_undegraded":"2024-03-29T09:09:01.420635+0000","last_fullsized":"2024-03-29T09:09:01.420635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530864+0000","last_change":"2024-03-29T09:09:01.438759+0000","last_active":"2024-03-29T09:09:01.530864+0000","last_peered":"2024-03-29T09:09:01.530864+0000","last_clean":"2024-03-29T09:09:01.530864+0000","last_became_active":"2024-03-29T09:09:01.438275+0000","last_became_peered":"2024-03-29T09:09:01.438275+0000","last_unstale":"2024-03-29T09:09:01.530864+0000","last_undegraded":"2024-03-29T09:09:01.530864+0000","last_fullsized":"2024-03-29T09:09:01.530864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:02:29.227678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525418+0000","last_change":"2024-03-29T09:08:57.418469+0000","last_active":"2024-03-29T09:09:01.525418+0000","last_peered":"2024-03-29T09:09:01.525418+0000","last_clean":"2024-03-29T09:09:01.525418+0000","last_became_active":"2024-03-29T09:08:57.416011+0000","last_became_peered":"2024-03-29T09:08:57.416011+0000","last_unstale":"2024-03-29T09:09:01.525418+0000","last_undegraded":"2024-03-29T09:09:01.525418+0000","last_fullsized":"2024-03-29T09:09:01.525418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414118+0000","last_change":"2024-03-29T09:08:56.900382+0000","last_active":"2024-03-29T09:09:01.414118+0000","last_peered":"2024-03-29T09:09:01.414118+0000","last_clean":"2024-03-29T09:09:01.414118+0000","last_became_active":"2024-03-29T09:08:56.899884+0000","last_became_peered":"2024-03-29T09:08:56.899884+0000","last_unstale":"2024-03-29T09:09:01.414118+0000","last_undegraded":"2024-03-29T09:09:01.414118+0000","last_fullsized":"2024-03-29T09:09:01.414118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:51.548290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525469+0000","last_change":"2024-03-29T09:09:01.447790+0000","last_active":"2024-03-29T09:09:01.525469+0000","last_peered":"2024-03-29T09:09:01.525469+0000","last_clean":"2024-03-29T09:09:01.525469+0000","last_became_active":"2024-03-29T09:09:01.447254+0000","last_became_peered":"2024-03-29T09:09:01.447254+0000","last_unstale":"2024-03-29T09:09:01.525469+0000","last_undegraded":"2024-03-29T09:09:01.525469+0000","last_fullsized":"2024-03-29T09:09:01.525469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:27.709797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.415102+0000","last_change":"2024-03-29T09:08:56.904347+0000","last_active":"2024-03-29T09:09:01.415102+0000","last_peered":"2024-03-29T09:09:01.415102+0000","last_clean":"2024-03-29T09:09:01.415102+0000","last_became_active":"2024-03-29T09:08:56.903387+0000","last_became_peered":"2024-03-29T09:08:56.903387+0000","last_unstale":"2024-03-29T09:09:01.415102+0000","last_undegraded":"2024-03-29T09:09:01.415102+0000","last_fullsized":"2024-03-29T09:09:01.415102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525510+0000","last_change":"2024-03-29T09:08:57.419559+0000","last_active":"2024-03-29T09:09:01.525510+0000","last_peered":"2024-03-29T09:09:01.525510+0000","last_clean":"2024-03-29T09:09:01.525510+0000","last_became_active":"2024-03-29T09:08:57.419295+0000","last_became_peered":"2024-03-29T09:08:57.419295+0000","last_unstale":"2024-03-29T09:09:01.525510+0000","last_undegraded":"2024-03-29T09:09:01.525510+0000","last_fullsized":"2024-03-29T09:09:01.525510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527366+0000","last_change":"2024-03-29T09:09:01.433109+0000","last_active":"2024-03-29T09:09:01.527366+0000","last_peered":"2024-03-29T09:09:01.527366+0000","last_clean":"2024-03-29T09:09:01.527366+0000","last_became_active":"2024-03-29T09:09:01.432662+0000","last_became_peered":"2024-03-29T09:09:01.432662+0000","last_unstale":"2024-03-29T09:09:01.527366+0000","last_undegraded":"2024-03-29T09:09:01.527366+0000","last_fullsized":"2024-03-29T09:09:01.527366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422867+0000","last_change":"2024-03-29T09:08:56.912181+0000","last_active":"2024-03-29T09:09:01.422867+0000","last_peered":"2024-03-29T09:09:01.422867+0000","last_clean":"2024-03-29T09:09:01.422867+0000","last_became_active":"2024-03-29T09:08:56.911842+0000","last_became_peered":"2024-03-29T09:08:56.911842+0000","last_unstale":"2024-03-29T09:09:01.422867+0000","last_undegraded":"2024-03-29T09:09:01.422867+0000","last_fullsized":"2024-03-29T09:09:01.422867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527556+0000","last_change":"2024-03-29T09:08:57.423784+0000","last_active":"2024-03-29T09:09:01.527556+0000","last_peered":"2024-03-29T09:09:01.527556+0000","last_clean":"2024-03-29T09:09:01.527556+0000","last_became_active":"2024-03-29T09:08:57.423442+0000","last_became_peered":"2024-03-29T09:08:57.423442+0000","last_unstale":"2024-03-29T09:09:01.527556+0000","last_undegraded":"2024-03-29T09:09:01.527556+0000","last_fullsized":"2024-03-29T09:09:01.527556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:09:01.451554+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:09:01.450348+0000","last_became_peered":"2024-03-29T09:09:01.450348+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414924+0000","last_change":"2024-03-29T09:08:56.911357+0000","last_active":"2024-03-29T09:09:01.414924+0000","last_peered":"2024-03-29T09:09:01.414924+0000","last_clean":"2024-03-29T09:09:01.414924+0000","last_became_active":"2024-03-29T09:08:56.910692+0000","last_became_peered":"2024-03-29T09:08:56.910692+0000","last_unstale":"2024-03-29T09:09:01.414924+0000","last_undegraded":"2024-03-29T09:09:01.414924+0000","last_fullsized":"2024-03-29T09:09:01.414924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:32.927312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527540+0000","last_change":"2024-03-29T09:08:57.415718+0000","last_active":"2024-03-29T09:09:01.527540+0000","last_peered":"2024-03-29T09:09:01.527540+0000","last_clean":"2024-03-29T09:09:01.527540+0000","last_became_active":"2024-03-29T09:08:57.415442+0000","last_became_peered":"2024-03-29T09:08:57.415442+0000","last_unstale":"2024-03-29T09:09:01.527540+0000","last_undegraded":"2024-03-29T09:09:01.527540+0000","last_fullsized":"2024-03-29T09:09:01.527540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:44.638689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531018+0000","last_change":"2024-03-29T09:09:01.444447+0000","last_active":"2024-03-29T09:09:01.531018+0000","last_peered":"2024-03-29T09:09:01.531018+0000","last_clean":"2024-03-29T09:09:01.531018+0000","last_became_active":"2024-03-29T09:09:01.443976+0000","last_became_peered":"2024-03-29T09:09:01.443976+0000","last_unstale":"2024-03-29T09:09:01.531018+0000","last_undegraded":"2024-03-29T09:09:01.531018+0000","last_fullsized":"2024-03-29T09:09:01.531018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422915+0000","last_change":"2024-03-29T09:08:56.904777+0000","last_active":"2024-03-29T09:09:01.422915+0000","last_peered":"2024-03-29T09:09:01.422915+0000","last_clean":"2024-03-29T09:09:01.422915+0000","last_became_active":"2024-03-29T09:08:56.904503+0000","last_became_peered":"2024-03-29T09:08:56.904503+0000","last_unstale":"2024-03-29T09:09:01.422915+0000","last_undegraded":"2024-03-29T09:09:01.422915+0000","last_fullsized":"2024-03-29T09:09:01.422915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527647+0000","last_change":"2024-03-29T09:08:57.409679+0000","last_active":"2024-03-29T09:09:01.527647+0000","last_peered":"2024-03-29T09:09:01.527647+0000","last_clean":"2024-03-29T09:09:01.527647+0000","last_became_active":"2024-03-29T09:08:57.409346+0000","last_became_peered":"2024-03-29T09:08:57.409346+0000","last_unstale":"2024-03-29T09:09:01.527647+0000","last_undegraded":"2024-03-29T09:09:01.527647+0000","last_fullsized":"2024-03-29T09:09:01.527647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527392+0000","last_change":"2024-03-29T09:09:01.451808+0000","last_active":"2024-03-29T09:09:01.527392+0000","last_peered":"2024-03-29T09:09:01.527392+0000","last_clean":"2024-03-29T09:09:01.527392+0000","last_became_active":"2024-03-29T09:09:01.451262+0000","last_became_peered":"2024-03-29T09:09:01.451262+0000","last_unstale":"2024-03-29T09:09:01.527392+0000","last_undegraded":"2024-03-29T09:09:01.527392+0000","last_fullsized":"2024-03-29T09:09:01.527392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:25.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422122+0000","last_change":"2024-03-29T09:08:56.912120+0000","last_active":"2024-03-29T09:09:01.422122+0000","last_peered":"2024-03-29T09:09:01.422122+0000","last_clean":"2024-03-29T09:09:01.422122+0000","last_became_active":"2024-03-29T09:08:56.911834+0000","last_became_peered":"2024-03-29T09:08:56.911834+0000","last_unstale":"2024-03-29T09:09:01.422122+0000","last_undegraded":"2024-03-29T09:09:01.422122+0000","last_fullsized":"2024-03-29T09:09:01.422122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527406+0000","last_change":"2024-03-29T09:08:57.413619+0000","last_active":"2024-03-29T09:09:01.527406+0000","last_peered":"2024-03-29T09:09:01.527406+0000","last_clean":"2024-03-29T09:09:01.527406+0000","last_became_active":"2024-03-29T09:08:57.409612+0000","last_became_peered":"2024-03-29T09:08:57.409612+0000","last_unstale":"2024-03-29T09:09:01.527406+0000","last_undegraded":"2024-03-29T09:09:01.527406+0000","last_fullsized":"2024-03-29T09:09:01.527406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533989+0000","last_change":"2024-03-29T09:09:01.436295+0000","last_active":"2024-03-29T09:09:01.533989+0000","last_peered":"2024-03-29T09:09:01.533989+0000","last_clean":"2024-03-29T09:09:01.533989+0000","last_became_active":"2024-03-29T09:09:01.435972+0000","last_became_peered":"2024-03-29T09:09:01.435972+0000","last_unstale":"2024-03-29T09:09:01.533989+0000","last_undegraded":"2024-03-29T09:09:01.533989+0000","last_fullsized":"2024-03-29T09:09:01.533989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429481+0000","last_change":"2024-03-29T09:08:56.907160+0000","last_active":"2024-03-29T09:09:01.429481+0000","last_peered":"2024-03-29T09:09:01.429481+0000","last_clean":"2024-03-29T09:09:01.429481+0000","last_became_active":"2024-03-29T09:08:56.906801+0000","last_became_peered":"2024-03-29T09:08:56.906801+0000","last_unstale":"2024-03-29T09:09:01.429481+0000","last_undegraded":"2024-03-29T09:09:01.429481+0000","last_fullsized":"2024-03-29T09:09:01.429481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527323+0000","last_change":"2024-03-29T09:08:57.425389+0000","last_active":"2024-03-29T09:09:01.527323+0000","last_peered":"2024-03-29T09:09:01.527323+0000","last_clean":"2024-03-29T09:09:01.527323+0000","last_became_active":"2024-03-29T09:08:57.425127+0000","last_became_peered":"2024-03-29T09:08:57.425127+0000","last_unstale":"2024-03-29T09:09:01.527323+0000","last_undegraded":"2024-03-29T09:09:01.527323+0000","last_fullsized":"2024-03-29T09:09:01.527323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527793+0000","last_change":"2024-03-29T09:09:01.432980+0000","last_active":"2024-03-29T09:09:01.527793+0000","last_peered":"2024-03-29T09:09:01.527793+0000","last_clean":"2024-03-29T09:09:01.527793+0000","last_became_active":"2024-03-29T09:09:01.432597+0000","last_became_peered":"2024-03-29T09:09:01.432597+0000","last_unstale":"2024-03-29T09:09:01.527793+0000","last_undegraded":"2024-03-29T09:09:01.527793+0000","last_fullsized":"2024-03-29T09:09:01.527793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:52.143266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414584+0000","last_change":"2024-03-29T09:08:56.901765+0000","last_active":"2024-03-29T09:09:01.414584+0000","last_peered":"2024-03-29T09:09:01.414584+0000","last_clean":"2024-03-29T09:09:01.414584+0000","last_became_active":"2024-03-29T09:08:56.901513+0000","last_became_peered":"2024-03-29T09:08:56.901513+0000","last_unstale":"2024-03-29T09:09:01.414584+0000","last_undegraded":"2024-03-29T09:09:01.414584+0000","last_fullsized":"2024-03-29T09:09:01.414584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:55.597965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527664+0000","last_change":"2024-03-29T09:08:57.406785+0000","last_active":"2024-03-29T09:09:01.527664+0000","last_peered":"2024-03-29T09:09:01.527664+0000","last_clean":"2024-03-29T09:09:01.527664+0000","last_became_active":"2024-03-29T09:08:57.406261+0000","last_became_peered":"2024-03-29T09:08:57.406261+0000","last_unstale":"2024-03-29T09:09:01.527664+0000","last_undegraded":"2024-03-29T09:09:01.527664+0000","last_fullsized":"2024-03-29T09:09:01.527664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:09:43.625475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530845+0000","last_change":"2024-03-29T09:09:01.445228+0000","last_active":"2024-03-29T09:09:01.530845+0000","last_peered":"2024-03-29T09:09:01.530845+0000","last_clean":"2024-03-29T09:09:01.530845+0000","last_became_active":"2024-03-29T09:09:01.444917+0000","last_became_peered":"2024-03-29T09:09:01.444917+0000","last_unstale":"2024-03-29T09:09:01.530845+0000","last_undegraded":"2024-03-29T09:09:01.530845+0000","last_fullsized":"2024-03-29T09:09:01.530845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.431916+0000","last_change":"2024-03-29T09:08:56.914015+0000","last_active":"2024-03-29T09:09:01.431916+0000","last_peered":"2024-03-29T09:09:01.431916+0000","last_clean":"2024-03-29T09:09:01.431916+0000","last_became_active":"2024-03-29T09:08:56.913673+0000","last_became_peered":"2024-03-29T09:08:56.913673+0000","last_unstale":"2024-03-29T09:09:01.431916+0000","last_undegraded":"2024-03-29T09:09:01.431916+0000","last_fullsized":"2024-03-29T09:09:01.431916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527811+0000","last_change":"2024-03-29T09:08:57.410999+0000","last_active":"2024-03-29T09:09:01.527811+0000","last_peered":"2024-03-29T09:09:01.527811+0000","last_clean":"2024-03-29T09:09:01.527811+0000","last_became_active":"2024-03-29T09:08:57.410742+0000","last_became_peered":"2024-03-29T09:08:57.410742+0000","last_unstale":"2024-03-29T09:09:01.527811+0000","last_undegraded":"2024-03-29T09:09:01.527811+0000","last_fullsized":"2024-03-29T09:09:01.527811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:24.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530767+0000","last_change":"2024-03-29T09:09:01.434584+0000","last_active":"2024-03-29T09:09:01.530767+0000","last_peered":"2024-03-29T09:09:01.530767+0000","last_clean":"2024-03-29T09:09:01.530767+0000","last_became_active":"2024-03-29T09:09:01.434233+0000","last_became_peered":"2024-03-29T09:09:01.434233+0000","last_unstale":"2024-03-29T09:09:01.530767+0000","last_undegraded":"2024-03-29T09:09:01.530767+0000","last_fullsized":"2024-03-29T09:09:01.530767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:18:14.395090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420352+0000","last_change":"2024-03-29T09:08:56.908753+0000","last_active":"2024-03-29T09:09:01.420352+0000","last_peered":"2024-03-29T09:09:01.420352+0000","last_clean":"2024-03-29T09:09:01.420352+0000","last_became_active":"2024-03-29T09:08:56.908505+0000","last_became_peered":"2024-03-29T09:08:56.908505+0000","last_unstale":"2024-03-29T09:09:01.420352+0000","last_undegraded":"2024-03-29T09:09:01.420352+0000","last_fullsized":"2024-03-29T09:09:01.420352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533999+0000","last_change":"2024-03-29T09:08:57.418006+0000","last_active":"2024-03-29T09:09:01.533999+0000","last_peered":"2024-03-29T09:09:01.533999+0000","last_clean":"2024-03-29T09:09:01.533999+0000","last_became_active":"2024-03-29T09:08:57.417644+0000","last_became_peered":"2024-03-29T09:08:57.417644+0000","last_unstale":"2024-03-29T09:09:01.533999+0000","last_undegraded":"2024-03-29T09:09:01.533999+0000","last_fullsized":"2024-03-29T09:09:01.533999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527566+0000","last_change":"2024-03-29T09:09:01.431602+0000","last_active":"2024-03-29T09:09:01.527566+0000","last_peered":"2024-03-29T09:09:01.527566+0000","last_clean":"2024-03-29T09:09:01.527566+0000","last_became_active":"2024-03-29T09:09:01.430422+0000","last_became_peered":"2024-03-29T09:09:01.430422+0000","last_unstale":"2024-03-29T09:09:01.527566+0000","last_undegraded":"2024-03-29T09:09:01.527566+0000","last_fullsized":"2024-03-29T09:09:01.527566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:21:33.289630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429605+0000","last_change":"2024-03-29T09:08:56.907602+0000","last_active":"2024-03-29T09:09:01.429605+0000","last_peered":"2024-03-29T09:09:01.429605+0000","last_clean":"2024-03-29T09:09:01.429605+0000","last_became_active":"2024-03-29T09:08:56.907321+0000","last_became_peered":"2024-03-29T09:08:56.907321+0000","last_unstale":"2024-03-29T09:09:01.429605+0000","last_undegraded":"2024-03-29T09:09:01.429605+0000","last_fullsized":"2024-03-29T09:09:01.429605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534106+0000","last_change":"2024-03-29T09:08:57.419001+0000","last_active":"2024-03-29T09:09:01.534106+0000","last_peered":"2024-03-29T09:09:01.534106+0000","last_clean":"2024-03-29T09:09:01.534106+0000","last_became_active":"2024-03-29T09:08:57.418769+0000","last_became_peered":"2024-03-29T09:08:57.418769+0000","last_unstale":"2024-03-29T09:09:01.534106+0000","last_undegraded":"2024-03-29T09:09:01.534106+0000","last_fullsized":"2024-03-29T09:09:01.534106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530742+0000","last_change":"2024-03-29T09:08:57.408417+0000","last_active":"2024-03-29T09:09:01.530742+0000","last_peered":"2024-03-29T09:09:01.530742+0000","last_clean":"2024-03-29T09:09:01.530742+0000","last_became_active":"2024-03-29T09:08:57.408131+0000","last_became_peered":"2024-03-29T09:08:57.408131+0000","last_unstale":"2024-03-29T09:09:01.530742+0000","last_undegraded":"2024-03-29T09:09:01.530742+0000","last_fullsized":"2024-03-29T09:09:01.530742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527306+0000","last_change":"2024-03-29T09:08:57.418220+0000","last_active":"2024-03-29T09:09:01.527306+0000","last_peered":"2024-03-29T09:09:01.527306+0000","last_clean":"2024-03-29T09:09:01.527306+0000","last_became_active":"2024-03-29T09:08:57.417900+0000","last_became_peered":"2024-03-29T09:08:57.417900+0000","last_unstale":"2024-03-29T09:09:01.527306+0000","last_undegraded":"2024-03-29T09:09:01.527306+0000","last_fullsized":"2024-03-29T09:09:01.527306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527708+0000","last_change":"2024-03-29T09:08:57.424647+0000","last_active":"2024-03-29T09:09:01.527708+0000","last_peered":"2024-03-29T09:09:01.527708+0000","last_clean":"2024-03-29T09:09:01.527708+0000","last_became_active":"2024-03-29T09:08:57.424404+0000","last_became_peered":"2024-03-29T09:08:57.424404+0000","last_unstale":"2024-03-29T09:09:01.527708+0000","last_undegraded":"2024-03-29T09:09:01.527708+0000","last_fullsized":"2024-03-29T09:09:01.527708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:06:29.651626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530787+0000","last_change":"2024-03-29T09:08:57.419276+0000","last_active":"2024-03-29T09:09:01.530787+0000","last_peered":"2024-03-29T09:09:01.530787+0000","last_clean":"2024-03-29T09:09:01.530787+0000","last_became_active":"2024-03-29T09:08:57.418974+0000","last_became_peered":"2024-03-29T09:08:57.418974+0000","last_unstale":"2024-03-29T09:09:01.530787+0000","last_undegraded":"2024-03-29T09:09:01.530787+0000","last_fullsized":"2024-03-29T09:09:01.530787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525521+0000","last_change":"2024-03-29T09:08:57.429156+0000","last_active":"2024-03-29T09:09:01.525521+0000","last_peered":"2024-03-29T09:09:01.525521+0000","last_clean":"2024-03-29T09:09:01.525521+0000","last_became_active":"2024-03-29T09:08:57.428873+0000","last_became_peered":"2024-03-29T09:08:57.428873+0000","last_unstale":"2024-03-29T09:09:01.525521+0000","last_undegraded":"2024-03-29T09:09:01.525521+0000","last_fullsized":"2024-03-29T09:09:01.525521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525556+0000","last_change":"2024-03-29T09:08:57.423650+0000","last_active":"2024-03-29T09:09:01.525556+0000","last_peered":"2024-03-29T09:09:01.525556+0000","last_clean":"2024-03-29T09:09:01.525556+0000","last_became_active":"2024-03-29T09:08:57.423385+0000","last_became_peered":"2024-03-29T09:08:57.423385+0000","last_unstale":"2024-03-29T09:09:01.525556+0000","last_undegraded":"2024-03-29T09:09:01.525556+0000","last_fullsized":"2024-03-29T09:09:01.525556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531076+0000","last_change":"2024-03-29T09:08:57.413866+0000","last_active":"2024-03-29T09:09:01.531076+0000","last_peered":"2024-03-29T09:09:01.531076+0000","last_clean":"2024-03-29T09:09:01.531076+0000","last_became_active":"2024-03-29T09:08:57.413483+0000","last_became_peered":"2024-03-29T09:08:57.413483+0000","last_unstale":"2024-03-29T09:09:01.531076+0000","last_undegraded":"2024-03-29T09:09:01.531076+0000","last_fullsized":"2024-03-29T09:09:01.531076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530897+0000","last_change":"2024-03-29T09:08:57.422855+0000","last_active":"2024-03-29T09:09:01.530897+0000","last_peered":"2024-03-29T09:09:01.530897+0000","last_clean":"2024-03-29T09:09:01.530897+0000","last_became_active":"2024-03-29T09:08:57.422350+0000","last_became_peered":"2024-03-29T09:08:57.422350+0000","last_unstale":"2024-03-29T09:09:01.530897+0000","last_undegraded":"2024-03-29T09:09:01.530897+0000","last_fullsized":"2024-03-29T09:09:01.530897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527637+0000","last_change":"2024-03-29T09:08:57.427109+0000","last_active":"2024-03-29T09:09:01.527637+0000","last_peered":"2024-03-29T09:09:01.527637+0000","last_clean":"2024-03-29T09:09:01.527637+0000","last_became_active":"2024-03-29T09:08:57.424525+0000","last_became_peered":"2024-03-29T09:08:57.424525+0000","last_unstale":"2024-03-29T09:09:01.527637+0000","last_undegraded":"2024-03-29T09:09:01.527637+0000","last_fullsized":"2024-03-29T09:09:01.527637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:52.774904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530755+0000","last_change":"2024-03-29T09:08:57.416922+0000","last_active":"2024-03-29T09:09:01.530755+0000","last_peered":"2024-03-29T09:09:01.530755+0000","last_clean":"2024-03-29T09:09:01.530755+0000","last_became_active":"2024-03-29T09:08:57.415900+0000","last_became_peered":"2024-03-29T09:08:57.415900+0000","last_unstale":"2024-03-29T09:09:01.530755+0000","last_undegraded":"2024-03-29T09:09:01.530755+0000","last_fullsized":"2024-03-29T09:09:01.530755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528724+0000","last_change":"2024-03-29T09:08:57.416177+0000","last_active":"2024-03-29T09:09:01.528724+0000","last_peered":"2024-03-29T09:09:01.528724+0000","last_clean":"2024-03-29T09:09:01.528724+0000","last_became_active":"2024-03-29T09:08:57.415844+0000","last_became_peered":"2024-03-29T09:08:57.415844+0000","last_unstale":"2024-03-29T09:09:01.528724+0000","last_undegraded":"2024-03-29T09:09:01.528724+0000","last_fullsized":"2024-03-29T09:09:01.528724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527479+0000","last_change":"2024-03-29T09:08:57.411579+0000","last_active":"2024-03-29T09:09:01.527479+0000","last_peered":"2024-03-29T09:09:01.527479+0000","last_clean":"2024-03-29T09:09:01.527479+0000","last_became_active":"2024-03-29T09:08:57.411297+0000","last_became_peered":"2024-03-29T09:08:57.411297+0000","last_unstale":"2024-03-29T09:09:01.527479+0000","last_undegraded":"2024-03-29T09:09:01.527479+0000","last_fullsized":"2024-03-29T09:09:01.527479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:13.990570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534013+0000","last_change":"2024-03-29T09:08:57.417300+0000","last_active":"2024-03-29T09:09:01.534013+0000","last_peered":"2024-03-29T09:09:01.534013+0000","last_clean":"2024-03-29T09:09:01.534013+0000","last_became_active":"2024-03-29T09:08:57.416733+0000","last_became_peered":"2024-03-29T09:08:57.416733+0000","last_unstale":"2024-03-29T09:09:01.534013+0000","last_undegraded":"2024-03-29T09:09:01.534013+0000","last_fullsized":"2024-03-29T09:09:01.534013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:51.863125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":690226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:09:26.786 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T09:09:26.786 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T09:09:26.786 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-29T09:09:26.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-29T09:09:26.787 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-29T09:09:27.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 Processor -- start 2024-03-29T09:09:27.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 -- start start 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c1608e0 0x7f223c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f223c160cc0 0x7f223c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f223c0d3a70 0x7f223c0d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223c0cddc0 con 0x7f223c160cc0 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223c0ce0a0 con 0x7f223c0d3a70 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f2242a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0ce380 con 0x7f223c1608e0 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.012+0000 7f223b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f223c160cc0 0x7f223c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f223c160cc0 0x7f223c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48408/0 (socket says 172.21.15.99:48408) 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 -- 172.21.15.99:0/1990249852 learned_addr learned my addr 172.21.15.99:0/1990249852 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f2240a00700 1 --2- 172.21.15.99:0/1990249852 >> v2:172.21.15.133:3300/0 conn(0x7f223c0d3a70 0x7f223c0d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223be00700 1 --2- 172.21.15.99:0/1990249852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c1608e0 0x7f223c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 -- 172.21.15.99:0/1990249852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c1608e0 msgr2=0x7f223c0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:27.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 --2- 172.21.15.99:0/1990249852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c1608e0 0x7f223c0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 -- 172.21.15.99:0/1990249852 >> v2:172.21.15.133:3300/0 conn(0x7f223c0d3a70 msgr2=0x7f223c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:27.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 --2- 172.21.15.99:0/1990249852 >> v2:172.21.15.133:3300/0 conn(0x7f223c0d3a70 0x7f223c0d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 -- 172.21.15.99:0/1990249852 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0ce660 con 0x7f223c160cc0 2024-03-29T09:09:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223b400700 1 --2- 172.21.15.99:0/1990249852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c0ceb80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f223c08d890 tx=0x7f2228009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6667e3bcc4231a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.013+0000 7f223aa00700 1 -- 172.21.15.99:0/1990249852 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228011830 con 0x7f223c160cc0 2024-03-29T09:09:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f223aa00700 1 -- 172.21.15.99:0/1990249852 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2228011990 con 0x7f223c160cc0 2024-03-29T09:09:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f223aa00700 1 -- 172.21.15.99:0/1990249852 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228011d20 con 0x7f223c160cc0 2024-03-29T09:09:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 -- 172.21.15.99:0/1990249852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 msgr2=0x7f223c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 --2- 172.21.15.99:0/1990249852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c0ceb80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f223c08d890 tx=0x7f2228009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 -- 172.21.15.99:0/1990249852 shutdown_connections 2024-03-29T09:09:27.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 --2- 172.21.15.99:0/1990249852 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c1608e0 0x7f223c0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 --2- 172.21.15.99:0/1990249852 >> v2:172.21.15.133:3300/0 conn(0x7f223c0d3a70 0x7f223c0d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 --2- 172.21.15.99:0/1990249852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c0ceb80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 -- 172.21.15.99:0/1990249852 >> 172.21.15.99:0/1990249852 conn(0x7f223c09c490 msgr2=0x7f223c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:27.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 -- 172.21.15.99:0/1990249852 shutdown_connections 2024-03-29T09:09:27.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.014+0000 7f2242a00700 1 -- 172.21.15.99:0/1990249852 wait complete. 2024-03-29T09:09:27.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.016+0000 7f2242a00700 1 Processor -- start 2024-03-29T09:09:27.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.016+0000 7f2242a00700 1 -- start start 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2242a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c0d3a70 0x7f223c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2242a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f223c1608e0 0x7f223c15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2242a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2242a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0ca220 con 0x7f223c160cc0 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2242a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c154b30 con 0x7f223c1608e0 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2242a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cb170 con 0x7f223c0d3a70 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2240a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f223b400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f223c1608e0 0x7f223c15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2240a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c15f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48424/0 (socket says 172.21.15.99:48424) 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f2240a00700 1 -- 172.21.15.99:0/2486499491 learned_addr learned my addr 172.21.15.99:0/2486499491 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:27.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.017+0000 7f223be00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c0d3a70 0x7f223c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2240a00700 1 -- 172.21.15.99:0/2486499491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c0d3a70 msgr2=0x7f223c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2240a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c0d3a70 0x7f223c0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2240a00700 1 -- 172.21.15.99:0/2486499491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f223c1608e0 msgr2=0x7f223c15f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2240a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f223c1608e0 0x7f223c15f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2240a00700 1 -- 172.21.15.99:0/2486499491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2228009550 con 0x7f223c160cc0 2024-03-29T09:09:27.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2240a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c15f930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f223400efc0 tx=0x7f223400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2233e00700 1 -- 172.21.15.99:0/2486499491 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234009ea0 con 0x7f223c160cc0 2024-03-29T09:09:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2233e00700 1 -- 172.21.15.99:0/2486499491 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223400f040 con 0x7f223c160cc0 2024-03-29T09:09:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c0ca4c0 con 0x7f223c160cc0 2024-03-29T09:09:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2233e00700 1 -- 172.21.15.99:0/2486499491 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223400ced0 con 0x7f223c160cc0 2024-03-29T09:09:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.018+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c154dd0 con 0x7f223c160cc0 2024-03-29T09:09:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.019+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c0677a0 con 0x7f223c160cc0 2024-03-29T09:09:27.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.020+0000 7f2233e00700 1 -- 172.21.15.99:0/2486499491 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2234014480 con 0x7f223c160cc0 2024-03-29T09:09:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.021+0000 7f2233e00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f221c06ac90 0x7f221c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.021+0000 7f2233e00700 1 -- 172.21.15.99:0/2486499491 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2234089ea0 con 0x7f223c160cc0 2024-03-29T09:09:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.021+0000 7f223be00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f221c06ac90 0x7f221c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.022+0000 7f223be00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f221c06ac90 0x7f221c06d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f222c009b60 tx=0x7f222c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.025+0000 7f2233e00700 1 -- 172.21.15.99:0/2486499491 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2234054cf0 con 0x7f223c160cc0 2024-03-29T09:09:27.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.284+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f223c0d0700 con 0x7f223c160cc0 2024-03-29T09:09:27.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.288+0000 7f2233e00700 1 -- 172.21.15.99:0/2486499491 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170927 (secure 0 0 0) 0x7f2234018020 con 0x7f223c160cc0 2024-03-29T09:09:27.290 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:27.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f221c06ac90 msgr2=0x7f221c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f221c06ac90 0x7f221c06d150 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f222c009b60 tx=0x7f222c009210 comp rx=0 tx=0).stop 2024-03-29T09:09:27.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 msgr2=0x7f223c15f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c15f930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f223400efc0 tx=0x7f223400c3a0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 shutdown_connections 2024-03-29T09:09:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f221c06ac90 0x7f221c06d150 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f223c160cc0 0x7f223c15f930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f223c0d3a70 0x7f223c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 --2- 172.21.15.99:0/2486499491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f223c1608e0 0x7f223c15f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 >> 172.21.15.99:0/2486499491 conn(0x7f223c09c490 msgr2=0x7f223c15e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:27.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 shutdown_connections 2024-03-29T09:09:27.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.292+0000 7f2242a00700 1 -- 172.21.15.99:0/2486499491 wait complete. 2024-03-29T09:09:27.309 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":41898},{"type":"v1","addr":"172.21.15.99:6833","nonce":41898}]},"active_addr":"172.21.15.99:6833/41898","active_change":"2024-03-29T09:08:40.428993+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4101,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":4248249875}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":1449381871}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3745843199}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3140274009}]}]} 2024-03-29T09:09:27.314 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-29T09:09:27.314 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-29T09:09:27.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:27.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.503+0000 7f0a27200700 1 Processor -- start 2024-03-29T09:09:27.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.503+0000 7f0a27200700 1 -- start start 2024-03-29T09:09:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.504+0000 7f0a27200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a200d1b20 0x7f0a200d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a27200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a200ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a27200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a200cddc0 con 0x7f0a201608e0 2024-03-29T09:09:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a27200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a200ce0a0 con 0x7f0a200d1b20 2024-03-29T09:09:27.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a27200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200ce380 con 0x7f0a20160cc0 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a25200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a200ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a1f400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a200d1b20 0x7f0a200d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a1f400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0a200d1b20 0x7f0a200d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:51250/0 (socket says 172.21.15.99:51250) 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a1f400700 1 -- 172.21.15.99:0/2656988860 learned_addr learned my addr 172.21.15.99:0/2656988860 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a25200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a200ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:45586/0 (socket says 172.21.15.99:45586) 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.505+0000 7f0a1fe00700 1 --1- 172.21.15.99:0/2656988860 >> v1:172.21.15.133:6789/0 conn(0x7f0a201608e0 0x7f0a200d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:51520/0 (socket says 172.21.15.99:51520) 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1f400700 1 -- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 msgr2=0x7f0a200ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1f400700 1 --2- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a200ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1f400700 1 -- 172.21.15.99:0/2656988860 >> v1:172.21.15.133:6789/0 conn(0x7f0a201608e0 legacy=0x7f0a200d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1f400700 1 -- 172.21.15.99:0/2656988860 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200ce660 con 0x7f0a200d1b20 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a25200700 1 --2- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a200ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1ea00700 1 -- 172.21.15.99:0/2656988860 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309719900 0 0) 0x7f0a200cddc0 con 0x7f0a201608e0 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1f400700 1 --2- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a200d1b20 0x7f0a200d0150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0098f0 tx=0x7f0a0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd2d282a0ad674e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1ea00700 1 -- 172.21.15.99:0/2656988860 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c011830 con 0x7f0a200d1b20 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1ea00700 1 -- 172.21.15.99:0/2656988860 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c011990 con 0x7f0a200d1b20 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.507+0000 7f0a1ea00700 1 -- 172.21.15.99:0/2656988860 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c011d20 con 0x7f0a200d1b20 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 -- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a200d1b20 msgr2=0x7f0a200d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 --2- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a200d1b20 0x7f0a200d0150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0098f0 tx=0x7f0a0c009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 -- 172.21.15.99:0/2656988860 shutdown_connections 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 --2- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a200ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 --2- 172.21.15.99:0/2656988860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a200d1b20 0x7f0a200d0150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 -- 172.21.15.99:0/2656988860 >> 172.21.15.99:0/2656988860 conn(0x7f0a2009c490 msgr2=0x7f0a200d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 -- 172.21.15.99:0/2656988860 shutdown_connections 2024-03-29T09:09:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 -- 172.21.15.99:0/2656988860 wait complete. 2024-03-29T09:09:27.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.508+0000 7f0a27200700 1 Processor -- start 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.509+0000 7f0a27200700 1 -- start start 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.509+0000 7f0a27200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a200d1b20 0x7f0a201535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.509+0000 7f0a27200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a201608e0 0x7f0a20157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.509+0000 7f0a27200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a201584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.509+0000 7f0a27200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a201f3ed0 con 0x7f0a200d1b20 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.509+0000 7f0a27200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a20157890 con 0x7f0a201608e0 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.509+0000 7f0a27200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a201f3110 con 0x7f0a20160cc0 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a200d1b20 0x7f0a201535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a200d1b20 0x7f0a201535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:48438/0 (socket says 172.21.15.99:48438) 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 -- 172.21.15.99:0/156467407 learned_addr learned my addr 172.21.15.99:0/156467407 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a25200700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a201584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1f400700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a201608e0 0x7f0a20157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 -- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 msgr2=0x7f0a201584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a201584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 -- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a201608e0 msgr2=0x7f0a20157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a201608e0 0x7f0a20157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 -- 172.21.15.99:0/156467407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18009a70 con 0x7f0a200d1b20 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.510+0000 7f0a1fe00700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a200d1b20 0x7f0a201535f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a100011e0 tx=0x7f0a1000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.511+0000 7f0a17e00700 1 -- 172.21.15.99:0/156467407 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10004c80 con 0x7f0a200d1b20 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.511+0000 7f0a17e00700 1 -- 172.21.15.99:0/156467407 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10004e00 con 0x7f0a200d1b20 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.511+0000 7f0a17e00700 1 -- 172.21.15.99:0/156467407 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10007310 con 0x7f0a200d1b20 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.511+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c009550 con 0x7f0a200d1b20 2024-03-29T09:09:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.511+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a201f36d0 con 0x7f0a200d1b20 2024-03-29T09:09:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.512+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a200677a0 con 0x7f0a200d1b20 2024-03-29T09:09:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.513+0000 7f0a17e00700 1 -- 172.21.15.99:0/156467407 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0a10007470 con 0x7f0a200d1b20 2024-03-29T09:09:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.513+0000 7f0a17e00700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0a0006ac90 0x7f0a0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.515+0000 7f0a17e00700 1 -- 172.21.15.99:0/156467407 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a10089420 con 0x7f0a200d1b20 2024-03-29T09:09:27.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.515+0000 7f0a1f400700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0a0006ac90 0x7f0a0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.515+0000 7f0a1f400700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0a0006ac90 0x7f0a0006d150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000c00 tx=0x7f0a0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.519+0000 7f0a17e00700 1 -- 172.21.15.99:0/156467407 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0a10054300 con 0x7f0a200d1b20 2024-03-29T09:09:27.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.725+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a200d0700 con 0x7f0a200d1b20 2024-03-29T09:09:27.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.726+0000 7f0a17e00700 1 -- 172.21.15.99:0/156467407 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f0a10057920 con 0x7f0a200d1b20 2024-03-29T09:09:27.725 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:27.726 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:27.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0a0006ac90 msgr2=0x7f0a0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0a0006ac90 0x7f0a0006d150 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c000c00 tx=0x7f0a0c008030 comp rx=0 tx=0).stop 2024-03-29T09:09:27.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a200d1b20 msgr2=0x7f0a201535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a200d1b20 0x7f0a201535f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0a100011e0 tx=0x7f0a1000cae0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 shutdown_connections 2024-03-29T09:09:27.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0a0006ac90 0x7f0a0006d150 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a20160cc0 0x7f0a201584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a200d1b20 0x7f0a201535f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 --2- 172.21.15.99:0/156467407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a201608e0 0x7f0a20157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 >> 172.21.15.99:0/156467407 conn(0x7f0a2009c490 msgr2=0x7f0a2015e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:27.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 shutdown_connections 2024-03-29T09:09:27.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.729+0000 7f0a27200700 1 -- 172.21.15.99:0/156467407 wait complete. 2024-03-29T09:09:27.746 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-29T09:09:27.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.939+0000 7f055e600700 1 Processor -- start 2024-03-29T09:09:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.939+0000 7f055e600700 1 -- start start 2024-03-29T09:09:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f055e600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f055815d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f055e600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f05580d0150 0x7f055815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f055e600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05580d1a50 con 0x7f05580b0770 2024-03-29T09:09:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f055e600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05580d2260 con 0x7f05580d0150 2024-03-29T09:09:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f055e600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05580d1ca0 con 0x7f05580cfd70 2024-03-29T09:09:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f0556a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f05580d0150 0x7f055815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f0557e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f05580b0770 0x7f05580b0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49578/0 (socket says 172.21.15.99:49578) 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.940+0000 7f0557e00700 1 -- 172.21.15.99:0/1856579901 learned_addr learned my addr 172.21.15.99:0/1856579901 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0557400700 1 --2- 172.21.15.99:0/1856579901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f055815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556a00700 1 -- 172.21.15.99:0/1856579901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 msgr2=0x7f055815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556a00700 1 --2- 172.21.15.99:0/1856579901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f055815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556a00700 1 -- 172.21.15.99:0/1856579901 >> v1:172.21.15.99:6789/0 conn(0x7f05580b0770 legacy=0x7f05580b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556a00700 1 -- 172.21.15.99:0/1856579901 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05580d1f80 con 0x7f05580d0150 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556000700 1 -- 172.21.15.99:0/1856579901 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318830483 0 0) 0x7f05580d1a50 con 0x7f05580b0770 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556a00700 1 --2- 172.21.15.99:0/1856579901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f055815d7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0544009840 tx=0x7f0544009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81afad72628d11d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556000700 1 -- 172.21.15.99:0/1856579901 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544011830 con 0x7f05580d0150 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556000700 1 -- 172.21.15.99:0/1856579901 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544011990 con 0x7f05580d0150 2024-03-29T09:09:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.941+0000 7f0556000700 1 -- 172.21.15.99:0/1856579901 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544011d20 con 0x7f05580d0150 2024-03-29T09:09:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 -- 172.21.15.99:0/1856579901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 msgr2=0x7f055815d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 --2- 172.21.15.99:0/1856579901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f055815d7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0544009840 tx=0x7f0544009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 -- 172.21.15.99:0/1856579901 shutdown_connections 2024-03-29T09:09:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 --2- 172.21.15.99:0/1856579901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f055815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 --2- 172.21.15.99:0/1856579901 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f055815d7b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 -- 172.21.15.99:0/1856579901 >> 172.21.15.99:0/1856579901 conn(0x7f055809c470 msgr2=0x7f05580d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 -- 172.21.15.99:0/1856579901 shutdown_connections 2024-03-29T09:09:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.942+0000 7f055e600700 1 -- 172.21.15.99:0/1856579901 wait complete. 2024-03-29T09:09:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.944+0000 7f055e600700 1 Processor -- start 2024-03-29T09:09:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.944+0000 7f055e600700 1 -- start start 2024-03-29T09:09:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f055e600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05580b0770 0x7f05581e4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f055e600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f05581f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f055e600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f05581f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f055e600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05581e2e90 con 0x7f05580d0150 2024-03-29T09:09:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f055e600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05581e2280 con 0x7f05580b0770 2024-03-29T09:09:27.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f055e600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05581e2560 con 0x7f05580cfd70 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f05581f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f05581f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53106/0 (socket says 172.21.15.99:53106) 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 -- 172.21.15.99:0/19567724 learned_addr learned my addr 172.21.15.99:0/19567724 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557400700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05580b0770 0x7f05581e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0556a00700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f05581f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 -- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 msgr2=0x7f05581f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f05581f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 -- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05580b0770 msgr2=0x7f05581e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05580b0770 0x7f05581e4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.946+0000 7f0557e00700 1 -- 172.21.15.99:0/19567724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440094a0 con 0x7f05580d0150 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.947+0000 7f0557e00700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f05581f3320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0550002230 tx=0x7f055000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.947+0000 7f054f400700 1 -- 172.21.15.99:0/19567724 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000aac0 con 0x7f05580d0150 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.947+0000 7f054f400700 1 -- 172.21.15.99:0/19567724 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055000ac40 con 0x7f05580d0150 2024-03-29T09:09:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.947+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05581e2840 con 0x7f05580d0150 2024-03-29T09:09:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.947+0000 7f054f400700 1 -- 172.21.15.99:0/19567724 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500070d0 con 0x7f05580d0150 2024-03-29T09:09:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.947+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05581e4780 con 0x7f05580d0150 2024-03-29T09:09:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.948+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05580b4a30 con 0x7f05580d0150 2024-03-29T09:09:27.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.949+0000 7f054f400700 1 -- 172.21.15.99:0/19567724 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0550007250 con 0x7f05580d0150 2024-03-29T09:09:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.950+0000 7f054f400700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f053806ac90 0x7f053806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.950+0000 7f054f400700 1 -- 172.21.15.99:0/19567724 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f05500893f0 con 0x7f05580d0150 2024-03-29T09:09:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.950+0000 7f0557400700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f053806ac90 0x7f053806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:27.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.950+0000 7f0557400700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f053806ac90 0x7f053806d150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f05580b81c0 tx=0x7f0548008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:27.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:27.955+0000 7f054f400700 1 -- 172.21.15.99:0/19567724 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f05500542d0 con 0x7f05580d0150 2024-03-29T09:09:28.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.157+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0558067d80 con 0x7f05580d0150 2024-03-29T09:09:28.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.158+0000 7f054f400700 1 -- 172.21.15.99:0/19567724 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f05500578f0 con 0x7f05580d0150 2024-03-29T09:09:28.158 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:28.158 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f053806ac90 msgr2=0x7f053806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f053806ac90 0x7f053806d150 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f05580b81c0 tx=0x7f0548008040 comp rx=0 tx=0).stop 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 msgr2=0x7f05581f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f05581f3320 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0550002230 tx=0x7f055000cf50 comp rx=0 tx=0).stop 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 shutdown_connections 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f053806ac90 0x7f053806d150 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05580d0150 0x7f05581f3320 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05580b0770 0x7f05581e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 --2- 172.21.15.99:0/19567724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05580cfd70 0x7f05581f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 >> 172.21.15.99:0/19567724 conn(0x7f055809c470 msgr2=0x7f05580b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 shutdown_connections 2024-03-29T09:09:28.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.161+0000 7f055e600700 1 -- 172.21.15.99:0/19567724 wait complete. 2024-03-29T09:09:28.178 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-29T09:09:28.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-29T09:09:28.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-29T09:09:28.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-29T09:09:28.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-29T09:09:28.180 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-29T09:09:28.181 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-29T09:09:28.181 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-29T09:09:28.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.519+0000 7f75c6c00700 1 Processor -- start 2024-03-29T09:09:28.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.520+0000 7f75c6c00700 1 -- start start 2024-03-29T09:09:28.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.522+0000 7f75c6c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c015a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.522+0000 7f75c6c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c0071010 con 0x7f75c00b16e0 2024-03-29T09:09:28.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.522+0000 7f75c6c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c00706e0 con 0x7f75c0072330 2024-03-29T09:09:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.522+0000 7f75c6c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c015a970 con 0x7f75c0157070 2024-03-29T09:09:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.523+0000 7f75bfe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f75c00b16e0 0x7f75c0071c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49592/0 (socket says 172.21.15.99:49592) 2024-03-29T09:09:28.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.524+0000 7f75bfe00700 1 -- 172.21.15.99:0/3545797006 learned_addr learned my addr 172.21.15.99:0/3545797006 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:28.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.525+0000 7f75c4c00700 1 --2- 172.21.15.99:0/3545797006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.525+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700048292 0 0) 0x7f75c00706e0 con 0x7f75c0072330 2024-03-29T09:09:28.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.525+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a40035b0 con 0x7f75c0072330 2024-03-29T09:09:28.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.525+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884317537 0 0) 0x7f75c0071010 con 0x7f75c00b16e0 2024-03-29T09:09:28.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.525+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c00706e0 con 0x7f75c00b16e0 2024-03-29T09:09:28.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.526+0000 7f75c4c00700 1 -- 172.21.15.99:0/3545797006 >> v1:172.21.15.133:6789/0 conn(0x7f75c0072330 legacy=0x7f75c00cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.526+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2680340213 0 0) 0x7f75a40035b0 con 0x7f75c0072330 2024-03-29T09:09:28.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.526+0000 7f75c4c00700 1 -- 172.21.15.99:0/3545797006 >> v1:172.21.15.99:6789/0 conn(0x7f75c00b16e0 legacy=0x7f75c0071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.526+0000 7f75c4c00700 1 -- 172.21.15.99:0/3545797006 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00709c0 con 0x7f75c0157070 2024-03-29T09:09:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.527+0000 7f75c4c00700 1 --2- 172.21.15.99:0/3545797006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c015a390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009dc0 tx=0x7f75b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55709d1ad2f3990b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.528+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b800e070 con 0x7f75c0157070 2024-03-29T09:09:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.528+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b8003a20 con 0x7f75c0157070 2024-03-29T09:09:28.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.528+0000 7f75bf400700 1 -- 172.21.15.99:0/3545797006 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8001670 con 0x7f75c0157070 2024-03-29T09:09:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.529+0000 7f0a1a600700 1 Processor -- start 2024-03-29T09:09:28.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.529+0000 7f75c6c00700 1 -- 172.21.15.99:0/3545797006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 msgr2=0x7f75c015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.529+0000 7f75c6c00700 1 --2- 172.21.15.99:0/3545797006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c015a390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009dc0 tx=0x7f75b800b330 comp rx=0 tx=0).stop 2024-03-29T09:09:28.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7f8d12200700 1 Processor -- start 2024-03-29T09:09:28.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7fb774c00700 1 Processor -- start 2024-03-29T09:09:28.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7fb774c00700 1 -- start start 2024-03-29T09:09:28.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7f0a1a600700 1 -- start start 2024-03-29T09:09:28.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7f0a1a600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0a140b16e0 0x7f0a1415a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7f8d12200700 1 -- start start 2024-03-29T09:09:28.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7f8d12200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d0c0b1780 0x7f8d0c168350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7f8d12200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c0d1a50 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7f8d12200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c0d2260 con 0x7f8d0c15d980 2024-03-29T09:09:28.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7f8d12200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c0d1ca0 con 0x7f8d0c0b1780 2024-03-29T09:09:28.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7f0a1a600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a14071010 con 0x7f0a140ce3c0 2024-03-29T09:09:28.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7fb774c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7680e7490 0x7fb76805b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.532+0000 7fb774c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb768058120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f0a1a600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140709c0 con 0x7f0a140714a0 2024-03-29T09:09:28.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f0a1a600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1415a970 con 0x7f0a140b16e0 2024-03-29T09:09:28.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f0a13400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0a140b16e0 0x7f0a1415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f8d0be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d0c0b1780 0x7f8d0c168350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f0a13400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0a140b16e0 0x7f0a1415a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53122/0 (socket says 172.21.15.99:53122) 2024-03-29T09:09:28.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f0a13400700 1 -- 172.21.15.99:0/2310983604 learned_addr learned my addr 172.21.15.99:0/2310983604 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f8d0be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d0c0b1780 0x7f8d0c168350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53126/0 (socket says 172.21.15.99:53126) 2024-03-29T09:09:28.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f8d0be00700 1 -- 172.21.15.99:0/3047259512 learned_addr learned my addr 172.21.15.99:0/3047259512 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7f75c6c00700 1 -- 172.21.15.99:0/3545797006 shutdown_connections 2024-03-29T09:09:28.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7f75c6c00700 1 --2- 172.21.15.99:0/3545797006 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c015a390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.530+0000 7f75c6c00700 1 -- 172.21.15.99:0/3545797006 >> 172.21.15.99:0/3545797006 conn(0x7f75c00b0bc0 msgr2=0x7f75c00b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7fb76f400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7680e7490 0x7fb76805b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f75c6c00700 1 -- 172.21.15.99:0/3545797006 shutdown_connections 2024-03-29T09:09:28.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7f75c6c00700 1 -- 172.21.15.99:0/3545797006 wait complete. 2024-03-29T09:09:28.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7fb76f400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7680e7490 0x7fb76805b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55190/0 (socket says 172.21.15.99:55190) 2024-03-29T09:09:28.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.533+0000 7fb774c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb76805b9d0 0x7fb768058660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7f8d0be00700 1 -- 172.21.15.99:0/3047259512 >> v1:172.21.15.133:6789/0 conn(0x7f8d0c15d980 legacy=0x7f8d0c0cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7f8d0be00700 1 -- 172.21.15.99:0/3047259512 >> v1:172.21.15.99:6789/0 conn(0x7f8d0c0cfd70 legacy=0x7f8d0c15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:28.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb774c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76805bee0 con 0x7fb76805b9d0 2024-03-29T09:09:28.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7f8d0be00700 1 -- 172.21.15.99:0/3047259512 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0d1f80 con 0x7f8d0c0b1780 2024-03-29T09:09:28.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.535+0000 7f0a12000700 1 -- 172.21.15.99:0/2310983604 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954582952 0 0) 0x7f0a14071010 con 0x7f0a140ce3c0 2024-03-29T09:09:28.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.535+0000 7f0a12000700 1 -- 172.21.15.99:0/2310983604 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f80035b0 con 0x7f0a140ce3c0 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb76f400700 1 -- 172.21.15.99:0/3088594832 learned_addr learned my addr 172.21.15.99:0/3088594832 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb774c00700 1 -- 172.21.15.99:0/3088594832 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76805c1c0 con 0x7fb7680e7490 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb774c00700 1 -- 172.21.15.99:0/3088594832 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76805c4a0 con 0x7fb7680e7870 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb76ea00700 1 --2- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb768058120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb76f400700 1 -- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 msgr2=0x7fb768058120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb76f400700 1 --2- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb768058120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb76f400700 1 -- 172.21.15.99:0/3088594832 >> v2:172.21.15.99:3300/0 conn(0x7fb76805b9d0 msgr2=0x7fb768058660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb76f400700 1 --2- 172.21.15.99:0/3088594832 >> v2:172.21.15.99:3300/0 conn(0x7fb76805b9d0 0x7fb768058660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.534+0000 7fb76f400700 1 -- 172.21.15.99:0/3088594832 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76805c780 con 0x7fb7680e7490 2024-03-29T09:09:28.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.535+0000 7fb76f400700 1 --2- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7680e7490 0x7fb76805b490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb76800cb10 tx=0x7fb764009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7066ba910f658f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.535+0000 7f8d0be00700 1 --2- 172.21.15.99:0/3047259512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0b1780 0x7f8d0c168350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35121f099d1f8efb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.536+0000 7f0a13400700 1 -- 172.21.15.99:0/2310983604 >> v1:172.21.15.133:6789/0 conn(0x7f0a140714a0 legacy=0x7f0a14071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.536+0000 7f0a13400700 1 -- 172.21.15.99:0/2310983604 >> v1:172.21.15.99:6789/0 conn(0x7f0a140ce3c0 legacy=0x7f0a14156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.536+0000 7f0a13400700 1 -- 172.21.15.99:0/2310983604 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14070ca0 con 0x7f0a140b16e0 2024-03-29T09:09:28.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.536+0000 7f8d0a000700 1 -- 172.21.15.99:0/3047259512 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400e070 con 0x7f8d0c0b1780 2024-03-29T09:09:28.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.536+0000 7f0a13400700 1 --2- 172.21.15.99:0/2310983604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 0x7f0a1415a390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a040098f0 tx=0x7f0a04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbccf9756d147b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7f0a12000700 1 -- 172.21.15.99:0/2310983604 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04011830 con 0x7f0a140b16e0 2024-03-29T09:09:28.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7f8d0a000700 1 -- 172.21.15.99:0/3047259512 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0400b9c0 con 0x7f8d0c0b1780 2024-03-29T09:09:28.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7f8d0a000700 1 -- 172.21.15.99:0/3047259512 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400bd70 con 0x7f8d0c0b1780 2024-03-29T09:09:28.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7f0a12000700 1 -- 172.21.15.99:0/2310983604 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a04011990 con 0x7f0a140b16e0 2024-03-29T09:09:28.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7f0a12000700 1 -- 172.21.15.99:0/2310983604 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04011830 con 0x7f0a140b16e0 2024-03-29T09:09:28.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7f75c6c00700 1 Processor -- start 2024-03-29T09:09:28.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7fb76e000700 1 -- 172.21.15.99:0/3088594832 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400b870 con 0x7fb7680e7490 2024-03-29T09:09:28.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7fb76e000700 1 -- 172.21.15.99:0/3088594832 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76400b9d0 con 0x7fb7680e7490 2024-03-29T09:09:28.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7fb76e000700 1 -- 172.21.15.99:0/3088594832 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400bd60 con 0x7fb7680e7490 2024-03-29T09:09:28.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.537+0000 7f75c6c00700 1 -- start start 2024-03-29T09:09:28.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7f8d12200700 1 -- 172.21.15.99:0/3047259512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0b1780 msgr2=0x7f8d0c168350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7f8d12200700 1 --2- 172.21.15.99:0/3047259512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0b1780 0x7f8d0c168350 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:28.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7f0a1a600700 1 -- 172.21.15.99:0/2310983604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 msgr2=0x7f0a1415a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7f0a1a600700 1 --2- 172.21.15.99:0/2310983604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 0x7f0a1415a390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0a040098f0 tx=0x7f0a04009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:28.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7fb774c00700 1 -- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7680e7490 msgr2=0x7fb76805b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7fb774c00700 1 --2- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7680e7490 0x7fb76805b490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb76800cb10 tx=0x7fb764009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7f8d12200700 1 -- 172.21.15.99:0/3047259512 shutdown_connections 2024-03-29T09:09:28.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7f8d12200700 1 --2- 172.21.15.99:0/3047259512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0b1780 0x7f8d0c168350 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.538+0000 7f8d12200700 1 -- 172.21.15.99:0/3047259512 >> 172.21.15.99:0/3047259512 conn(0x7f8d0c09c470 msgr2=0x7f8d0c0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f8d12200700 1 -- 172.21.15.99:0/3047259512 shutdown_connections 2024-03-29T09:09:28.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f0a1a600700 1 -- 172.21.15.99:0/2310983604 shutdown_connections 2024-03-29T09:09:28.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f0a1a600700 1 --2- 172.21.15.99:0/2310983604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 0x7f0a1415a390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f0a1a600700 1 -- 172.21.15.99:0/2310983604 >> 172.21.15.99:0/2310983604 conn(0x7f0a140b0bc0 msgr2=0x7f0a140b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f75c6c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0072330 0x7f75c00b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f75c6c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75c00b16e0 0x7f75c01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f0a1a600700 1 -- 172.21.15.99:0/2310983604 shutdown_connections 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75c6c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c013b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75bfe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0072330 0x7f75c00b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75bfe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0072330 0x7f75c00b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55192/0 (socket says 172.21.15.99:55192) 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75c00b16e0 0x7f75c01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75bfe00700 1 -- 172.21.15.99:0/1970861631 learned_addr learned my addr 172.21.15.99:0/1970861631 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00b2b50 con 0x7f75c00b16e0 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0149520 con 0x7f75c0072330 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00b2d60 con 0x7f75c0157070 2024-03-29T09:09:28.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7fb774c00700 1 -- 172.21.15.99:0/3088594832 shutdown_connections 2024-03-29T09:09:28.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7fb774c00700 1 --2- 172.21.15.99:0/3088594832 >> v2:172.21.15.99:3300/0 conn(0x7fb76805b9d0 0x7fb768058660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7fb774c00700 1 --2- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7680e7490 0x7fb76805b490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7fb774c00700 1 --2- 172.21.15.99:0/3088594832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb768058120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.540+0000 7fb774c00700 1 -- 172.21.15.99:0/3088594832 >> 172.21.15.99:0/3088594832 conn(0x7fb768059dd0 msgr2=0x7fb76805a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f0a1a600700 1 -- 172.21.15.99:0/2310983604 wait complete. 2024-03-29T09:09:28.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.541+0000 7f75c4c00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c013b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.539+0000 7f8d12200700 1 -- 172.21.15.99:0/3047259512 wait complete. 2024-03-29T09:09:28.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.541+0000 7fb774c00700 1 -- 172.21.15.99:0/3088594832 shutdown_connections 2024-03-29T09:09:28.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.542+0000 7fb774c00700 1 -- 172.21.15.99:0/3088594832 wait complete. 2024-03-29T09:09:28.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.541+0000 7f75bfe00700 1 -- 172.21.15.99:0/1970861631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 msgr2=0x7f75c013b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.541+0000 7f75bfe00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75c0157070 0x7f75c013b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.542+0000 7f75bfe00700 1 -- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75c00b16e0 msgr2=0x7f75c01589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.542+0000 7f75bfe00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75c00b16e0 0x7f75c01589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.542+0000 7f75bfe00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b8009a70 con 0x7f75c0072330 2024-03-29T09:09:28.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.542+0000 7f8d12200700 1 Processor -- start 2024-03-29T09:09:28.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.542+0000 7f75bfe00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0072330 0x7f75c00b2610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c6b0 tx=0x7f75b000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.543+0000 7f0a1a600700 1 Processor -- start 2024-03-29T09:09:28.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.543+0000 7f0a1a600700 1 -- start start 2024-03-29T09:09:28.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.543+0000 7f8d12200700 1 -- start start 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.544+0000 7f0a1a600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a140714a0 0x7f0a1415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.544+0000 7f0a1a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 0x7f0a141590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a1a600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a140ce3c0 0x7f0a1415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a1a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415baa0 con 0x7f0a140b16e0 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a1a600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415aab0 con 0x7f0a140ce3c0 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a1a600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1414eff0 con 0x7f0a140714a0 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a13400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a140714a0 0x7f0a1415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7fb774c00700 1 Processor -- start 2024-03-29T09:09:28.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a13400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a140714a0 0x7f0a1415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:60986/0 (socket says 172.21.15.99:60986) 2024-03-29T09:09:28.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a13400700 1 -- 172.21.15.99:0/283050923 learned_addr learned my addr 172.21.15.99:0/283050923 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a12a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 0x7f0a141590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a13e00700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a140ce3c0 0x7f0a1415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f7717200700 1 Processor -- start 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0005d70 con 0x7f75c0072330 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b00070d0 con 0x7f75c0072330 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0007480 con 0x7f75c0072330 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a13400700 1 -- 172.21.15.99:0/283050923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a140ce3c0 msgr2=0x7f0a1415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7f0a13400700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0a140ce3c0 0x7f0a1415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a13400700 1 -- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 msgr2=0x7f0a141590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a13400700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 0x7f0a141590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a13400700 1 -- 172.21.15.99:0/283050923 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009550 con 0x7f0a140714a0 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a12a00700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a140b16e0 0x7f0a141590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:28.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a13400700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a140714a0 0x7f0a1415b4c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.545+0000 7fb774c00700 1 -- start start 2024-03-29T09:09:28.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04011830 con 0x7f0a140714a0 2024-03-29T09:09:28.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0401f070 con 0x7f0a140714a0 2024-03-29T09:09:28.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0401adc0 con 0x7f0a140714a0 2024-03-29T09:09:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c00b3040 con 0x7f75c0072330 2024-03-29T09:09:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c00a84a0 con 0x7f75c0072330 2024-03-29T09:09:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.547+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1415d950 con 0x7f0a140714a0 2024-03-29T09:09:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.547+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1415afa0 con 0x7f0a140714a0 2024-03-29T09:09:28.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.547+0000 7fb774c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76805b9d0 0x7fb768180be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.546+0000 7f7717200700 1 -- start start 2024-03-29T09:09:28.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a140677a0 con 0x7f0a140714a0 2024-03-29T09:09:28.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb76f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76805b9d0 0x7fb768180be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb76f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76805b9d0 0x7fb768180be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55218/0 (socket says 172.21.15.99:55218) 2024-03-29T09:09:28.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb76f400700 1 -- 172.21.15.99:0/1303997821 learned_addr learned my addr 172.21.15.99:0/1303997821 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.547+0000 7f7717200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b3b20 0x7f771806c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f7717200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77181623c0 con 0x7f771806ce60 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f7717200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77181626a0 con 0x7f77180b21e0 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f7717200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77181621d0 con 0x7f77180b3b20 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f770d400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b3b20 0x7f771806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f770d400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b3b20 0x7f771806c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:60998/0 (socket says 172.21.15.99:60998) 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f7715e00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f77180b21e0 0x7f7718161ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:52014/0 (socket says 172.21.15.99:52014) 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f7715e00700 1 -- 172.21.15.99:0/3064563824 learned_addr learned my addr 172.21.15.99:0/3064563824 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:28.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7598000fc0 con 0x7f75c0072330 2024-03-29T09:09:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb774c00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7680e7490 0x7fb768055f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb774c00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb7680e4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768056730 con 0x7fb7680e7490 2024-03-29T09:09:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768054550 con 0x7fb76805b9d0 2024-03-29T09:09:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb768056490 con 0x7fb7680e7870 2024-03-29T09:09:28.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f8d12200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d0c0b1780 0x7f8d0c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f8d12200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0cfd70 0x7f8d0c0c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f8d12200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c15d980 0x7f8d0c0b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f8d12200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0c7d80 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f8d12200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0c8060 con 0x7f8d0c15d980 2024-03-29T09:09:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.548+0000 7f8d12200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0cba80 con 0x7f8d0c0b1780 2024-03-29T09:09:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7f770d400700 1 -- 172.21.15.99:0/3064563824 >> v1:172.21.15.133:6789/0 conn(0x7f77180b21e0 legacy=0x7f7718161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76fe00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb7680e4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76f400700 1 -- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 msgr2=0x7fb7680e4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76f400700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb7680e4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76f400700 1 -- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7680e7490 msgr2=0x7fb768055f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76f400700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7680e7490 0x7fb768055f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76f400700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764009550 con 0x7fb76805b9d0 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76fe00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7680e7870 0x7fb7680e4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7fb76f400700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76805b9d0 0x7fb768180be0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb76805c300 tx=0x7fb764004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7f8d0b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d0c0b1780 0x7f8d0c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7f8d0aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0cfd70 0x7f8d0c0c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7f8d0aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0cfd70 0x7f8d0c0c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53158/0 (socket says 172.21.15.99:53158) 2024-03-29T09:09:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.549+0000 7f8d0aa00700 1 -- 172.21.15.99:0/4005978444 learned_addr learned my addr 172.21.15.99:0/4005978444 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.550+0000 7f8d0be00700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c15d980 0x7f8d0c0b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.550+0000 7f770d400700 1 -- 172.21.15.99:0/3064563824 >> v1:172.21.15.99:6789/0 conn(0x7f771806ce60 legacy=0x7f77180abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.550+0000 7f770d400700 1 -- 172.21.15.99:0/3064563824 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718162980 con 0x7f77180b3b20 2024-03-29T09:09:28.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.550+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7640038b0 con 0x7fb76805b9d0 2024-03-29T09:09:28.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.550+0000 7f770d400700 1 --2- 172.21.15.99:0/3064563824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b3b20 0x7f771806c920 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7700009dc0 tx=0x7f770000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e9957d7106d625 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.550+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb764003a30 con 0x7fb76805b9d0 2024-03-29T09:09:28.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0a0401b430 con 0x7f0a140714a0 2024-03-29T09:09:28.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7680547f0 con 0x7fb76805b9d0 2024-03-29T09:09:28.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb768054db0 con 0x7fb76805b9d0 2024-03-29T09:09:28.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f7715400700 1 -- 172.21.15.99:0/3064563824 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000e070 con 0x7f77180b3b20 2024-03-29T09:09:28.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f8d0aa00700 1 -- 172.21.15.99:0/4005978444 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d0c0b1780 msgr2=0x7f8d0c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f8d0aa00700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d0c0b1780 0x7f8d0c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f8d0aa00700 1 -- 172.21.15.99:0/4005978444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c15d980 msgr2=0x7f8d0c0b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f8d0aa00700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d0c15d980 0x7f8d0c0b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f75b00075e0 con 0x7f75c0072330 2024-03-29T09:09:28.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f8d0aa00700 1 -- 172.21.15.99:0/4005978444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04009b30 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f7715400700 1 -- 172.21.15.99:0/3064563824 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770000bad0 con 0x7f77180b3b20 2024-03-29T09:09:28.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.551+0000 7f7715400700 1 -- 172.21.15.99:0/3064563824 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770000be80 con 0x7f77180b3b20 2024-03-29T09:09:28.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.552+0000 7f8d0aa00700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0cfd70 0x7f8d0c0c7410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00c450 tx=0x7f8cfc00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.552+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb764003be0 con 0x7fb76805b9d0 2024-03-29T09:09:28.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.552+0000 7f7717200700 1 -- 172.21.15.99:0/3064563824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b3b20 msgr2=0x7f771806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.552+0000 7f7717200700 1 --2- 172.21.15.99:0/3064563824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b3b20 0x7f771806c920 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7700009dc0 tx=0x7f770000b330 comp rx=0 tx=0).stop 2024-03-29T09:09:28.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.552+0000 7f41e3000700 1 Processor -- start 2024-03-29T09:09:28.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0052d0 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc005430 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.552+0000 7f0a0b400700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09f406f0a0 0x7f09f4071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.552+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0a0408b1e0 con 0x7f0a140714a0 2024-03-29T09:09:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f0a0b400700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7f09f4078660 0x7f09f407aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 --> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09f407b170 con 0x7f09f4078660 2024-03-29T09:09:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb74c000fc0 con 0x7fb76805b9d0 2024-03-29T09:09:28.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a0408b550 con 0x7f0a140714a0 2024-03-29T09:09:28.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f75bcc00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f759c06ace0 0x7f759c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f41e3000700 1 -- start start 2024-03-29T09:09:28.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f0a12a00700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09f406f0a0 0x7f09f4071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c0cbd60 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.553+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c1549e0 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.554+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0038c0 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.554+0000 7f75b7400700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f759c06ace0 0x7f759c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f75b7400700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f759c06ace0 0x7f759c06d1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75c0146fb0 tx=0x7f75a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f0a12a00700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09f406f0a0 0x7f09f4071560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0a14156960 tx=0x7f0a0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f41e3000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc15d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f41e3000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc0d1a50 con 0x7f41dc0d0150 2024-03-29T09:09:28.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f41e3000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41dc0d2260 con 0x7f41dc15d7b0 2024-03-29T09:09:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f41e3000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc15dc00 con 0x7f41dc0cfd70 2024-03-29T09:09:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f41dbe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f41dbe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc15d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32784/0 (socket says 172.21.15.99:32784) 2024-03-29T09:09:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f41dbe00700 1 -- 172.21.15.99:0/2302974141 learned_addr learned my addr 172.21.15.99:0/2302974141 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.556+0000 7f61eee00700 1 Processor -- start 2024-03-29T09:09:28.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.556+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d0c0b2440 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f7717200700 1 -- 172.21.15.99:0/3064563824 shutdown_connections 2024-03-29T09:09:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f7717200700 1 --2- 172.21.15.99:0/3064563824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b3b20 0x7f771806c920 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.555+0000 7f7717200700 1 -- 172.21.15.99:0/3064563824 >> 172.21.15.99:0/3064563824 conn(0x7f7718066960 msgr2=0x7f7718066d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.556+0000 7f61eee00700 1 -- start start 2024-03-29T09:09:28.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.556+0000 7f41dbe00700 1 -- 172.21.15.99:0/2302974141 >> v1:172.21.15.133:6789/0 conn(0x7f41dc15d7b0 legacy=0x7f41dc0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.556+0000 7f41dbe00700 1 -- 172.21.15.99:0/2302974141 >> v1:172.21.15.99:6789/0 conn(0x7f41dc0d0150 legacy=0x7f41dc0b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.556+0000 7f41dbe00700 1 -- 172.21.15.99:0/2302974141 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0d1ca0 con 0x7f41dc0cfd70 2024-03-29T09:09:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f41dbe00700 1 --2- 172.21.15.99:0/2302974141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc15d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009920 tx=0x7f41c4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=230695fa7ea73123 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f75b0089170 con 0x7f75c0072330 2024-03-29T09:09:28.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f75bcc00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7f759c0742a0 0x7f759c0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f41db400700 1 -- 172.21.15.99:0/2302974141 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4004230 con 0x7f41dc0cfd70 2024-03-29T09:09:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f41db400700 1 -- 172.21.15.99:0/2302974141 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c4004390 con 0x7f41dc0cfd70 2024-03-29T09:09:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f759c076db0 con 0x7f759c0742a0 2024-03-29T09:09:28.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f41db400700 1 -- 172.21.15.99:0/2302974141 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4009e70 con 0x7f41dc0cfd70 2024-03-29T09:09:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75b0089610 con 0x7f75c0072330 2024-03-29T09:09:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f0a13e00700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7f09f4078660 0x7f09f407aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f75c4c00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7f759c0742a0 0x7f759c0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f7717200700 1 -- 172.21.15.99:0/3064563824 shutdown_connections 2024-03-29T09:09:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f75c4c00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7f759c0742a0 0x7f759c0766e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8cfc003a20 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f61eee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e00576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f41e3000700 1 -- 172.21.15.99:0/2302974141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 msgr2=0x7f41dc15d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f41e3000700 1 --2- 172.21.15.99:0/2302974141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc15d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009920 tx=0x7f41c4009c30 comp rx=0 tx=0).stop 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f41e3000700 1 -- 172.21.15.99:0/2302974141 shutdown_connections 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f41e3000700 1 --2- 172.21.15.99:0/2302974141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc15d270 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f41e3000700 1 -- 172.21.15.99:0/2302974141 >> 172.21.15.99:0/2302974141 conn(0x7f41dc09c470 msgr2=0x7f41dc0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f61eee00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e0038080 con 0x7f61e003bdb0 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f61eee00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e0038360 con 0x7f61e00e4560 2024-03-29T09:09:28.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f61eee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0038640 con 0x7f61e00e4940 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.556+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb76402c440 con 0x7fb76805b9d0 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7fb763400700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb75006ac90 0x7fb75006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb76408ab80 con 0x7fb76805b9d0 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7fb763400700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fb7500742d0 0x7fb750076710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb750076de0 con 0x7fb7500742d0 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb76408f2a0 con 0x7fb76805b9d0 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.557+0000 7fb76ea00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb75006ac90 0x7fb75006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7fb76ea00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb75006ac90 0x7fb75006d150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb768041060 tx=0x7fb75c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f61ee400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f61e003bdb0 0x7f61e003c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49632/0 (socket says 172.21.15.99:49632) 2024-03-29T09:09:28.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f61ee400700 1 -- 172.21.15.99:0/2373715484 learned_addr learned my addr 172.21.15.99:0/2373715484 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:28.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f7446800700 1 Processor -- start 2024-03-29T09:09:28.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.559+0000 7f61ed000700 1 --2- 172.21.15.99:0/2373715484 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e00576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f61ed000700 1 -- 172.21.15.99:0/2373715484 >> v1:172.21.15.133:6789/0 conn(0x7f61e00e4560 legacy=0x7f61e00549a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T09:09:28.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f61ed000700 1 -- 172.21.15.99:0/2373715484 >> v1:172.21.15.99:6789/0 conn(0x7f61e003bdb0 legacy=0x7f61e003c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f61ed000700 1 -- 172.21.15.99:0/2373715484 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0007340 con 0x7f61e00e4940 2024-03-29T09:09:28.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7fb76fe00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fb7500742d0 0x7fb750076710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f61ed000700 1 --2- 172.21.15.99:0/2373715484 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e00576f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61e8050300 tx=0x7f61e80624f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c543b0e4a98040 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.558+0000 7f7717200700 1 -- 172.21.15.99:0/3064563824 wait complete. 2024-03-29T09:09:28.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f8d03400700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ce806ace0 0x7f8ce806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f7717200700 1 Processor -- start 2024-03-29T09:09:28.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8cfc088930 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.560+0000 7f7446800700 1 -- start start 2024-03-29T09:09:28.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f41e3000700 1 -- 172.21.15.99:0/2302974141 shutdown_connections 2024-03-29T09:09:28.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f7717200700 1 -- start start 2024-03-29T09:09:28.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f61e7e00700 1 -- 172.21.15.99:0/2373715484 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e806bb50 con 0x7f61e00e4940 2024-03-29T09:09:28.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f61e7e00700 1 -- 172.21.15.99:0/2373715484 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e806bcb0 con 0x7f61e00e4940 2024-03-29T09:09:28.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f61e7e00700 1 -- 172.21.15.99:0/2373715484 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80732b0 con 0x7f61e00e4940 2024-03-29T09:09:28.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f41e3000700 1 -- 172.21.15.99:0/2302974141 wait complete. 2024-03-29T09:09:28.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f0a13e00700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7f09f4078660 0x7f09f407aaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f8d03400700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f8ce8074320 0x7f8ce8076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 --> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ce8076e30 con 0x7f8ce8074320 2024-03-29T09:09:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cfc08d2c0 con 0x7f8d0c0cfd70 2024-03-29T09:09:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.561+0000 7f8d0b400700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ce806ace0 0x7f8ce806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7fb76fe00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fb7500742d0 0x7fb750076710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f7446800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f744015d980 0x7f74400cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f7446800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f61eee00700 1 -- 172.21.15.99:0/2373715484 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 msgr2=0x7f61e00576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f61eee00700 1 --2- 172.21.15.99:0/2373715484 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e00576f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f61e8050300 tx=0x7f61e80624f0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f8d0be00700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f8ce8074320 0x7f8ce8076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f743f400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f74400cfd70 0x7f744015d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:52046/0 (socket says 172.21.15.99:52046) 2024-03-29T09:09:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f743f400700 1 -- 172.21.15.99:0/3109662864 learned_addr learned my addr 172.21.15.99:0/3109662864 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74400d1a50 con 0x7f74400cfd70 2024-03-29T09:09:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74400d2260 con 0x7f744015d980 2024-03-29T09:09:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74400d1ca0 con 0x7f74400b1780 2024-03-29T09:09:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== osd.0 v2:172.21.15.99:6824/42219 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f759c076db0 con 0x7f759c0742a0 2024-03-29T09:09:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f8d0be00700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f8ce8074320 0x7f8ce8076760 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f8d0b400700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ce806ace0 0x7f8ce806d1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4007fd0 tx=0x7f8cf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f7717200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f771806ce60 0x7f7718146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f7717200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b21e0 0x7f77181eb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f7717200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77180b3b20 0x7f771809b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f7717200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718147b90 con 0x7f771806ce60 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f7717200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77181477d0 con 0x7f77180b3b20 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f7717200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77181ec3a0 con 0x7f77180b21e0 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b21e0 0x7f77181eb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b21e0 0x7f77181eb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32804/0 (socket says 172.21.15.99:32804) 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 -- 172.21.15.99:0/1236779914 learned_addr learned my addr 172.21.15.99:0/1236779914 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f7716800700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77180b3b20 0x7f771809b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f743ea00700 1 --2- 172.21.15.99:0/3109662864 >> v2:172.21.15.133:3300/0 conn(0x7f744015d980 0x7f74400cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.563+0000 7f743fe00700 1 --2- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f61eee00700 1 -- 172.21.15.99:0/2373715484 shutdown_connections 2024-03-29T09:09:28.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f61eee00700 1 --2- 172.21.15.99:0/2373715484 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e00576f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f61eee00700 1 -- 172.21.15.99:0/2373715484 >> 172.21.15.99:0/2373715484 conn(0x7f61e00200a0 msgr2=0x7f61e0056dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f61eee00700 1 -- 172.21.15.99:0/2373715484 shutdown_connections 2024-03-29T09:09:28.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.562+0000 7f61eee00700 1 -- 172.21.15.99:0/2373715484 wait complete. 2024-03-29T09:09:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f41e3000700 1 Processor -- start 2024-03-29T09:09:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== osd.2 v2:172.21.15.99:6814/42218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb750076de0 con 0x7fb7500742d0 2024-03-29T09:09:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f41e3000700 1 -- start start 2024-03-29T09:09:28.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== osd.1 v2:172.21.15.99:6808/42220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f09f407b170 con 0x7f09f4078660 2024-03-29T09:09:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 -- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77180b3b20 msgr2=0x7f771809b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77180b3b20 0x7f771809b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 -- 172.21.15.99:0/1236779914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f771806ce60 msgr2=0x7f7718146e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f771806ce60 0x7f7718146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 -- 172.21.15.99:0/1236779914 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7700009a70 con 0x7f77180b21e0 2024-03-29T09:09:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.564+0000 7f770d400700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b21e0 0x7f77181eb900 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f771806e3b0 tx=0x7f7700004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f61eee00700 1 Processor -- start 2024-03-29T09:09:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f41e3000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc14a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f41e3000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc0d0150 0x7f41dc0cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f41e3000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41dc15d7b0 0x7f41dc0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f41e3000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0c95e0 con 0x7f41dc15d7b0 2024-03-29T09:09:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f41e3000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc14aa90 con 0x7f41dc0d0150 2024-03-29T09:09:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f41e3000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0c8c40 con 0x7f41dc0cfd70 2024-03-29T09:09:28.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.565+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700004800 con 0x7f77180b21e0 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.565+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7700004960 con 0x7f77180b21e0 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.565+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700012670 con 0x7f77180b21e0 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.565+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77181ec680 con 0x7f77180b21e0 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.565+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7718157b70 con 0x7f77180b21e0 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f771806c400 con 0x7f77180b21e0 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f743e000700 1 -- 172.21.15.99:0/3109662864 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134217682 0 0) 0x7f74400d1a50 con 0x7f74400cfd70 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.566+0000 7f743e000700 1 -- 172.21.15.99:0/3109662864 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74240035b0 con 0x7f74400cfd70 2024-03-29T09:09:28.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f743ea00700 1 -- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 msgr2=0x7f74400ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f743ea00700 1 --2- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f743ea00700 1 -- 172.21.15.99:0/3109662864 >> v1:172.21.15.133:6789/0 conn(0x7f74400cfd70 legacy=0x7f744015d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f743ea00700 1 -- 172.21.15.99:0/3109662864 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74400d1f80 con 0x7f744015d980 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f41d3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc0d0150 0x7f41dc0cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f41dbe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc14a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f41e1000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41dc15d7b0 0x7f41dc0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f41e1000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41dc15d7b0 0x7f41dc0cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53168/0 (socket says 172.21.15.99:53168) 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.567+0000 7f41e1000700 1 -- 172.21.15.99:0/3329284527 learned_addr learned my addr 172.21.15.99:0/3329284527 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== osd.3 v2:172.21.15.99:6800/42217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f8ce8076e30 con 0x7f8ce8074320 2024-03-29T09:09:28.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f41e1000700 1 -- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 msgr2=0x7f41dc14a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f41e1000700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41dc0cfd70 0x7f41dc14a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f41e1000700 1 -- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc0d0150 msgr2=0x7f41dc0cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f41e1000700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41dc0d0150 0x7f41dc0cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f41e1000700 1 -- 172.21.15.99:0/3329284527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c4009550 con 0x7f41dc15d7b0 2024-03-29T09:09:28.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f41e1000700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41dc15d7b0 0x7f41dc0cc920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c450 tx=0x7f41d400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.568+0000 7f743ea00700 1 --2- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f744015d980 0x7f74400cdd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f742c009840 tx=0x7f742c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4bf54a927a4c109 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4011790 con 0x7f41dc15d7b0 2024-03-29T09:09:28.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d40118f0 con 0x7f41dc15d7b0 2024-03-29T09:09:28.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4011790 con 0x7f41dc15d7b0 2024-03-29T09:09:28.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f77000127d0 con 0x7f77180b21e0 2024-03-29T09:09:28.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f743e000700 1 -- 172.21.15.99:0/3109662864 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430001670 con 0x7f744015d980 2024-03-29T09:09:28.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f743e000700 1 -- 172.21.15.99:0/3109662864 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c004030 con 0x7f744015d980 2024-03-29T09:09:28.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f743e000700 1 -- 172.21.15.99:0/3109662864 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430001670 con 0x7f744015d980 2024-03-29T09:09:28.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.569+0000 7f61eee00700 1 -- start start 2024-03-29T09:09:28.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.570+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc0c8f20 con 0x7f41dc15d7b0 2024-03-29T09:09:28.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.570+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc0c9a00 con 0x7f41dc15d7b0 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.570+0000 7f770ea00700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f76f406ad60 0x7f76f406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f770008b540 con 0x7f77180b21e0 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f770ea00700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f76f40743a0 0x7f76f40767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 --> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76f4076eb0 con 0x7f76f40743a0 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f770008db60 con 0x7f77180b21e0 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f7716800700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f76f40743a0 0x7f76f40767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.570+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f744015d980 msgr2=0x7f74400cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.570+0000 7f7446800700 1 --2- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f744015d980 0x7f74400cdd30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f742c009840 tx=0x7f742c009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f61eee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e003bdb0 0x7f61e00ddfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f61eee00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e00e4560 0x7f61e004cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f61eee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e0056190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f61eee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0051f80 con 0x7f61e003bdb0 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f61ed000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e00e4560 0x7f61e004cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.572+0000 7f61ed000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e00e4560 0x7f61e004cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55250/0 (socket says 172.21.15.99:55250) 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.572+0000 7f61ed000700 1 -- 172.21.15.99:0/4193456946 learned_addr learned my addr 172.21.15.99:0/4193456946 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.571+0000 7f61ee400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e0056190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.572+0000 7f7715e00700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f76f406ad60 0x7f76f406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.572+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e0185840 con 0x7f61e00e4560 2024-03-29T09:09:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.572+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e01893c0 con 0x7f61e00e4940 2024-03-29T09:09:28.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.572+0000 7f7716800700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f76f40743a0 0x7f76f40767e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f61ed000700 1 -- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 msgr2=0x7f61e0056190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f61ed000700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e0056190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== osd.4 v2:172.21.15.133:6800/42489 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f76f4076eb0 con 0x7f76f40743a0 2024-03-29T09:09:28.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f61ed000700 1 -- 172.21.15.99:0/4193456946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e003bdb0 msgr2=0x7f61e00ddfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41b4000fc0 con 0x7f41dc15d7b0 2024-03-29T09:09:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.574+0000 7f7715e00700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f76f406ad60 0x7f76f406d220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7708000f60 tx=0x7f7708007a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.574+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 shutdown_connections 2024-03-29T09:09:28.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.574+0000 7f7446800700 1 --2- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.574+0000 7f7446800700 1 --2- 172.21.15.99:0/3109662864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f744015d980 0x7f74400cdd30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.574+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 >> 172.21.15.99:0/3109662864 conn(0x7f744009c470 msgr2=0x7f74400d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f61ed000700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61e003bdb0 0x7f61e00ddfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.574+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 shutdown_connections 2024-03-29T09:09:28.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.574+0000 7f7446800700 1 -- 172.21.15.99:0/3109662864 wait complete. 2024-03-29T09:09:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f61ed000700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80689d0 con 0x7f61e00e4560 2024-03-29T09:09:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.573+0000 7f61ee400700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61e00e4940 0x7f61e0056190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.575+0000 7f61ed000700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e00e4560 0x7f61e004cf00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61e8069e70 tx=0x7f61e8062580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.576+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8093660 con 0x7f61e00e4560 2024-03-29T09:09:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.576+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f41d4011a60 con 0x7f41dc15d7b0 2024-03-29T09:09:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.577+0000 7f7446800700 1 Processor -- start 2024-03-29T09:09:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.576+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e01896a0 con 0x7f61e00e4560 2024-03-29T09:09:28.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.576+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e0189c60 con 0x7f61e00e4560 2024-03-29T09:09:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.577+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80937c0 con 0x7f61e00e4560 2024-03-29T09:09:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.577+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8093b30 con 0x7f61e00e4560 2024-03-29T09:09:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.577+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61e003c930 con 0x7f61e00e4560 2024-03-29T09:09:28.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.578+0000 7f41d8c00700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f41b806acf0 0x7f41b806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.578+0000 7f7446800700 1 -- start start 2024-03-29T09:09:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.580+0000 7f7446800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400cb670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.580+0000 7f7446800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74400cfd70 0x7f744015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f743f400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f743ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74400cfd70 0x7f744015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f41d4088fc0 con 0x7f41dc15d7b0 2024-03-29T09:09:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f41dbe00700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f41b806acf0 0x7f41b806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f743ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74400cfd70 0x7f744015b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55266/0 (socket says 172.21.15.99:55266) 2024-03-29T09:09:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f743f400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32840/0 (socket says 172.21.15.99:32840) 2024-03-29T09:09:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f41d8c00700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7f41b8074330 0x7f41b8076770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 --> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41b8076e40 con 0x7f41b8074330 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f7446800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744015d980 0x7f74400cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.583+0000 7f743f400700 1 -- 172.21.15.99:0/3778366804 learned_addr learned my addr 172.21.15.99:0/3778366804 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.583+0000 7f7446800700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74400cb250 con 0x7f744015d980 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.583+0000 7f7446800700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74400cbbb0 con 0x7f74400cfd70 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.583+0000 7f7446800700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74400ca5d0 con 0x7f74400b1780 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f743ea00700 1 -- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 msgr2=0x7f74400cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f743ea00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74400b1780 0x7f74400cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f743ea00700 1 -- 172.21.15.99:0/3778366804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744015d980 msgr2=0x7f74400cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f743ea00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f744015d980 0x7f74400cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f743ea00700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c0094a0 con 0x7f74400cfd70 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f743ea00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74400cfd70 0x7f744015b980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f742c014000 tx=0x7f742c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.585+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c003e20 con 0x7f74400cfd70 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.585+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c010590 con 0x7f74400cfd70 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.585+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c010940 con 0x7f74400cfd70 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.585+0000 7f7446800700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74400ca8b0 con 0x7f74400cfd70 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.585+0000 7f7446800700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74400cae70 con 0x7f74400cfd70 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.579+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f61e8090020 con 0x7f61e00e4560 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f61e5600700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61c806ace0 0x7f61c806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f61e8104d30 con 0x7f61e00e4560 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f61eda00700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61c806ace0 0x7f61c806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f61e5600700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f61c8074320 0x7f61c8076760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61c8076e30 con 0x7f61c8074320 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.581+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61e81050a0 con 0x7f61e00e4560 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f61ee400700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f61c8074320 0x7f61c8076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f61eda00700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61c806ace0 0x7f61c806d1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009770 tx=0x7f61dc006cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.584+0000 7f61ee400700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f61c8074320 0x7f61c8076760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.585+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== osd.6 v2:172.21.15.133:6808/42490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f61c8076e30 con 0x7f61c8074320 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.582+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41d408b5e0 con 0x7f41dc15d7b0 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.583+0000 7f41dbe00700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f41b806acf0 0x7f41b806d1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000c00 tx=0x7f41c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.586+0000 7f41d3400700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7f41b8074330 0x7f41b8076770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.586+0000 7f7446800700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7418000fc0 con 0x7f74400cfd70 2024-03-29T09:09:28.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.588+0000 7f41d3400700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7f41b8074330 0x7f41b8076770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.589+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== osd.7 v2:172.21.15.133:6824/42500 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f41b8076e40 con 0x7f41b8074330 2024-03-29T09:09:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.589+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f742c010aa0 con 0x7f74400cfd70 2024-03-29T09:09:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.590+0000 7f7437400700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f742006ac90 0x7f742006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.590+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f742c08a920 con 0x7f74400cfd70 2024-03-29T09:09:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.590+0000 7f7437400700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7f7420074250 0x7f7420076690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.590+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7420076d60 con 0x7f7420074250 2024-03-29T09:09:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.590+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f742c08f2a0 con 0x7f74400cfd70 2024-03-29T09:09:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.591+0000 7f743f400700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f742006ac90 0x7f742006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.591+0000 7f743fe00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7f7420074250 0x7f7420076690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.592+0000 7f743fe00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7f7420074250 0x7f7420076690 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.592+0000 7f743f400700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f742006ac90 0x7f742006d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7430001200 tx=0x7f7430001190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.593+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== osd.5 v2:172.21.15.133:6816/42491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7420076d60 con 0x7f7420074250 2024-03-29T09:09:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.599+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 --> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a14071d20 con 0x7f09f4078660 2024-03-29T09:09:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.599+0000 7f0a0b400700 1 -- 172.21.15.99:0/283050923 <== osd.1 v2:172.21.15.99:6808/42220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0a14071d20 con 0x7f09f4078660 2024-03-29T09:09:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.600+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7f09f4078660 msgr2=0x7f09f407aaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.600+0000 7f0a1a600700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7f09f4078660 0x7f09f407aaa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.601+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09f406f0a0 msgr2=0x7f09f4071560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.601+0000 7f0a1a600700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09f406f0a0 0x7f09f4071560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0a14156960 tx=0x7f0a0c009040 comp rx=0 tx=0).stop 2024-03-29T09:09:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.601+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a140714a0 msgr2=0x7f0a1415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.601+0000 7f0a1a600700 1 --2- 172.21.15.99:0/283050923 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0a140714a0 0x7f0a1415b4c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000c00 tx=0x7f0a04004750 comp rx=0 tx=0).stop 2024-03-29T09:09:28.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.601+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 --> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d0c0b0250 con 0x7f8ce8074320 2024-03-29T09:09:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.601+0000 7f0a13e00700 1 -- 172.21.15.99:0/283050923 reap_dead start 2024-03-29T09:09:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.602+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 shutdown_connections 2024-03-29T09:09:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.602+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 >> 172.21.15.99:0/283050923 conn(0x7f0a140b0bc0 msgr2=0x7f0a14151310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.602+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 shutdown_connections 2024-03-29T09:09:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.602+0000 7f0a1a600700 1 -- 172.21.15.99:0/283050923 wait complete. 2024-03-29T09:09:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.603+0000 7f8d03400700 1 -- 172.21.15.99:0/4005978444 <== osd.3 v2:172.21.15.99:6800/42217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8d04003a20 con 0x7f8ce8074320 2024-03-29T09:09:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f8ce8074320 msgr2=0x7f8ce8076760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.605+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 --> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb74c002d40 con 0x7fb7500742d0 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f8ce8074320 0x7f8ce8076760 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ce806ace0 msgr2=0x7f8ce806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ce806ace0 0x7f8ce806d1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4007fd0 tx=0x7f8cf4009040 comp rx=0 tx=0).stop 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0cfd70 msgr2=0x7f8d0c0c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 --2- 172.21.15.99:0/4005978444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d0c0cfd70 0x7f8d0c0c7410 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00c450 tx=0x7f8cfc00c760 comp rx=0 tx=0).stop 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d0be00700 1 -- 172.21.15.99:0/4005978444 reap_dead start 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 shutdown_connections 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 >> 172.21.15.99:0/4005978444 conn(0x7f8d0c09c470 msgr2=0x7f8d0c0cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 shutdown_connections 2024-03-29T09:09:28.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.604+0000 7f8d12200700 1 -- 172.21.15.99:0/4005978444 wait complete. 2024-03-29T09:09:28.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.606+0000 7fb763400700 1 -- 172.21.15.99:0/1303997821 <== osd.2 v2:172.21.15.99:6814/42218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb74c002d40 con 0x7fb7500742d0 2024-03-29T09:09:28.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.607+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 --> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7598002d40 con 0x7f759c0742a0 2024-03-29T09:09:28.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.607+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fb7500742d0 msgr2=0x7fb750076710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.607+0000 7fb774c00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fb7500742d0 0x7fb750076710 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.607+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb75006ac90 msgr2=0x7fb75006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.607+0000 7fb774c00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb75006ac90 0x7fb75006d150 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb768041060 tx=0x7fb75c008040 comp rx=0 tx=0).stop 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.608+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76805b9d0 msgr2=0x7fb768180be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.608+0000 7fb774c00700 1 --2- 172.21.15.99:0/1303997821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb76805b9d0 0x7fb768180be0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb76805c300 tx=0x7fb764004750 comp rx=0 tx=0).stop 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.608+0000 7fb76fe00700 1 -- 172.21.15.99:0/1303997821 reap_dead start 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.608+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 shutdown_connections 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.608+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 >> 172.21.15.99:0/1303997821 conn(0x7fb768059dd0 msgr2=0x7fb7680e9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.608+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 shutdown_connections 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.608+0000 7fb774c00700 1 -- 172.21.15.99:0/1303997821 wait complete. 2024-03-29T09:09:28.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.609+0000 7f75bcc00700 1 -- 172.21.15.99:0/1970861631 <== osd.0 v2:172.21.15.99:6824/42219 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7598002d40 con 0x7f759c0742a0 2024-03-29T09:09:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7f759c0742a0 msgr2=0x7f759c0766e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f75c6c00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7f759c0742a0 0x7f759c0766e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f759c06ace0 msgr2=0x7f759c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f75c6c00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f759c06ace0 0x7f759c06d1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f75c0146fb0 tx=0x7f75a8009040 comp rx=0 tx=0).stop 2024-03-29T09:09:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0072330 msgr2=0x7f75c00b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f75c6c00700 1 --2- 172.21.15.99:0/1970861631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75c0072330 0x7f75c00b2610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c6b0 tx=0x7f75b000c9c0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f75c4c00700 1 -- 172.21.15.99:0/1970861631 reap_dead start 2024-03-29T09:09:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.611+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 --> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f77180b3f80 con 0x7f76f40743a0 2024-03-29T09:09:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.612+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 shutdown_connections 2024-03-29T09:09:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.612+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 >> 172.21.15.99:0/1970861631 conn(0x7f75c00b0bc0 msgr2=0x7f75c00b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.612+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 shutdown_connections 2024-03-29T09:09:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.612+0000 7f770ea00700 1 -- 172.21.15.99:0/1236779914 <== osd.4 v2:172.21.15.133:6800/42489 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7710001440 con 0x7f76f40743a0 2024-03-29T09:09:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.612+0000 7f75c6c00700 1 -- 172.21.15.99:0/1970861631 wait complete. 2024-03-29T09:09:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.613+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f76f40743a0 msgr2=0x7f76f40767e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.613+0000 7f7717200700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f76f40743a0 0x7f76f40767e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.615+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f76f406ad60 msgr2=0x7f76f406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.615+0000 7f7717200700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f76f406ad60 0x7f76f406d220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7708000f60 tx=0x7f7708007a70 comp rx=0 tx=0).stop 2024-03-29T09:09:28.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.615+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b21e0 msgr2=0x7f77181eb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.615+0000 7f7717200700 1 --2- 172.21.15.99:0/1236779914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77180b21e0 0x7f77181eb900 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f771806e3b0 tx=0x7f7700004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.618+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 --> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61e0054a10 con 0x7f61c8074320 2024-03-29T09:09:28.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.615+0000 7f7716800700 1 -- 172.21.15.99:0/1236779914 reap_dead start 2024-03-29T09:09:28.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.618+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 shutdown_connections 2024-03-29T09:09:28.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.618+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 >> 172.21.15.99:0/1236779914 conn(0x7f7718066960 msgr2=0x7f7718165040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.619+0000 7f61e5600700 1 -- 172.21.15.99:0/4193456946 <== osd.6 v2:172.21.15.133:6808/42490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61e0054a10 con 0x7f61c8074320 2024-03-29T09:09:28.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.619+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 shutdown_connections 2024-03-29T09:09:28.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.619+0000 7f7717200700 1 -- 172.21.15.99:0/1236779914 wait complete. 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.620+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f61c8074320 msgr2=0x7f61c8076760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.620+0000 7f61eee00700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f61c8074320 0x7f61c8076760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.620+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61c806ace0 msgr2=0x7f61c806d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.620+0000 7f61eee00700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61c806ace0 0x7f61c806d1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009770 tx=0x7f61dc006cd0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.620+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e00e4560 msgr2=0x7f61e004cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.620+0000 7f61eee00700 1 --2- 172.21.15.99:0/4193456946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61e00e4560 0x7f61e004cf00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61e8069e70 tx=0x7f61e8062580 comp rx=0 tx=0).stop 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.620+0000 7f61ee400700 1 -- 172.21.15.99:0/4193456946 reap_dead start 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.621+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 shutdown_connections 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.621+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 >> 172.21.15.99:0/4193456946 conn(0x7f61e00200a0 msgr2=0x7f61e00060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.623+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 shutdown_connections 2024-03-29T09:09:28.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.623+0000 7f61eee00700 1 -- 172.21.15.99:0/4193456946 wait complete. 2024-03-29T09:09:28.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.626+0000 7f7446800700 1 -- 172.21.15.99:0/3778366804 --> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7418002d40 con 0x7f7420074250 2024-03-29T09:09:28.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.627+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 --> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f41b4002d40 con 0x7f41b8074330 2024-03-29T09:09:28.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.627+0000 7f7437400700 1 -- 172.21.15.99:0/3778366804 <== osd.5 v2:172.21.15.133:6816/42491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7418002d40 con 0x7f7420074250 2024-03-29T09:09:28.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.628+0000 7f41d8c00700 1 -- 172.21.15.99:0/3329284527 <== osd.7 v2:172.21.15.133:6824/42500 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f41b4002d40 con 0x7f41b8074330 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.628+0000 7f7434c00700 1 -- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7f7420074250 msgr2=0x7f7420076690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.628+0000 7f7434c00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7f7420074250 0x7f7420076690 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f7434c00700 1 -- 172.21.15.99:0/3778366804 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f742006ac90 msgr2=0x7f742006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f7434c00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f742006ac90 0x7f742006d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7430001200 tx=0x7f7430001190 comp rx=0 tx=0).stop 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f7434c00700 1 -- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74400cfd70 msgr2=0x7f744015b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f7434c00700 1 --2- 172.21.15.99:0/3778366804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74400cfd70 0x7f744015b980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f742c014000 tx=0x7f742c004750 comp rx=0 tx=0).stop 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f743fe00700 1 -- 172.21.15.99:0/3778366804 reap_dead start 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f7434c00700 1 -- 172.21.15.99:0/3778366804 shutdown_connections 2024-03-29T09:09:28.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f7434c00700 1 -- 172.21.15.99:0/3778366804 >> 172.21.15.99:0/3778366804 conn(0x7f744009c470 msgr2=0x7f74400b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7f41b8074330 msgr2=0x7f41b8076770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7f41b8074330 0x7f41b8076770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f41b806acf0 msgr2=0x7f41b806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f41b806acf0 0x7f41b806d1b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f41c4000c00 tx=0x7f41c4008030 comp rx=0 tx=0).stop 2024-03-29T09:09:28.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41dc15d7b0 msgr2=0x7f41dc0cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 --2- 172.21.15.99:0/3329284527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41dc15d7b0 0x7f41dc0cc920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c450 tx=0x7f41d400c810 comp rx=0 tx=0).stop 2024-03-29T09:09:28.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e1000700 1 -- 172.21.15.99:0/3329284527 reap_dead start 2024-03-29T09:09:28.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 shutdown_connections 2024-03-29T09:09:28.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.629+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 >> 172.21.15.99:0/3329284527 conn(0x7f41dc09c470 msgr2=0x7f41dc06c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.630+0000 7f7434c00700 1 -- 172.21.15.99:0/3778366804 shutdown_connections 2024-03-29T09:09:28.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.630+0000 7f7434c00700 1 -- 172.21.15.99:0/3778366804 wait complete. 2024-03-29T09:09:28.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.632+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 shutdown_connections 2024-03-29T09:09:28.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.632+0000 7f41e3000700 1 -- 172.21.15.99:0/3329284527 wait complete. 2024-03-29T09:09:28.634 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:28.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T09:09:28.644 INFO:teuthology.orchestra.run.smithi099.stdout:55834574858 2024-03-29T09:09:28.645 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:28.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T09:09:28.646 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T09:09:28.648 INFO:teuthology.orchestra.run.smithi099.stdout:55834574860 2024-03-29T09:09:28.649 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T09:09:28.654 INFO:teuthology.orchestra.run.smithi099.stdout:55834574860 2024-03-29T09:09:28.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T09:09:28.655 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:28.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T09:09:28.662 INFO:teuthology.orchestra.run.smithi099.stdout:55834574860 2024-03-29T09:09:28.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T09:09:28.663 INFO:teuthology.orchestra.run.smithi099.stdout:55834574861 2024-03-29T09:09:28.664 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T09:09:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.974+0000 7f51b3e00700 1 Processor -- start 2024-03-29T09:09:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.974+0000 7f51b3e00700 1 -- start start 2024-03-29T09:09:28.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.976+0000 7f51b3e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b406ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.976+0000 7f51b3e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f51b40cef40 0x7f51b40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.977+0000 7f51b3e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b406c120 con 0x7f51b4161b20 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b3400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f51b4161b20 0x7f51b4161fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49638/0 (socket says 172.21.15.99:49638) 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b3400700 1 -- 172.21.15.99:0/2806307169 learned_addr learned my addr 172.21.15.99:0/2806307169 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b2a00700 1 --2- 172.21.15.99:0/2806307169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b406ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b3e00700 1 -- 172.21.15.99:0/2806307169 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b40cf900 con 0x7f51b40cef40 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b3e00700 1 -- 172.21.15.99:0/2806307169 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b406c9c0 con 0x7f51b414afe0 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b1600700 1 -- 172.21.15.99:0/2806307169 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363318276 0 0) 0x7f51b406c120 con 0x7f51b4161b20 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b1600700 1 -- 172.21.15.99:0/2806307169 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51980035b0 con 0x7f51b4161b20 2024-03-29T09:09:28.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.978+0000 7f51b2000700 1 --2- 172.21.15.99:0/2806307169 >> v2:172.21.15.99:3300/0 conn(0x7f51b40cef40 0x7f51b40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.979+0000 7f51b2a00700 1 -- 172.21.15.99:0/2806307169 >> v2:172.21.15.99:3300/0 conn(0x7f51b40cef40 msgr2=0x7f51b40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:28.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.979+0000 7f51b2a00700 1 --2- 172.21.15.99:0/2806307169 >> v2:172.21.15.99:3300/0 conn(0x7f51b40cef40 0x7f51b40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.979+0000 7f51b2a00700 1 -- 172.21.15.99:0/2806307169 >> v1:172.21.15.99:6789/0 conn(0x7f51b4161b20 legacy=0x7f51b4161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.979+0000 7f51b2a00700 1 -- 172.21.15.99:0/2806307169 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4162870 con 0x7f51b414afe0 2024-03-29T09:09:28.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.980+0000 7f51b2a00700 1 --2- 172.21.15.99:0/2806307169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b406ce50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009840 tx=0x7f51ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5dfe1fe035f04c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.980+0000 7f51b1600700 1 -- 172.21.15.99:0/2806307169 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac011830 con 0x7f51b414afe0 2024-03-29T09:09:28.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.980+0000 7f51b1600700 1 -- 172.21.15.99:0/2806307169 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac011990 con 0x7f51b414afe0 2024-03-29T09:09:28.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.980+0000 7f51b1600700 1 -- 172.21.15.99:0/2806307169 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac011d20 con 0x7f51b414afe0 2024-03-29T09:09:28.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.981+0000 7f51b3e00700 1 -- 172.21.15.99:0/2806307169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 msgr2=0x7f51b406ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.981+0000 7f51b3e00700 1 --2- 172.21.15.99:0/2806307169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b406ce50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009840 tx=0x7f51ac009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:28.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.982+0000 7f51b3e00700 1 -- 172.21.15.99:0/2806307169 shutdown_connections 2024-03-29T09:09:28.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.982+0000 7f51b3e00700 1 --2- 172.21.15.99:0/2806307169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b406ce50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.982+0000 7f51b3e00700 1 --2- 172.21.15.99:0/2806307169 >> v2:172.21.15.99:3300/0 conn(0x7f51b40cef40 0x7f51b40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.982+0000 7f51b3e00700 1 -- 172.21.15.99:0/2806307169 >> 172.21.15.99:0/2806307169 conn(0x7f51b4066540 msgr2=0x7f51b4066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:28.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.982+0000 7f51b3e00700 1 -- 172.21.15.99:0/2806307169 shutdown_connections 2024-03-29T09:09:28.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.982+0000 7f51b3e00700 1 -- 172.21.15.99:0/2806307169 wait complete. 2024-03-29T09:09:28.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.985+0000 7f51b3e00700 1 Processor -- start 2024-03-29T09:09:28.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.986+0000 7f51b3e00700 1 -- start start 2024-03-29T09:09:28.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.987+0000 7f51b3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b40cef40 0x7f51b409c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.988+0000 7f51b2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b40cef40 0x7f51b409c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.988+0000 7f51b2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b40cef40 0x7f51b409c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53218/0 (socket says 172.21.15.99:53218) 2024-03-29T09:09:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.989+0000 7f51b3e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b41ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.990+0000 7f51b2a00700 1 -- 172.21.15.99:0/3353278683 learned_addr learned my addr 172.21.15.99:0/3353278683 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.990+0000 7f51b3e00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b4161b20 0x7f51b415c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.990+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41ff5d0 con 0x7f51b40cef40 2024-03-29T09:09:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.990+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415c910 con 0x7f51b4161b20 2024-03-29T09:09:28.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.990+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b4136740 con 0x7f51b414afe0 2024-03-29T09:09:28.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.990+0000 7f51b2000700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b41ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.990+0000 7f51b3400700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b4161b20 0x7f51b415c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:28.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.991+0000 7f51b3400700 1 -- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 msgr2=0x7f51b41ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.991+0000 7f51b3400700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b41ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.991+0000 7f51b3400700 1 -- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b40cef40 msgr2=0x7f51b409c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:28.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.991+0000 7f51b3400700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b40cef40 0x7f51b409c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:28.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.991+0000 7f51b3400700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0094a0 con 0x7f51b4161b20 2024-03-29T09:09:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.993+0000 7f51b3400700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b4161b20 0x7f51b415c3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c390 tx=0x7f519c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.993+0000 7f51ab400700 1 -- 172.21.15.99:0/3353278683 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c005d70 con 0x7f51b4161b20 2024-03-29T09:09:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.993+0000 7f51ab400700 1 -- 172.21.15.99:0/3353278683 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c00ce10 con 0x7f51b4161b20 2024-03-29T09:09:28.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.993+0000 7f51ab400700 1 -- 172.21.15.99:0/3353278683 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c0072d0 con 0x7f51b4161b20 2024-03-29T09:09:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.993+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b4136e00 con 0x7f51b4161b20 2024-03-29T09:09:28.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.994+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b409a2a0 con 0x7f51b4161b20 2024-03-29T09:09:28.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.995+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b4136f90 con 0x7f51b4161b20 2024-03-29T09:09:28.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.996+0000 7f5013200700 1 Processor -- start 2024-03-29T09:09:28.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.997+0000 7f51ab400700 1 -- 172.21.15.99:0/3353278683 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f519c007430 con 0x7f51b4161b20 2024-03-29T09:09:28.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.998+0000 7f51ab400700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f519006ac90 0x7f519006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:28.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.998+0000 7f51ab400700 1 -- 172.21.15.99:0/3353278683 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f519c088940 con 0x7f51b4161b20 2024-03-29T09:09:28.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.999+0000 7f51b2a00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f519006ac90 0x7f519006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.997+0000 7f5013200700 1 -- start start 2024-03-29T09:09:29.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.998+0000 7f5013200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f500c0532f0 0x7f500c0df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.998+0000 7f5013200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0df6e0 0x7f500c04feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.998+0000 7f5013200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500c0537d0 con 0x7f500c00c100 2024-03-29T09:09:29.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.999+0000 7f5013200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c053ab0 con 0x7f500c0532f0 2024-03-29T09:09:29.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.001+0000 7f51b2a00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f519006ac90 0x7f519006d150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009250 tx=0x7f51ac008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:28.999+0000 7f5013200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c053d90 con 0x7f500c0df6e0 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.000+0000 7f5011400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f500c00c100 0x7f500c04f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:52052/0 (socket says 172.21.15.99:52052) 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.000+0000 7f5011400700 1 -- 172.21.15.99:0/2465012169 learned_addr learned my addr 172.21.15.99:0/2465012169 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.000+0000 7f5012800700 1 --2- 172.21.15.99:0/2465012169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0df6e0 0x7f500c04feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.001+0000 7f5011e00700 1 --2- 172.21.15.99:0/2465012169 >> v2:172.21.15.99:3300/0 conn(0x7f500c0532f0 0x7f500c0df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.001+0000 7f5012800700 1 -- 172.21.15.99:0/2465012169 >> v2:172.21.15.99:3300/0 conn(0x7f500c0532f0 msgr2=0x7f500c0df1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5012800700 1 --2- 172.21.15.99:0/2465012169 >> v2:172.21.15.99:3300/0 conn(0x7f500c0532f0 0x7f500c0df1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5012800700 1 -- 172.21.15.99:0/2465012169 >> v1:172.21.15.133:6789/0 conn(0x7f500c00c100 legacy=0x7f500c04f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5012800700 1 -- 172.21.15.99:0/2465012169 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c054070 con 0x7f500c0df6e0 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5011e00700 1 --2- 172.21.15.99:0/2465012169 >> v2:172.21.15.99:3300/0 conn(0x7f500c0532f0 0x7f500c0df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5012800700 1 --2- 172.21.15.99:0/2465012169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0df6e0 0x7f500c04feb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5014000eb0 tx=0x7f5014067a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77037939ae0b7b17 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5010a00700 1 -- 172.21.15.99:0/2465012169 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501406b950 con 0x7f500c0df6e0 2024-03-29T09:09:29.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5010a00700 1 -- 172.21.15.99:0/2465012169 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501406baf0 con 0x7f500c0df6e0 2024-03-29T09:09:29.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.002+0000 7f5010a00700 1 -- 172.21.15.99:0/2465012169 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501406bea0 con 0x7f500c0df6e0 2024-03-29T09:09:29.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.003+0000 7f5013200700 1 -- 172.21.15.99:0/2465012169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0df6e0 msgr2=0x7f500c04feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.003+0000 7f5013200700 1 --2- 172.21.15.99:0/2465012169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0df6e0 0x7f500c04feb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5014000eb0 tx=0x7f5014067a10 comp rx=0 tx=0).stop 2024-03-29T09:09:29.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.004+0000 7f51ab400700 1 -- 172.21.15.99:0/3353278683 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f519c0537b0 con 0x7f51b4161b20 2024-03-29T09:09:29.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.004+0000 7f5013200700 1 -- 172.21.15.99:0/2465012169 shutdown_connections 2024-03-29T09:09:29.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.004+0000 7f5013200700 1 --2- 172.21.15.99:0/2465012169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0df6e0 0x7f500c04feb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.004+0000 7f5013200700 1 --2- 172.21.15.99:0/2465012169 >> v2:172.21.15.99:3300/0 conn(0x7f500c0532f0 0x7f500c0df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.004+0000 7f5013200700 1 -- 172.21.15.99:0/2465012169 >> 172.21.15.99:0/2465012169 conn(0x7f500c054b20 msgr2=0x7f500c054f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.004+0000 7f5013200700 1 -- 172.21.15.99:0/2465012169 shutdown_connections 2024-03-29T09:09:29.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.004+0000 7f5013200700 1 -- 172.21.15.99:0/2465012169 wait complete. 2024-03-29T09:09:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.007+0000 7f5013200700 1 Processor -- start 2024-03-29T09:09:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.007+0000 7f5013200700 1 -- start start 2024-03-29T09:09:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.007+0000 7fac0aa00700 1 Processor -- start 2024-03-29T09:09:29.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.008+0000 7fac0aa00700 1 -- start start 2024-03-29T09:09:29.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.008+0000 7f5013200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f500c00c100 0x7f500c04aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7fac0aa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7fac0aa00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fabfc0e0200 0x7fabfc0e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7fac0aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fabfc006100 0x7fabfc006780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7fac0aa00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc0e4950 con 0x7fabfc006100 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7fac0aa00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc054060 con 0x7fabfc0e0200 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7fac0aa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0532a0 con 0x7fabfc0e3af0 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.008+0000 7f5013200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0532f0 0x7f500c180c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.008+0000 7f5013200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f500c0df6e0 0x7f500c181180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.008+0000 7f5013200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0ce940 con 0x7f500c0df6e0 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.008+0000 7f5013200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c004f00 con 0x7f500c00c100 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.008+0000 7f5013200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0ce010 con 0x7f500c0532f0 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7f5011e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f500c00c100 0x7f500c04aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7f5011e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f500c00c100 0x7f500c04aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55286/0 (socket says 172.21.15.99:55286) 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7f5011e00700 1 -- 172.21.15.99:0/4091185560 learned_addr learned my addr 172.21.15.99:0/4091185560 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7f5011400700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0532f0 0x7f500c180c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.009+0000 7f5012800700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f500c0df6e0 0x7f500c181180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.010+0000 7fac08a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fabfc006100 0x7fabfc006780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.010+0000 7fac08a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fabfc006100 0x7fabfc006780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53250/0 (socket says 172.21.15.99:53250) 2024-03-29T09:09:29.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.010+0000 7fac08a00700 1 -- 172.21.15.99:0/2875103342 learned_addr learned my addr 172.21.15.99:0/2875103342 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.010+0000 7fac03400700 1 --2- 172.21.15.99:0/2875103342 >> v2:172.21.15.133:3300/0 conn(0x7fabfc0e0200 0x7fabfc0e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.010+0000 7fac03e00700 1 --2- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7fac03400700 1 -- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 msgr2=0x7fabfc0e3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7fac03400700 1 --2- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7fac03400700 1 -- 172.21.15.99:0/2875103342 >> v2:172.21.15.99:3300/0 conn(0x7fabfc006100 msgr2=0x7fabfc006780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7fac03400700 1 --2- 172.21.15.99:0/2875103342 >> v2:172.21.15.99:3300/0 conn(0x7fabfc006100 0x7fabfc006780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7fac03400700 1 -- 172.21.15.99:0/2875103342 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc053580 con 0x7fabfc0e0200 2024-03-29T09:09:29.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.010+0000 7f1ca0e00700 1 Processor -- start 2024-03-29T09:09:29.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.011+0000 7f1ca0e00700 1 -- start start 2024-03-29T09:09:29.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.011+0000 7f5011400700 1 -- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f500c00c100 msgr2=0x7f500c04aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7fa8e4c00700 1 Processor -- start 2024-03-29T09:09:29.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7fa8e4c00700 1 -- start start 2024-03-29T09:09:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f1ca0e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1c9c0715f0 0x7f1c9c0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f1ca0e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0b3b00 0x7f1c9c0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f1ca0e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9c0727a0 con 0x7f1c9c1439f0 2024-03-29T09:09:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f1ca0e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c06ec40 con 0x7f1c9c0715f0 2024-03-29T09:09:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f1ca0e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0b3f90 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.011+0000 7f5011400700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f500c00c100 0x7f500c04aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.011+0000 7f5011400700 1 -- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f500c0df6e0 msgr2=0x7f500c181180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.011+0000 7f5011400700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f500c0df6e0 0x7f500c181180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.011+0000 7f5011400700 1 -- 172.21.15.99:0/4091185560 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c0ce2f0 con 0x7f500c0532f0 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.012+0000 7f5011e00700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f500c00c100 0x7f500c04aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f5011400700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0532f0 0x7f500c180c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f500000c4d0 tx=0x7f500000c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f500a000700 1 -- 172.21.15.99:0/4091185560 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5000005d70 con 0x7f500c0532f0 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7f500a000700 1 -- 172.21.15.99:0/4091185560 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500000ce80 con 0x7f500c0532f0 2024-03-29T09:09:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7f500a000700 1 -- 172.21.15.99:0/4091185560 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000072d0 con 0x7f500c0532f0 2024-03-29T09:09:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.013+0000 7f1c9a800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0b3b00 0x7f1c9c0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7f1c9a800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0b3b00 0x7f1c9c0b2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32914/0 (socket says 172.21.15.99:32914) 2024-03-29T09:09:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7f1c9a800700 1 -- 172.21.15.99:0/1480157795 learned_addr learned my addr 172.21.15.99:0/1480157795 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500c0ce5d0 con 0x7f500c0532f0 2024-03-29T09:09:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c04afd0 con 0x7f500c0532f0 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fac03400700 1 --2- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0e0200 0x7fabfc0e4410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabfc055c20 tx=0x7fabf0009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59f00718d3053b14 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fa8e4c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8e0159220 0x7fa8e0159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fa8e4c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8e00b1ec0 0x7fa8e00ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fa8e4c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e006e630 0x7fa8e0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fa8e4c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e015a950 con 0x7fa8e0159220 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fa8e4c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e0159b40 con 0x7fa8e00b1ec0 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fa8e4c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e0072740 con 0x7fa8e006e630 2024-03-29T09:09:29.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.014+0000 7fac02a00700 1 -- 172.21.15.99:0/2875103342 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00109d0 con 0x7fabfc0e0200 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7fac02a00700 1 -- 172.21.15.99:0/2875103342 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf0010b30 con 0x7fabfc0e0200 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7fac02a00700 1 -- 172.21.15.99:0/2875103342 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf00109d0 con 0x7fabfc0e0200 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7fa8de800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e006e630 0x7fa8e0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7fa8de800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e006e630 0x7fa8e0070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32920/0 (socket says 172.21.15.99:32920) 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7fa8de800700 1 -- 172.21.15.99:0/2854178552 learned_addr learned my addr 172.21.15.99:0/2854178552 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7fa8dde00700 1 --2- 172.21.15.99:0/2854178552 >> v2:172.21.15.99:3300/0 conn(0x7fa8e0159220 0x7fa8e0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7fa8dd400700 1 --2- 172.21.15.99:0/2854178552 >> v2:172.21.15.133:3300/0 conn(0x7fa8e00b1ec0 0x7fa8e00ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7fa8de800700 1 -- 172.21.15.99:0/2854178552 >> v2:172.21.15.133:3300/0 conn(0x7fa8e00b1ec0 msgr2=0x7fa8e00ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7fa8de800700 1 --2- 172.21.15.99:0/2854178552 >> v2:172.21.15.133:3300/0 conn(0x7fa8e00b1ec0 0x7fa8e00ce500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7fa8de800700 1 -- 172.21.15.99:0/2854178552 >> v2:172.21.15.99:3300/0 conn(0x7fa8e0159220 msgr2=0x7fa8e0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7fa8de800700 1 --2- 172.21.15.99:0/2854178552 >> v2:172.21.15.99:3300/0 conn(0x7fa8e0159220 0x7fa8e0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7fa8de800700 1 -- 172.21.15.99:0/2854178552 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0159dd0 con 0x7fa8e006e630 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7fa8de800700 1 --2- 172.21.15.99:0/2854178552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e006e630 0x7fa8e0070b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00aead0 tx=0x7fa8d0009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7edea48e9ace48d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.015+0000 7f1c9a800700 1 -- 172.21.15.99:0/1480157795 >> v2:172.21.15.99:3300/0 conn(0x7f1c9c0715f0 msgr2=0x7f1c9c0b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7f1c98a00700 1 -- 172.21.15.99:0/1480157795 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533876196 0 0) 0x7f1c9c0727a0 con 0x7f1c9c1439f0 2024-03-29T09:09:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7f1c99400700 1 --2- 172.21.15.99:0/1480157795 >> v2:172.21.15.99:3300/0 conn(0x7f1c9c0715f0 0x7f1c9c0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7f1c9a800700 1 --2- 172.21.15.99:0/1480157795 >> v2:172.21.15.99:3300/0 conn(0x7f1c9c0715f0 0x7f1c9c0b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7f1c9a800700 1 -- 172.21.15.99:0/1480157795 >> v1:172.21.15.133:6789/0 conn(0x7f1c9c1439f0 legacy=0x7f1c9c15e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7f1c9a800700 1 -- 172.21.15.99:0/1480157795 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c0ac8b0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.016+0000 7f1c9a800700 1 --2- 172.21.15.99:0/1480157795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0b3b00 0x7f1c9c0b2230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009e50 tx=0x7f1c9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a5e46eefdb3b28 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7f1c98a00700 1 -- 172.21.15.99:0/1480157795 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400e070 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c04b1c0 con 0x7f500c0532f0 2024-03-29T09:09:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7f1c98a00700 1 -- 172.21.15.99:0/1480157795 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9400b9c0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7f1c98a00700 1 -- 172.21.15.99:0/1480157795 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400e070 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7f500a000700 1 -- 172.21.15.99:0/4091185560 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5000007430 con 0x7f500c0532f0 2024-03-29T09:09:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7fac0aa00700 1 -- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0e0200 msgr2=0x7fabfc0e4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7fac0aa00700 1 --2- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0e0200 0x7fabfc0e4410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabfc055c20 tx=0x7fabf0009b20 comp rx=0 tx=0).stop 2024-03-29T09:09:29.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7fa8dca00700 1 -- 172.21.15.99:0/2854178552 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0011830 con 0x7fa8e006e630 2024-03-29T09:09:29.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7fa8dca00700 1 -- 172.21.15.99:0/2854178552 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d0011990 con 0x7fa8e006e630 2024-03-29T09:09:29.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.017+0000 7fa8dca00700 1 -- 172.21.15.99:0/2854178552 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0011d20 con 0x7fa8e006e630 2024-03-29T09:09:29.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.018+0000 7fac0aa00700 1 -- 172.21.15.99:0/2875103342 shutdown_connections 2024-03-29T09:09:29.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.018+0000 7fac0aa00700 1 --2- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.018+0000 7fac0aa00700 1 --2- 172.21.15.99:0/2875103342 >> v2:172.21.15.99:3300/0 conn(0x7fabfc006100 0x7fabfc006780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.018+0000 7fac0aa00700 1 --2- 172.21.15.99:0/2875103342 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0e0200 0x7fabfc0e4410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.018+0000 7fac0aa00700 1 -- 172.21.15.99:0/2875103342 >> 172.21.15.99:0/2875103342 conn(0x7fabfc054b20 msgr2=0x7fabfc054f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.018+0000 7eff81a00700 1 Processor -- start 2024-03-29T09:09:29.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.018+0000 7fac0aa00700 1 -- 172.21.15.99:0/2875103342 shutdown_connections 2024-03-29T09:09:29.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 -- 172.21.15.99:0/1480157795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0b3b00 msgr2=0x7f1c9c0b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/1480157795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0b3b00 0x7f1c9c0b2230 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009e50 tx=0x7f1c9400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:29.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fac0aa00700 1 -- 172.21.15.99:0/2875103342 wait complete. 2024-03-29T09:09:29.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f500a000700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ff406ace0 0x7f4ff406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 -- 172.21.15.99:0/1480157795 shutdown_connections 2024-03-29T09:09:29.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/1480157795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0b3b00 0x7f1c9c0b2230 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/1480157795 >> v2:172.21.15.99:3300/0 conn(0x7f1c9c0715f0 0x7f1c9c0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 -- 172.21.15.99:0/1480157795 >> 172.21.15.99:0/1480157795 conn(0x7f1c9c0b0bc0 msgr2=0x7f1c9c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fa8e4c00700 1 -- 172.21.15.99:0/2854178552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e006e630 msgr2=0x7fa8e0070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/2854178552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e006e630 0x7fa8e0070b20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00aead0 tx=0x7fa8d0009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7eff81a00700 1 -- start start 2024-03-29T09:09:29.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f500a000700 1 -- 172.21.15.99:0/4091185560 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5000088730 con 0x7f500c0532f0 2024-03-29T09:09:29.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 -- 172.21.15.99:0/1480157795 shutdown_connections 2024-03-29T09:09:29.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f1ca0e00700 1 -- 172.21.15.99:0/1480157795 wait complete. 2024-03-29T09:09:29.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7f5011e00700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ff406ace0 0x7f4ff406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fa8e4c00700 1 -- 172.21.15.99:0/2854178552 shutdown_connections 2024-03-29T09:09:29.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/2854178552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e006e630 0x7fa8e0070b20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/2854178552 >> v2:172.21.15.99:3300/0 conn(0x7fa8e0159220 0x7fa8e0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/2854178552 >> v2:172.21.15.133:3300/0 conn(0x7fa8e00b1ec0 0x7fa8e00ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.019+0000 7fa8e4c00700 1 -- 172.21.15.99:0/2854178552 >> 172.21.15.99:0/2854178552 conn(0x7fa8e0066540 msgr2=0x7fa8e0066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.020+0000 7fa8e4c00700 1 -- 172.21.15.99:0/2854178552 shutdown_connections 2024-03-29T09:09:29.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.020+0000 7fa8e4c00700 1 -- 172.21.15.99:0/2854178552 wait complete. 2024-03-29T09:09:29.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.021+0000 7f5011e00700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ff406ace0 0x7f4ff406d1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f500c0df650 tx=0x7f5004008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.022+0000 7f1ca0e00700 1 Processor -- start 2024-03-29T09:09:29.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.022+0000 7eff81a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff7c1439f0 0x7eff7c0715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.022+0000 7f1ca0e00700 1 -- start start 2024-03-29T09:09:29.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.023+0000 7eff81a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.023+0000 7fa8e4c00700 1 Processor -- start 2024-03-29T09:09:29.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.023+0000 7eff7aa00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff7c1439f0 0x7eff7c0715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.023+0000 7eff7aa00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff7c1439f0 0x7eff7c0715f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55306/0 (socket says 172.21.15.99:55306) 2024-03-29T09:09:29.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.023+0000 7fa8e4c00700 1 -- start start 2024-03-29T09:09:29.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7eff7aa00700 1 -- 172.21.15.99:0/253103544 learned_addr learned my addr 172.21.15.99:0/253103544 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7fac0aa00700 1 Processor -- start 2024-03-29T09:09:29.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7eff7a000700 1 --2- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7f1ca0e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0715f0 0x7f1c9c146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7f1ca0e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9c0b3b00 0x7f1c9c136470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7f1ca0e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c9c1439f0 0x7f1c9c142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7f1ca0e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c146690 con 0x7f1c9c1439f0 2024-03-29T09:09:29.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7f1ca0e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c145a80 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.024+0000 7f1ca0e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c145d60 con 0x7f1c9c0715f0 2024-03-29T09:09:29.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7fac0aa00700 1 -- start start 2024-03-29T09:09:29.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0715f0 0x7f1c9c146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0715f0 0x7f1c9c146b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32940/0 (socket says 172.21.15.99:32940) 2024-03-29T09:09:29.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f500a000700 1 -- 172.21.15.99:0/4091185560 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f50000537b0 con 0x7f500c0532f0 2024-03-29T09:09:29.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99e00700 1 -- 172.21.15.99:0/122047961 learned_addr learned my addr 172.21.15.99:0/122047961 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99400700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9c0b3b00 0x7f1c9c136470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99400700 1 -- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0715f0 msgr2=0x7f1c9c146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7c0b3f60 con 0x7eff7c0b3b00 2024-03-29T09:09:29.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7c06ec40 con 0x7eff7c1439f0 2024-03-29T09:09:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7fac0aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabfc006100 0x7fabfc0d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7fac0aa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7fac0aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0dbea0 0x7fabfc0de0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7fac0aa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0e2d40 con 0x7fabfc006100 2024-03-29T09:09:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7fac0aa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0e2130 con 0x7fabfc0dbea0 2024-03-29T09:09:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7fac0aa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc0e2410 con 0x7fabfc0e3af0 2024-03-29T09:09:29.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c0727a0 con 0x7eff7c15e300 2024-03-29T09:09:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7fa8e4c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8e006e630 0x7fa8e014e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7fa8e4c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e00b1ec0 0x7fa8e01522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7fa8e4c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8e0159220 0x7fa8e0152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7fa8dd400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e00b1ec0 0x7fa8e01522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7fa8dd400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e00b1ec0 0x7fa8e01522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32952/0 (socket says 172.21.15.99:32952) 2024-03-29T09:09:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99400700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0715f0 0x7f1c9c146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99400700 1 -- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c9c1439f0 msgr2=0x7f1c9c142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99400700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c9c1439f0 0x7f1c9c142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.025+0000 7f1c99400700 1 -- 172.21.15.99:0/122047961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94009b30 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7eff79600700 1 -- 172.21.15.99:0/253103544 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177333487 0 0) 0x7eff7c0b3f60 con 0x7eff7c0b3b00 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7eff79600700 1 -- 172.21.15.99:0/253103544 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff600035b0 con 0x7eff7c0b3b00 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7eff7aa00700 1 -- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 msgr2=0x7eff7c0b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7eff7aa00700 1 --2- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7eff7aa00700 1 -- 172.21.15.99:0/253103544 >> v1:172.21.15.133:6789/0 conn(0x7eff7c0b3b00 legacy=0x7eff7c160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7eff7aa00700 1 -- 172.21.15.99:0/253103544 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c0ac8b0 con 0x7eff7c1439f0 2024-03-29T09:09:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7fa8dde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8e006e630 0x7fa8e014e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.026+0000 7f1c99400700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9c0b3b00 0x7f1c9c136470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800c7c0 tx=0x7f1c8800cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7f1c92000700 1 -- 172.21.15.99:0/122047961 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88011800 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7f1c92000700 1 -- 172.21.15.99:0/122047961 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88011960 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c146040 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7f1c92000700 1 -- 172.21.15.99:0/122047961 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88011d10 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c1369b0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7fa8dde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8e006e630 0x7fa8e014e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53286/0 (socket says 172.21.15.99:53286) 2024-03-29T09:09:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7fac03400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7fac03400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32942/0 (socket says 172.21.15.99:32942) 2024-03-29T09:09:29.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.027+0000 7fac03400700 1 -- 172.21.15.99:0/18603529 learned_addr learned my addr 172.21.15.99:0/18603529 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.028+0000 7eff7aa00700 1 --2- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 0x7eff7c0715f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0098f0 tx=0x7eff6c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1ef05f39f270cc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fa8e4c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e0152fb0 con 0x7fa8e006e630 2024-03-29T09:09:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fa8e4c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01531c0 con 0x7fa8e0159220 2024-03-29T09:09:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fa8e4c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01f3450 con 0x7fa8e00b1ec0 2024-03-29T09:09:29.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7eff79600700 1 -- 172.21.15.99:0/253103544 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c004230 con 0x7eff7c1439f0 2024-03-29T09:09:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7eff79600700 1 -- 172.21.15.99:0/253103544 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6c004390 con 0x7eff7c1439f0 2024-03-29T09:09:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7eff79600700 1 -- 172.21.15.99:0/253103544 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c0036a0 con 0x7eff7c1439f0 2024-03-29T09:09:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fa8de800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8e0159220 0x7fa8e0152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 msgr2=0x7eff7c0715f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7eff81a00700 1 --2- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 0x7eff7c0715f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0098f0 tx=0x7eff6c009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fac08a00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0dbea0 0x7fabfc0de0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fac03400700 1 -- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0dbea0 msgr2=0x7fabfc0de0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7fac03400700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0dbea0 0x7fabfc0de0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7fac03400700 1 -- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabfc006100 msgr2=0x7fabfc0d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7fac03400700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabfc006100 0x7fabfc0d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7fac03400700 1 -- 172.21.15.99:0/18603529 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00094a0 con 0x7fabfc0e3af0 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7fac08a00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0dbea0 0x7fabfc0de0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fa8de800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8e0159220 0x7fa8e0152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55318/0 (socket says 172.21.15.99:55318) 2024-03-29T09:09:29.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7fa8de800700 1 -- 172.21.15.99:0/3930024312 learned_addr learned my addr 172.21.15.99:0/3930024312 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.029+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c071ee0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.030+0000 7fac03400700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e17c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fabf00090b0 tx=0x7fabf0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fa8de800700 1 -- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e00b1ec0 msgr2=0x7fa8e01522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fa8de800700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e00b1ec0 0x7fa8e01522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fa8de800700 1 -- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8e006e630 msgr2=0x7fa8e014e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fa8de800700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8e006e630 0x7fa8e014e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fa8de800700 1 -- 172.21.15.99:0/3930024312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0009550 con 0x7fa8e0159220 2024-03-29T09:09:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fa8dde00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8e006e630 0x7fa8e014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 shutdown_connections 2024-03-29T09:09:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7eff81a00700 1 --2- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 0x7eff7c0715f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7eff81a00700 1 --2- 172.21.15.99:0/253103544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 >> 172.21.15.99:0/253103544 conn(0x7eff7c0b0bc0 msgr2=0x7eff7c0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fabf7e00700 1 -- 172.21.15.99:0/18603529 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0023830 con 0x7fabfc0e3af0 2024-03-29T09:09:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fabf7e00700 1 -- 172.21.15.99:0/18603529 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf00239b0 con 0x7fabfc0e3af0 2024-03-29T09:09:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fabf7e00700 1 -- 172.21.15.99:0/18603529 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf0023d60 con 0x7fabfc0e3af0 2024-03-29T09:09:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7f1c92000700 1 -- 172.21.15.99:0/122047961 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1c88018360 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.031+0000 7fa8de800700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8e0159220 0x7fa8e0152a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009fd0 tx=0x7fa8d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.032+0000 7fa8dd400700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e00b1ec0 0x7fa8e01522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:29.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.032+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc0e26f0 con 0x7fabfc0e3af0 2024-03-29T09:09:29.047 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.032+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc0d1320 con 0x7fabfc0e3af0 2024-03-29T09:09:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.033+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 shutdown_connections 2024-03-29T09:09:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.033+0000 7eff81a00700 1 -- 172.21.15.99:0/253103544 wait complete. 2024-03-29T09:09:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.033+0000 7fa8d6000700 1 -- 172.21.15.99:0/3930024312 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004450 con 0x7fa8e0159220 2024-03-29T09:09:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.033+0000 7fa8d6000700 1 -- 172.21.15.99:0/3930024312 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d00045b0 con 0x7fa8e0159220 2024-03-29T09:09:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.033+0000 7fa8d6000700 1 -- 172.21.15.99:0/3930024312 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d00038e0 con 0x7fa8e0159220 2024-03-29T09:09:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.033+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e01f36f0 con 0x7fa8e0159220 2024-03-29T09:09:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.033+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e01f3cb0 con 0x7fa8e0159220 2024-03-29T09:09:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.034+0000 7f1c92000700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c7c06ace0 0x7f1c7c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.034+0000 7f1c92000700 1 -- 172.21.15.99:0/122047961 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1c880890c0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.035+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e00677a0 con 0x7fa8e0159220 2024-03-29T09:09:29.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.035+0000 7eff81a00700 1 Processor -- start 2024-03-29T09:09:29.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.036+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc0053f0 con 0x7fabfc0e3af0 2024-03-29T09:09:29.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.036+0000 7eff81a00700 1 -- start start 2024-03-29T09:09:29.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.036+0000 7f1c99e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c7c06ace0 0x7f1c7c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.036+0000 7fabf7e00700 1 -- 172.21.15.99:0/18603529 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fabf001f040 con 0x7fabfc0e3af0 2024-03-29T09:09:29.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.038+0000 7f1c99e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c7c06ace0 0x7f1c7c06d1a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c000f60 tx=0x7f1c8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.038+0000 7fabf7e00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ace0 0x7fabe406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.038+0000 7eff81a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff7c0b3b00 0x7eff7c0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7fac03e00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ace0 0x7fabe406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7eff81a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 0x7eff7c13f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7eff81a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7fabf7e00700 1 -- 172.21.15.99:0/18603529 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fabf005e550 con 0x7fabfc0e3af0 2024-03-29T09:09:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.038+0000 7fa8d6000700 1 -- 172.21.15.99:0/3930024312 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa8d0003a40 con 0x7fa8e0159220 2024-03-29T09:09:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7fa8d6000700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8c006ace0 0x7fa8c006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7eff81a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c0cc540 con 0x7eff7c0b3b00 2024-03-29T09:09:29.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7eff81a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c13fcb0 con 0x7eff7c1439f0 2024-03-29T09:09:29.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7fa8d6000700 1 -- 172.21.15.99:0/3930024312 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8d0059ce0 con 0x7fa8e0159220 2024-03-29T09:09:29.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7eff81a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c0cd5a0 con 0x7eff7c15e300 2024-03-29T09:09:29.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.039+0000 7eff7b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:32968/0 (socket says 172.21.15.99:32968) 2024-03-29T09:09:29.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7b400700 1 -- 172.21.15.99:0/3438478286 learned_addr learned my addr 172.21.15.99:0/3438478286 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7a000700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 0x7eff7c13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7aa00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff7c0b3b00 0x7eff7c0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7b400700 1 -- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 msgr2=0x7eff7c13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7b400700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 0x7eff7c13f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7b400700 1 -- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff7c0b3b00 msgr2=0x7eff7c0cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7fa8dde00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8c006ace0 0x7fa8c006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7b400700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff7c0b3b00 0x7eff7c0cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7b400700 1 -- 172.21.15.99:0/3438478286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c009550 con 0x7eff7c15e300 2024-03-29T09:09:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7eff7aa00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff7c0b3b00 0x7eff7c0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.040+0000 7f1c92000700 1 -- 172.21.15.99:0/122047961 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1c9c071ee0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.041+0000 7eff7b400700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0cd060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c790 tx=0x7eff7400cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.041+0000 7fac03e00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ace0 0x7fabe406d1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabf8006b20 tx=0x7fabf8006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.042+0000 7fa8dde00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8c006ace0 0x7fa8c006d1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009b20 tx=0x7fa8d8003a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.043+0000 7eff72a00700 1 -- 172.21.15.99:0/3438478286 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74003e30 con 0x7eff7c15e300 2024-03-29T09:09:29.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.043+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c0cbca0 con 0x7eff7c15e300 2024-03-29T09:09:29.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.043+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c0cc260 con 0x7eff7c15e300 2024-03-29T09:09:29.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.044+0000 7fabf7e00700 1 -- 172.21.15.99:0/18603529 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fabf005a920 con 0x7fabfc0e3af0 2024-03-29T09:09:29.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.044+0000 7eff72a00700 1 -- 172.21.15.99:0/3438478286 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff74005070 con 0x7eff7c15e300 2024-03-29T09:09:29.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.044+0000 7eff72a00700 1 -- 172.21.15.99:0/3438478286 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74003e30 con 0x7eff7c15e300 2024-03-29T09:09:29.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.045+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c0677a0 con 0x7eff7c15e300 2024-03-29T09:09:29.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.045+0000 7fa8d6000700 1 -- 172.21.15.99:0/3930024312 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa8d00560b0 con 0x7fa8e0159220 2024-03-29T09:09:29.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.047+0000 7eff72a00700 1 -- 172.21.15.99:0/3438478286 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7eff740051e0 con 0x7eff7c15e300 2024-03-29T09:09:29.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.049+0000 7eff72a00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff5c06acf0 0x7eff5c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.050+0000 7eff72a00700 1 -- 172.21.15.99:0/3438478286 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7eff740581e0 con 0x7eff7c15e300 2024-03-29T09:09:29.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.050+0000 7eff7aa00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff5c06acf0 0x7eff5c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.052+0000 7eff7aa00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff5c06acf0 0x7eff5c06d1b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff7c1f6170 tx=0x7eff6c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.055+0000 7eff72a00700 1 -- 172.21.15.99:0/3438478286 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7eff740545b0 con 0x7eff7c15e300 2024-03-29T09:09:29.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.068+0000 7fe66aa00700 1 Processor -- start 2024-03-29T09:09:29.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.068+0000 7fe66aa00700 1 -- start start 2024-03-29T09:09:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.072+0000 7fe66aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe65c050290 0x7fe65c0df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.073+0000 7fe66aa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0532e0 0x7fe65c006380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.073+0000 7fe66aa00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65c005ef0 con 0x7fe65c0dd040 2024-03-29T09:09:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.073+0000 7fe66aa00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c04f890 con 0x7fe65c050290 2024-03-29T09:09:29.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.073+0000 7fe66aa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c053730 con 0x7fe65c0532e0 2024-03-29T09:09:29.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.074+0000 7fe668c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe65c050290 0x7fe65c0df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.074+0000 7fe669600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe65c0dd040 0x7fe65c04fb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49652/0 (socket says 172.21.15.99:49652) 2024-03-29T09:09:29.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.074+0000 7fe669600700 1 -- 172.21.15.99:0/4104861033 learned_addr learned my addr 172.21.15.99:0/4104861033 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:29.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.074+0000 7fe66a000700 1 --2- 172.21.15.99:0/4104861033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0532e0 0x7fe65c006380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.074+0000 7fe668c00700 1 -- 172.21.15.99:0/4104861033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0532e0 msgr2=0x7fe65c006380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.074+0000 7fe668c00700 1 --2- 172.21.15.99:0/4104861033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0532e0 0x7fe65c006380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.074+0000 7fe668c00700 1 -- 172.21.15.99:0/4104861033 >> v1:172.21.15.99:6789/0 conn(0x7fe65c0dd040 legacy=0x7fe65c04fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.075+0000 7fe668c00700 1 -- 172.21.15.99:0/4104861033 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0539e0 con 0x7fe65c050290 2024-03-29T09:09:29.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.076+0000 7fe668c00700 1 --2- 172.21.15.99:0/4104861033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c050290 0x7fe65c0df870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b8f0 tx=0x7fe65800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5c5fc035b54c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.077+0000 7fe663e00700 1 -- 172.21.15.99:0/4104861033 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65800c9e0 con 0x7fe65c050290 2024-03-29T09:09:29.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.078+0000 7fe663e00700 1 -- 172.21.15.99:0/4104861033 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65800cb40 con 0x7fe65c050290 2024-03-29T09:09:29.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.078+0000 7fe663e00700 1 -- 172.21.15.99:0/4104861033 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65800c9e0 con 0x7fe65c050290 2024-03-29T09:09:29.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.078+0000 7fe66aa00700 1 -- 172.21.15.99:0/4104861033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c050290 msgr2=0x7fe65c0df870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.078+0000 7fe66aa00700 1 --2- 172.21.15.99:0/4104861033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c050290 0x7fe65c0df870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe65800b8f0 tx=0x7fe65800bc00 comp rx=0 tx=0).stop 2024-03-29T09:09:29.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.079+0000 7fe66aa00700 1 -- 172.21.15.99:0/4104861033 shutdown_connections 2024-03-29T09:09:29.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.079+0000 7fe66aa00700 1 --2- 172.21.15.99:0/4104861033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0532e0 0x7fe65c006380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.079+0000 7fe66aa00700 1 --2- 172.21.15.99:0/4104861033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c050290 0x7fe65c0df870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.080+0000 7fe66aa00700 1 -- 172.21.15.99:0/4104861033 >> 172.21.15.99:0/4104861033 conn(0x7fe65c054b20 msgr2=0x7fe65c054f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.080+0000 7fe66aa00700 1 -- 172.21.15.99:0/4104861033 shutdown_connections 2024-03-29T09:09:29.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.080+0000 7fe66aa00700 1 -- 172.21.15.99:0/4104861033 wait complete. 2024-03-29T09:09:29.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.083+0000 7fe66aa00700 1 Processor -- start 2024-03-29T09:09:29.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.084+0000 7fe66aa00700 1 -- start start 2024-03-29T09:09:29.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.086+0000 7fe66aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65c050290 0x7fe65c02bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.086+0000 7fe66aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c0532e0 0x7fe65c1880e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.086+0000 7fe669600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65c050290 0x7fe65c02bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.086+0000 7fe669600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65c050290 0x7fe65c02bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55352/0 (socket says 172.21.15.99:55352) 2024-03-29T09:09:29.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fe669600700 1 -- 172.21.15.99:0/678855597 learned_addr learned my addr 172.21.15.99:0/678855597 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fd112200700 1 Processor -- start 2024-03-29T09:09:29.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fe668c00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c0532e0 0x7fe65c1880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fe66aa00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0dd040 0x7fe65c0dd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fd112200700 1 -- start start 2024-03-29T09:09:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c02c7e0 con 0x7fe65c0532e0 2024-03-29T09:09:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c02cac0 con 0x7fe65c050290 2024-03-29T09:09:29.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.087+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c02cda0 con 0x7fe65c0dd040 2024-03-29T09:09:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.088+0000 7fe66a000700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0dd040 0x7fe65c0dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.088+0000 7fe66a000700 1 -- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65c050290 msgr2=0x7fe65c02bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.088+0000 7fe66a000700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65c050290 0x7fe65c02bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.088+0000 7fe66a000700 1 -- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c0532e0 msgr2=0x7fe65c1880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.088+0000 7fe66a000700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c0532e0 0x7fe65c1880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.088+0000 7fe66a000700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65800b550 con 0x7fe65c0dd040 2024-03-29T09:09:29.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.088+0000 7fe66a000700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0dd040 0x7fe65c0dd420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe66404f8d0 tx=0x7fe66406bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.089+0000 7fe661600700 1 -- 172.21.15.99:0/678855597 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640734a0 con 0x7fe65c0dd040 2024-03-29T09:09:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.089+0000 7fe661600700 1 -- 172.21.15.99:0/678855597 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe664073600 con 0x7fe65c0dd040 2024-03-29T09:09:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.089+0000 7fe661600700 1 -- 172.21.15.99:0/678855597 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6640933f0 con 0x7fe65c0dd040 2024-03-29T09:09:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.089+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c02d080 con 0x7fe65c0dd040 2024-03-29T09:09:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.089+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c02c440 con 0x7fe65c0dd040 2024-03-29T09:09:29.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.091+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c008a30 con 0x7fe65c0dd040 2024-03-29T09:09:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.092+0000 7fd112200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd104055640 0x7fd1040df870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.093+0000 7fd110e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd1040dd040 0x7fd1040e2e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49664/0 (socket says 172.21.15.99:49664) 2024-03-29T09:09:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.093+0000 7fd110e00700 1 -- 172.21.15.99:0/1306919947 learned_addr learned my addr 172.21.15.99:0/1306919947 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.093+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd104005ef0 con 0x7fd1040dd040 2024-03-29T09:09:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.093+0000 7fd10be00700 1 --2- 172.21.15.99:0/1306919947 >> v2:172.21.15.99:3300/0 conn(0x7fd104055640 0x7fd1040df870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.093+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd10404f890 con 0x7fd104007390 2024-03-29T09:09:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.092+0000 7fe661600700 1 -- 172.21.15.99:0/678855597 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe664093570 con 0x7fe65c0dd040 2024-03-29T09:09:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1040e3c80 con 0x7fd104055640 2024-03-29T09:09:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411110561 0 0) 0x7fd104005ef0 con 0x7fd1040dd040 2024-03-29T09:09:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.093+0000 7fe661600700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe64406ace0 0x7fe64406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.093+0000 7fe661600700 1 -- 172.21.15.99:0/678855597 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe664102d00 con 0x7fe65c0dd040 2024-03-29T09:09:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f40079c0 con 0x7fd1040dd040 2024-03-29T09:09:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486957706 0 0) 0x7fd10404f890 con 0x7fd104007390 2024-03-29T09:09:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd104005ef0 con 0x7fd104007390 2024-03-29T09:09:29.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fe669600700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe64406ace0 0x7fe64406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3304164167 0 0) 0x7fd104005ef0 con 0x7fd104007390 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 >> v2:172.21.15.99:3300/0 conn(0x7fd104055640 msgr2=0x7fd1040df870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 --2- 172.21.15.99:0/1306919947 >> v2:172.21.15.99:3300/0 conn(0x7fd104055640 0x7fd1040df870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 >> v1:172.21.15.99:6789/0 conn(0x7fd1040dd040 legacy=0x7fd1040e2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040539e0 con 0x7fd104007390 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 <== mon.1 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fd10c0623e0 con 0x7fd104007390 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.094+0000 7fd10be00700 1 --2- 172.21.15.99:0/1306919947 >> v2:172.21.15.99:3300/0 conn(0x7fd104055640 0x7fd1040df870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 >> v1:172.21.15.133:6789/0 conn(0x7fd104007390 legacy=0x7fd1040e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10b400700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4009000 0x7fd0f40093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10b400700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0f4007c00 0x7fd0f4008080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10b400700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0f40085c0 0x7fd0f4009df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100001230 con 0x7fd0f4009000 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040e3c80 con 0x7fd0f40085c0 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f400a330 con 0x7fd0f4007c00 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd10be00700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0f4007c00 0x7fd0f4008080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fd111800700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0f40085c0 0x7fd0f4009df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.096+0000 7fd10be00700 1 -- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0f40085c0 msgr2=0x7fd0f4009df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.096+0000 7fd10be00700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0f40085c0 0x7fd0f4009df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.096+0000 7fd10be00700 1 -- 172.21.15.99:0/1306919947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4009000 msgr2=0x7fd0f40093e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.096+0000 7fd10be00700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4009000 0x7fd0f40093e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.096+0000 7fd10be00700 1 -- 172.21.15.99:0/1306919947 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104005ef0 con 0x7fd0f4007c00 2024-03-29T09:09:29.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.096+0000 7fd10be00700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0f4007c00 0x7fd0f4008080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800be10 tx=0x7fd0f4009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e5cdc158b9e1af server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.095+0000 7fe669600700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe64406ace0 0x7fe64406d1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe650004930 tx=0x7fe65000a2e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.097+0000 7fd111800700 1 -- 172.21.15.99:0/1306919947 reap_dead start 2024-03-29T09:09:29.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.097+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800de20 con 0x7fd0f4007c00 2024-03-29T09:09:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.097+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f8004030 con 0x7fd0f4007c00 2024-03-29T09:09:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.097+0000 7fd10b400700 1 -- 172.21.15.99:0/1306919947 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80043a0 con 0x7fd0f4007c00 2024-03-29T09:09:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.098+0000 7fe661600700 1 -- 172.21.15.99:0/678855597 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe6640cdbd0 con 0x7fe65c0dd040 2024-03-29T09:09:29.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.098+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0f4007c00 msgr2=0x7fd0f4008080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.098+0000 7fd112200700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0f4007c00 0x7fd0f4008080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800be10 tx=0x7fd0f4009c90 comp rx=0 tx=0).stop 2024-03-29T09:09:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.099+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 shutdown_connections 2024-03-29T09:09:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.099+0000 7fd112200700 1 --2- 172.21.15.99:0/1306919947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0f4007c00 0x7fd0f4008080 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.099+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 >> 172.21.15.99:0/1306919947 conn(0x7fd104054b20 msgr2=0x7fd104054f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.100+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 shutdown_connections 2024-03-29T09:09:29.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.100+0000 7fd112200700 1 -- 172.21.15.99:0/1306919947 wait complete. 2024-03-29T09:09:29.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.104+0000 7fd112200700 1 Processor -- start 2024-03-29T09:09:29.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.104+0000 7fd112200700 1 -- start start 2024-03-29T09:09:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.106+0000 7fd112200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4007c00 0x7fd1040c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.106+0000 7fd112200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1040c9390 0x7fd1040c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.106+0000 7fd112200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1040c7220 0x7fd1040da830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.106+0000 7fd112200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040c8810 con 0x7fd0f4007c00 2024-03-29T09:09:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.106+0000 7fd112200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040c7b90 con 0x7fd1040c7220 2024-03-29T09:09:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.106+0000 7fd112200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040c7e70 con 0x7fd1040c9390 2024-03-29T09:09:29.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.107+0000 7fd10be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1040c9390 0x7fd1040c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.107+0000 7fd110e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4007c00 0x7fd1040c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.107+0000 7fd110e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4007c00 0x7fd1040c8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53348/0 (socket says 172.21.15.99:53348) 2024-03-29T09:09:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.107+0000 7fd110e00700 1 -- 172.21.15.99:0/792925662 learned_addr learned my addr 172.21.15.99:0/792925662 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.107+0000 7fd111800700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1040c7220 0x7fd1040da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.108+0000 7fd10be00700 1 -- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1040c7220 msgr2=0x7fd1040da830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.108+0000 7fd10be00700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1040c7220 0x7fd1040da830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.108+0000 7fd10be00700 1 -- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4007c00 msgr2=0x7fd1040c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.108+0000 7fd10be00700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4007c00 0x7fd1040c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.108+0000 7fd10be00700 1 -- 172.21.15.99:0/792925662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040c8150 con 0x7fd1040c9390 2024-03-29T09:09:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.109+0000 7fd10be00700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1040c9390 0x7fd1040c9c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8017000 tx=0x7fd0f800a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.109+0000 7fd108c00700 1 -- 172.21.15.99:0/792925662 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8003ea0 con 0x7fd1040c9390 2024-03-29T09:09:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.109+0000 7fd108c00700 1 -- 172.21.15.99:0/792925662 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f801f440 con 0x7fd1040c9390 2024-03-29T09:09:29.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.109+0000 7fd108c00700 1 -- 172.21.15.99:0/792925662 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f801f810 con 0x7fd1040c9390 2024-03-29T09:09:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.110+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1040c8430 con 0x7fd1040c9390 2024-03-29T09:09:29.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.110+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1040c76d0 con 0x7fd1040c9390 2024-03-29T09:09:29.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.113+0000 7fd108c00700 1 -- 172.21.15.99:0/792925662 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd0f801f970 con 0x7fd1040c9390 2024-03-29T09:09:29.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.114+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1040c7fb0 con 0x7fd1040c9390 2024-03-29T09:09:29.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.115+0000 7fd108c00700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd0ec06ad60 0x7fd0ec06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:29.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.115+0000 7fd108c00700 1 -- 172.21.15.99:0/792925662 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0f808a810 con 0x7fd1040c9390 2024-03-29T09:09:29.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.116+0000 7fd110e00700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd0ec06ad60 0x7fd0ec06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:29.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.118+0000 7fd110e00700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd0ec06ad60 0x7fd0ec06d220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd100000f60 tx=0x7fd1000019b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:29.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.121+0000 7fd108c00700 1 -- 172.21.15.99:0/792925662 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd1040c7fb0 con 0x7fd1040c9390 2024-03-29T09:09:29.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.467+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f51b4067730 con 0x7f51b4161b20 2024-03-29T09:09:29.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.468+0000 7f51ab400700 1 -- 172.21.15.99:0/3353278683 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f519c017090 con 0x7f51b4161b20 2024-03-29T09:09:29.467 INFO:teuthology.orchestra.run.smithi099.stdout:55834574858 2024-03-29T09:09:29.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.473+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f519006ac90 msgr2=0x7f519006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.473+0000 7f51b3e00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f519006ac90 0x7f519006d150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009250 tx=0x7f51ac008030 comp rx=0 tx=0).stop 2024-03-29T09:09:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.473+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b4161b20 msgr2=0x7f51b415c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.473+0000 7f51b3e00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b4161b20 0x7f51b415c3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c390 tx=0x7f519c00c6a0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 shutdown_connections 2024-03-29T09:09:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f519006ac90 0x7f519006d150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b4161b20 0x7f51b415c3d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51b40cef40 0x7f51b409c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 --2- 172.21.15.99:0/3353278683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51b414afe0 0x7f51b41ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 >> 172.21.15.99:0/3353278683 conn(0x7f51b4066540 msgr2=0x7f51b4070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 shutdown_connections 2024-03-29T09:09:29.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.474+0000 7f51b3e00700 1 -- 172.21.15.99:0/3353278683 wait complete. 2024-03-29T09:09:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.491+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f500c0ce150 con 0x7f500c0532f0 2024-03-29T09:09:29.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.492+0000 7f500a000700 1 -- 172.21.15.99:0/4091185560 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f500c0ce150 con 0x7f500c0532f0 2024-03-29T09:09:29.495 INFO:teuthology.orchestra.run.smithi099.stdout:55834574858 2024-03-29T09:09:29.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ff406ace0 msgr2=0x7f4ff406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ff406ace0 0x7f4ff406d1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f500c0df650 tx=0x7f5004008040 comp rx=0 tx=0).stop 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0532f0 msgr2=0x7f500c180c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0532f0 0x7f500c180c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f500000c4d0 tx=0x7f500000c7e0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 shutdown_connections 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ff406ace0 0x7f4ff406d1a0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f500c0df650 tx=0x7f5004008040 comp rx=0 tx=0).stop 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f500c0df6e0 0x7f500c181180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f500c00c100 0x7f500c04aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 --2- 172.21.15.99:0/4091185560 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f500c0532f0 0x7f500c180c40 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f500000c4d0 tx=0x7f500000c7e0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.497+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 >> 172.21.15.99:0/4091185560 conn(0x7f500c054b20 msgr2=0x7f500c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.498+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 shutdown_connections 2024-03-29T09:09:29.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.498+0000 7f5013200700 1 -- 172.21.15.99:0/4091185560 wait complete. 2024-03-29T09:09:29.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-03-29T09:09:29.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.509+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1c9c071ee0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.510+0000 7f1c92000700 1 -- 172.21.15.99:0/122047961 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c9c071ee0 con 0x7f1c9c0b3b00 2024-03-29T09:09:29.509 INFO:teuthology.orchestra.run.smithi099.stdout:55834574857 2024-03-29T09:09:29.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.514+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c7c06ace0 msgr2=0x7f1c7c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.514+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c7c06ace0 0x7f1c7c06d1a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c000f60 tx=0x7f1c8c009040 comp rx=0 tx=0).stop 2024-03-29T09:09:29.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.514+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9c0b3b00 msgr2=0x7f1c9c136470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.514+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9c0b3b00 0x7f1c9c136470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800c7c0 tx=0x7f1c8800cb80 comp rx=0 tx=0).stop 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 shutdown_connections 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c7c06ace0 0x7f1c7c06d1a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c9c1439f0 0x7f1c9c142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c9c0715f0 0x7f1c9c146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 --2- 172.21.15.99:0/122047961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c9c0b3b00 0x7f1c9c136470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 >> 172.21.15.99:0/122047961 conn(0x7f1c9c0b0bc0 msgr2=0x7f1c9c0aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 shutdown_connections 2024-03-29T09:09:29.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.515+0000 7f1ca0e00700 1 -- 172.21.15.99:0/122047961 wait complete. 2024-03-29T09:09:29.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.519+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7eff7c0b0970 con 0x7eff7c15e300 2024-03-29T09:09:29.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.519+0000 7eff72a00700 1 -- 172.21.15.99:0/3438478286 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff740545b0 con 0x7eff7c15e300 2024-03-29T09:09:29.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.520+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa8e01583d0 con 0x7fa8e0159220 2024-03-29T09:09:29.519 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:29.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.520+0000 7fa8d6000700 1 -- 172.21.15.99:0/3930024312 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8d0015020 con 0x7fa8e0159220 2024-03-29T09:09:29.520 INFO:teuthology.orchestra.run.smithi099.stdout:55834574858 2024-03-29T09:09:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.522+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff5c06acf0 msgr2=0x7eff5c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.522+0000 7eff81a00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff5c06acf0 0x7eff5c06d1b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7eff7c1f6170 tx=0x7eff6c008030 comp rx=0 tx=0).stop 2024-03-29T09:09:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.522+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 msgr2=0x7eff7c0cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.522+0000 7eff81a00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0cd060 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c790 tx=0x7eff7400cb50 comp rx=0 tx=0).stop 2024-03-29T09:09:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.523+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 shutdown_connections 2024-03-29T09:09:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.523+0000 7eff81a00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff5c06acf0 0x7eff5c06d1b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.523+0000 7eff81a00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff7c15e300 0x7eff7c0cd060 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.523+0000 7eff81a00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff7c0b3b00 0x7eff7c0cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.523+0000 7eff81a00700 1 --2- 172.21.15.99:0/3438478286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff7c1439f0 0x7eff7c13f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.523+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 >> 172.21.15.99:0/3438478286 conn(0x7eff7c0b0bc0 msgr2=0x7eff7c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.523+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fabfc037ef0 con 0x7fabfc0e3af0 2024-03-29T09:09:29.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.524+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 shutdown_connections 2024-03-29T09:09:29.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.524+0000 7eff81a00700 1 -- 172.21.15.99:0/3438478286 wait complete. 2024-03-29T09:09:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.524+0000 7fabf7e00700 1 -- 172.21.15.99:0/18603529 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fabf0031090 con 0x7fabfc0e3af0 2024-03-29T09:09:29.524 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8c006ace0 msgr2=0x7fa8c006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8c006ace0 0x7fa8c006d1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009b20 tx=0x7fa8d8003a60 comp rx=0 tx=0).stop 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8e0159220 msgr2=0x7fa8e0152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8e0159220 0x7fa8e0152a70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009fd0 tx=0x7fa8d0004750 comp rx=0 tx=0).stop 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 shutdown_connections 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8c006ace0 0x7fa8c006d1a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8e0159220 0x7fa8e0152a70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8e006e630 0x7fa8e014e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 --2- 172.21.15.99:0/3930024312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8e00b1ec0 0x7fa8e01522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.525+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 >> 172.21.15.99:0/3930024312 conn(0x7fa8e0066540 msgr2=0x7fa8e015b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.528+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ace0 msgr2=0x7fabe406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.528+0000 7fac0aa00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ace0 0x7fabe406d1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabf8006b20 tx=0x7fabf8006ab0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.528+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 msgr2=0x7fabfc0e17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.528+0000 7fac0aa00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e17c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fabf00090b0 tx=0x7fabf0004750 comp rx=0 tx=0).stop 2024-03-29T09:09:29.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-03-29T09:09:29.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.529+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 shutdown_connections 2024-03-29T09:09:29.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fa8e4c00700 1 -- 172.21.15.99:0/3930024312 wait complete. 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 shutdown_connections 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ace0 0x7fabe406d1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabfc0dbea0 0x7fabfc0de0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabfc006100 0x7fabfc0d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 --2- 172.21.15.99:0/18603529 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fabfc0e3af0 0x7fabfc0e17c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 >> 172.21.15.99:0/18603529 conn(0x7fabfc054b20 msgr2=0x7fabfc04dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 shutdown_connections 2024-03-29T09:09:29.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.530+0000 7fac0aa00700 1 -- 172.21.15.99:0/18603529 wait complete. 2024-03-29T09:09:29.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.0 2024-03-29T09:09:29.548 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-03-29T09:09:29.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-03-29T09:09:29.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.553+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe65c0053f0 con 0x7fe65c0dd040 2024-03-29T09:09:29.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.554+0000 7fe661600700 1 -- 172.21.15.99:0/678855597 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe6640d11f0 con 0x7fe65c0dd040 2024-03-29T09:09:29.553 INFO:teuthology.orchestra.run.smithi099.stdout:55834574860 2024-03-29T09:09:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe64406ace0 msgr2=0x7fe64406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe64406ace0 0x7fe64406d1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe650004930 tx=0x7fe65000a2e0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0dd040 msgr2=0x7fe65c0dd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0dd040 0x7fe65c0dd420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe66404f8d0 tx=0x7fe66406bc30 comp rx=0 tx=0).stop 2024-03-29T09:09:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 shutdown_connections 2024-03-29T09:09:29.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe64406ace0 0x7fe64406d1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe65c0dd040 0x7fe65c0dd420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65c050290 0x7fe65c02bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 --2- 172.21.15.99:0/678855597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe65c0532e0 0x7fe65c1880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 >> 172.21.15.99:0/678855597 conn(0x7fe65c054b20 msgr2=0x7fe65c0de600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 shutdown_connections 2024-03-29T09:09:29.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.557+0000 7fe66aa00700 1 -- 172.21.15.99:0/678855597 wait complete. 2024-03-29T09:09:29.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.6 2024-03-29T09:09:29.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.574+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd1040c8950 con 0x7fd1040c9390 2024-03-29T09:09:29.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.574+0000 7fd108c00700 1 -- 172.21.15.99:0/792925662 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1040c8950 con 0x7fd1040c9390 2024-03-29T09:09:29.573 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:29.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574860 for osd.5 2024-03-29T09:09:29.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd0ec06ad60 msgr2=0x7fd0ec06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd0ec06ad60 0x7fd0ec06d220 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd100000f60 tx=0x7fd1000019b0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1040c9390 msgr2=0x7fd1040c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:29.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1040c9390 0x7fd1040c9c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8017000 tx=0x7fd0f800a1f0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 shutdown_connections 2024-03-29T09:09:29.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd0ec06ad60 0x7fd0ec06d220 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1040c7220 0x7fd1040da830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd0f4007c00 0x7fd1040c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 --2- 172.21.15.99:0/792925662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd1040c9390 0x7fd1040c9c30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:29.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 >> 172.21.15.99:0/792925662 conn(0x7fd104054b20 msgr2=0x7fd1040e1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:29.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 shutdown_connections 2024-03-29T09:09:29.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:29.577+0000 7fd112200700 1 -- 172.21.15.99:0/792925662 wait complete. 2024-03-29T09:09:29.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.7 2024-03-29T09:09:30.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T09:09:30.530 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T09:09:30.548 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T09:09:30.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T09:09:30.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T09:09:30.561 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T09:09:30.575 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T09:09:30.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T09:09:30.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.815+0000 7f0422600700 1 Processor -- start 2024-03-29T09:09:30.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.815+0000 7f0422600700 1 -- start start 2024-03-29T09:09:30.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.818+0000 7f0422600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0414053790 0x7f0414053c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.818+0000 7f0422600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041403ee20 con 0x7f04140e32c0 2024-03-29T09:09:30.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.819+0000 7f0422600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0414054150 con 0x7f04140063d0 2024-03-29T09:09:30.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.819+0000 7f0422600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140578f0 con 0x7f0414053790 2024-03-29T09:09:30.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.819+0000 7f041be00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f04140063d0 0x7f0414006860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:52084/0 (socket says 172.21.15.99:52084) 2024-03-29T09:09:30.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.819+0000 7f041be00700 1 -- 172.21.15.99:0/1935642540 learned_addr learned my addr 172.21.15.99:0/1935642540 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:30.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.819+0000 7f041aa00700 1 --2- 172.21.15.99:0/1935642540 >> v2:172.21.15.133:3300/0 conn(0x7f0414053790 0x7f0414053c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.820+0000 7f041a000700 1 -- 172.21.15.99:0/1935642540 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83233030 0 0) 0x7f041c0689d0 con 0x7f04140063d0 2024-03-29T09:09:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.821+0000 7f041a000700 1 -- 172.21.15.99:0/1935642540 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04040035b0 con 0x7f04140063d0 2024-03-29T09:09:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.821+0000 7f041aa00700 1 -- 172.21.15.99:0/1935642540 >> v1:172.21.15.133:6789/0 conn(0x7f04140063d0 legacy=0x7f0414006860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.821+0000 7f041aa00700 1 -- 172.21.15.99:0/1935642540 >> v1:172.21.15.99:6789/0 conn(0x7f04140e32c0 legacy=0x7f0414058700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.821+0000 7f041aa00700 1 -- 172.21.15.99:0/1935642540 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041403e860 con 0x7f0414053790 2024-03-29T09:09:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.821+0000 7f041aa00700 1 --2- 172.21.15.99:0/1935642540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f0414053c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0410009840 tx=0x7f0410009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76ae976aa5f547ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.822+0000 7f041a000700 1 -- 172.21.15.99:0/1935642540 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410011830 con 0x7f0414053790 2024-03-29T09:09:30.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.822+0000 7f041a000700 1 -- 172.21.15.99:0/1935642540 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410011990 con 0x7f0414053790 2024-03-29T09:09:30.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.822+0000 7f041a000700 1 -- 172.21.15.99:0/1935642540 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410011d20 con 0x7f0414053790 2024-03-29T09:09:30.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.823+0000 7f0422600700 1 -- 172.21.15.99:0/1935642540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 msgr2=0x7f0414053c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.823+0000 7f0422600700 1 --2- 172.21.15.99:0/1935642540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f0414053c10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0410009840 tx=0x7f0410009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:30.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.823+0000 7f0422600700 1 -- 172.21.15.99:0/1935642540 shutdown_connections 2024-03-29T09:09:30.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.823+0000 7f0422600700 1 --2- 172.21.15.99:0/1935642540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f0414053c10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.823+0000 7f0422600700 1 -- 172.21.15.99:0/1935642540 >> 172.21.15.99:0/1935642540 conn(0x7f04140559c0 msgr2=0x7f0414055da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:30.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.823+0000 7f0422600700 1 -- 172.21.15.99:0/1935642540 shutdown_connections 2024-03-29T09:09:30.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.824+0000 7f0422600700 1 -- 172.21.15.99:0/1935642540 wait complete. 2024-03-29T09:09:30.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.827+0000 7f0422600700 1 Processor -- start 2024-03-29T09:09:30.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.827+0000 7f0422600700 1 -- start start 2024-03-29T09:09:30.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.829+0000 7f0422600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140063d0 0x7f04140d9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.830+0000 7f0422600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f04140de150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.830+0000 7f0422600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04140e32c0 0x7f04141849e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.830+0000 7f041aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f04140de150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.830+0000 7f041aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f04140de150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55404/0 (socket says 172.21.15.99:55404) 2024-03-29T09:09:30.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.830+0000 7f041aa00700 1 -- 172.21.15.99:0/3917646957 learned_addr learned my addr 172.21.15.99:0/3917646957 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.830+0000 7f041be00700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04140e32c0 0x7f04141849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041404eed0 con 0x7f04140e32c0 2024-03-29T09:09:30.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041404ebf0 con 0x7f0414053790 2024-03-29T09:09:30.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140d9470 con 0x7f04140063d0 2024-03-29T09:09:30.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f041aa00700 1 -- 172.21.15.99:0/3917646957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140063d0 msgr2=0x7f04140d9f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T09:09:30.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f041aa00700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140063d0 0x7f04140d9f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f041aa00700 1 -- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04140e32c0 msgr2=0x7f04141849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f041aa00700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04140e32c0 0x7f04141849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.831+0000 7f041aa00700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04100094a0 con 0x7f0414053790 2024-03-29T09:09:30.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.833+0000 7f041aa00700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f04140de150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0410000c00 tx=0x7f0410004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.833+0000 7f040f400700 1 -- 172.21.15.99:0/3917646957 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100036a0 con 0x7f0414053790 2024-03-29T09:09:30.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.833+0000 7f040f400700 1 -- 172.21.15.99:0/3917646957 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041001f070 con 0x7f0414053790 2024-03-29T09:09:30.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.833+0000 7f040f400700 1 -- 172.21.15.99:0/3917646957 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410003a60 con 0x7f0414053790 2024-03-29T09:09:30.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.834+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04140d9750 con 0x7f0414053790 2024-03-29T09:09:30.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.834+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04140de690 con 0x7f0414053790 2024-03-29T09:09:30.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.837+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04140d92d0 con 0x7f0414053790 2024-03-29T09:09:30.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.840+0000 7f040f400700 1 -- 172.21.15.99:0/3917646957 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0410003bc0 con 0x7f0414053790 2024-03-29T09:09:30.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.841+0000 7f040f400700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03fc06ac90 0x7f03fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.842+0000 7f041b400700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03fc06ac90 0x7f03fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.842+0000 7f040f400700 1 -- 172.21.15.99:0/3917646957 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0410021070 con 0x7f0414053790 2024-03-29T09:09:30.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.843+0000 7f041b400700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03fc06ac90 0x7f03fc06d150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f04080014e0 tx=0x7f0408007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.847+0000 7f040f400700 1 -- 172.21.15.99:0/3917646957 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f04140d92d0 con 0x7f0414053790 2024-03-29T09:09:30.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.870+0000 7f22bc600700 1 Processor -- start 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.871+0000 7f22bc600700 1 -- start start 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.873+0000 7f22bc600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f22b4153690 0x7f22b40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.873+0000 7f22bc600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.873+0000 7f22bc600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b406d360 con 0x7f22b40b3a00 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.873+0000 7f22bc600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b41597a0 con 0x7f22b4153690 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.873+0000 7f22bc600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4159a80 con 0x7f22b4070ab0 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.873+0000 7f22b9c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f22b4153690 0x7f22b40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.874+0000 7f22b9c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f22b4153690 0x7f22b40b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53364/0 (socket says 172.21.15.99:53364) 2024-03-29T09:09:30.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.874+0000 7f22b9c00700 1 -- 172.21.15.99:0/593780833 learned_addr learned my addr 172.21.15.99:0/593780833 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.875+0000 7f22b3e00700 1 -- 172.21.15.99:0/593780833 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553661357 0 0) 0x7f22b406d360 con 0x7f22b40b3a00 2024-03-29T09:09:30.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.875+0000 7f22b3e00700 1 -- 172.21.15.99:0/593780833 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f229c0035b0 con 0x7f22b40b3a00 2024-03-29T09:09:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.876+0000 7f22b3e00700 1 -- 172.21.15.99:0/593780833 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153227829 0 0) 0x7f229c0035b0 con 0x7f22b40b3a00 2024-03-29T09:09:30.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.876+0000 7f22b9c00700 1 -- 172.21.15.99:0/593780833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 msgr2=0x7f22b415cc80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T09:09:30.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.877+0000 7f22ba600700 1 --2- 172.21.15.99:0/593780833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.877+0000 7f22b9c00700 1 --2- 172.21.15.99:0/593780833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b415cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.877+0000 7f22b9c00700 1 -- 172.21.15.99:0/593780833 >> v1:172.21.15.99:6789/0 conn(0x7f22b40b3a00 legacy=0x7f22b40703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.877+0000 7f22b9c00700 1 -- 172.21.15.99:0/593780833 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40b3e60 con 0x7f22b4153690 2024-03-29T09:09:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.878+0000 7f22b9c00700 1 --2- 172.21.15.99:0/593780833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b4153690 0x7f22b40b34c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009e50 tx=0x7f22ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c382fdd68f432e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.879+0000 7f22b3e00700 1 -- 172.21.15.99:0/593780833 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8001670 con 0x7f22b4153690 2024-03-29T09:09:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.879+0000 7f22b3e00700 1 -- 172.21.15.99:0/593780833 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac003a20 con 0x7f22b4153690 2024-03-29T09:09:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.879+0000 7f22b3e00700 1 -- 172.21.15.99:0/593780833 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac003dd0 con 0x7f22b4153690 2024-03-29T09:09:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.881+0000 7f22bc600700 1 -- 172.21.15.99:0/593780833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b4153690 msgr2=0x7f22b40b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.881+0000 7f22bc600700 1 --2- 172.21.15.99:0/593780833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b4153690 0x7f22b40b34c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009e50 tx=0x7f22ac00b330 comp rx=0 tx=0).stop 2024-03-29T09:09:30.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.881+0000 7f22bc600700 1 -- 172.21.15.99:0/593780833 shutdown_connections 2024-03-29T09:09:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.881+0000 7f22bc600700 1 --2- 172.21.15.99:0/593780833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.881+0000 7f22bc600700 1 --2- 172.21.15.99:0/593780833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b4153690 0x7f22b40b34c0 secure :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009e50 tx=0x7f22ac00b330 comp rx=0 tx=0).stop 2024-03-29T09:09:30.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.881+0000 7f22bc600700 1 -- 172.21.15.99:0/593780833 >> 172.21.15.99:0/593780833 conn(0x7f22b4066540 msgr2=0x7f22b4066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.882+0000 7f22bc600700 1 -- 172.21.15.99:0/593780833 shutdown_connections 2024-03-29T09:09:30.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.882+0000 7f22bc600700 1 -- 172.21.15.99:0/593780833 wait complete. 2024-03-29T09:09:30.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.885+0000 7f22bc600700 1 Processor -- start 2024-03-29T09:09:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.885+0000 7f22bc600700 1 -- start start 2024-03-29T09:09:30.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.887+0000 7f22bc600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b409c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.887+0000 7f22bc600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b40b3a00 0x7f22b4202ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.888+0000 7f22bc600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b40afb20 0x7f22b4200120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.888+0000 7f22b9c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b409c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.888+0000 7f22b9c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b409c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:33034/0 (socket says 172.21.15.99:33034) 2024-03-29T09:09:30.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.888+0000 7f22b9c00700 1 -- 172.21.15.99:0/3885508810 learned_addr learned my addr 172.21.15.99:0/3885508810 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.888+0000 7f22b9200700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b40b3a00 0x7f22b4202ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.888+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40ccd00 con 0x7f22b40b3a00 2024-03-29T09:09:30.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.889+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4203730 con 0x7f22b40afb20 2024-03-29T09:09:30.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.889+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4203a10 con 0x7f22b4070ab0 2024-03-29T09:09:30.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.889+0000 7f22ba600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b40afb20 0x7f22b4200120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.890+0000 7f22b9200700 1 -- 172.21.15.99:0/3885508810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 msgr2=0x7f22b409c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.890+0000 7f22b9200700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b409c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.890+0000 7f22b9200700 1 -- 172.21.15.99:0/3885508810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b40afb20 msgr2=0x7f22b4200120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.890+0000 7f22b9200700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b40afb20 0x7f22b4200120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.890+0000 7f22b9200700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac009b30 con 0x7f22b40b3a00 2024-03-29T09:09:30.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.892+0000 7f22b9200700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b40b3a00 0x7f22b4202ee0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c430 tx=0x7f22a800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.893+0000 7fa8f3e00700 1 Processor -- start 2024-03-29T09:09:30.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.893+0000 7fa8f3e00700 1 -- start start 2024-03-29T09:09:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.894+0000 7f22b1600700 1 -- 172.21.15.99:0/3885508810 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a80054e0 con 0x7f22b40b3a00 2024-03-29T09:09:30.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.894+0000 7f22b1600700 1 -- 172.21.15.99:0/3885508810 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a800ce40 con 0x7f22b40b3a00 2024-03-29T09:09:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.895+0000 7f22b1600700 1 -- 172.21.15.99:0/3885508810 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a80054e0 con 0x7f22b40b3a00 2024-03-29T09:09:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.895+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b4203cf0 con 0x7f22b40b3a00 2024-03-29T09:09:30.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.895+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b41ff8f0 con 0x7f22b40b3a00 2024-03-29T09:09:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.896+0000 7fa8f3e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8f40b21e0 0x7fa8f40b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.896+0000 7fa8f3e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8f4161ac0 0x7fa8f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.896+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b41ffe40 con 0x7f22b40b3a00 2024-03-29T09:09:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f3e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f41623c0 con 0x7fa8f406ce60 2024-03-29T09:09:30.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f3e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f41626a0 con 0x7fa8f40b21e0 2024-03-29T09:09:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f3e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f4162980 con 0x7fa8f4161ac0 2024-03-29T09:09:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f2000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8f4161ac0 0x7fa8f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f2a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8f40b21e0 0x7fa8f40b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f2000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa8f4161ac0 0x7fa8f406c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55420/0 (socket says 172.21.15.99:55420) 2024-03-29T09:09:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f2000700 1 -- 172.21.15.99:0/613047641 learned_addr learned my addr 172.21.15.99:0/613047641 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7f80f5e00700 1 Processor -- start 2024-03-29T09:09:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f2000700 1 -- 172.21.15.99:0/613047641 >> v2:172.21.15.99:3300/0 conn(0x7fa8f40b21e0 msgr2=0x7fa8f40b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.898+0000 7fa8f1600700 1 -- 172.21.15.99:0/613047641 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274276807 0 0) 0x7fa8f41623c0 con 0x7fa8f406ce60 2024-03-29T09:09:30.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.898+0000 7f80f5e00700 1 -- start start 2024-03-29T09:09:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.897+0000 7fa8f2a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa8f40b21e0 0x7fa8f40b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53372/0 (socket says 172.21.15.99:53372) 2024-03-29T09:09:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.899+0000 7fa8f2000700 1 --2- 172.21.15.99:0/613047641 >> v2:172.21.15.99:3300/0 conn(0x7fa8f40b21e0 0x7fa8f40b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.899+0000 7fa8f2000700 1 -- 172.21.15.99:0/613047641 >> v1:172.21.15.133:6789/0 conn(0x7fa8f406ce60 legacy=0x7fa8f40abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.899+0000 7fa8f2000700 1 -- 172.21.15.99:0/613047641 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f406c120 con 0x7fa8f4161ac0 2024-03-29T09:09:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.899+0000 7fa8f2a00700 1 --2- 172.21.15.99:0/613047641 >> v2:172.21.15.99:3300/0 conn(0x7fa8f40b21e0 0x7fa8f40b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:30.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.899+0000 7f22b1600700 1 -- 172.21.15.99:0/3885508810 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f22a8003680 con 0x7f22b40b3a00 2024-03-29T09:09:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.899+0000 7f80f5e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f015b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7f80f5e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f015c220 con 0x7f80f00b3e00 2024-03-29T09:09:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7f80f5e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f015b8f0 con 0x7f80f00b0b50 2024-03-29T09:09:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7f80f5e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f015bbd0 con 0x7f80f015ae70 2024-03-29T09:09:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7f80ef400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f80f00b3e00 0x7f80f00b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49696/0 (socket says 172.21.15.99:49696) 2024-03-29T09:09:30.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7f80ef400700 1 -- 172.21.15.99:0/3479663845 learned_addr learned my addr 172.21.15.99:0/3479663845 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7fa8f2000700 1 --2- 172.21.15.99:0/613047641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f406c920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406e500 tx=0x7fa8e0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac878d003c588029 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7f22b1600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f229406acf0 0x7f229406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.901+0000 7f22b1600700 1 -- 172.21.15.99:0/3885508810 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f22a8088520 con 0x7f22b40b3a00 2024-03-29T09:09:30.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.901+0000 7fa839c00700 1 Processor -- start 2024-03-29T09:09:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.901+0000 7fa8f1600700 1 -- 172.21.15.99:0/613047641 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000b870 con 0x7fa8f4161ac0 2024-03-29T09:09:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.901+0000 7fa8f1600700 1 -- 172.21.15.99:0/613047641 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e000b9d0 con 0x7fa8f4161ac0 2024-03-29T09:09:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.901+0000 7fa8f1600700 1 -- 172.21.15.99:0/613047641 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000bd60 con 0x7fa8f4161ac0 2024-03-29T09:09:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.901+0000 7f22b9c00700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f229406acf0 0x7f229406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.901+0000 7fa839c00700 1 -- start start 2024-03-29T09:09:30.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.900+0000 7f80eea00700 1 --2- 172.21.15.99:0/3479663845 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7fa8f3e00700 1 -- 172.21.15.99:0/613047641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 msgr2=0x7fa8f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/613047641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f406c920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406e500 tx=0x7fa8e0009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7f80eea00700 1 -- 172.21.15.99:0/3479663845 >> v1:172.21.15.133:6789/0 conn(0x7f80f00b0b50 legacy=0x7f80f00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7fa8f3e00700 1 -- 172.21.15.99:0/613047641 shutdown_connections 2024-03-29T09:09:30.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/613047641 >> v2:172.21.15.99:3300/0 conn(0x7fa8f40b21e0 0x7fa8f40b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/613047641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f406c920 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7fa8f3e00700 1 -- 172.21.15.99:0/613047641 >> 172.21.15.99:0/613047641 conn(0x7fa8f4066960 msgr2=0x7fa8f4066d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:30.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7fa8f3e00700 1 -- 172.21.15.99:0/613047641 shutdown_connections 2024-03-29T09:09:30.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7f22b9c00700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f229406acf0 0x7f229406d1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009440 tx=0x7f22ac01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7f80eea00700 1 -- 172.21.15.99:0/3479663845 >> v1:172.21.15.99:6789/0 conn(0x7f80f00b3e00 legacy=0x7f80f00b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.903+0000 7fa8f3e00700 1 -- 172.21.15.99:0/613047641 wait complete. 2024-03-29T09:09:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7fa8f3e00700 1 Processor -- start 2024-03-29T09:09:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.903+0000 7fa839c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa834144660 0x7fa834162750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.903+0000 7fa839c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8341652c0 0x7fa834165740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.903+0000 7fa839c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa83406d4a0 0x7fa83406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.903+0000 7fa839c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83406de30 con 0x7fa834144660 2024-03-29T09:09:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.903+0000 7fa839c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa834161cd0 con 0x7fa83406d4a0 2024-03-29T09:09:30.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.903+0000 7fa839c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa834162c90 con 0x7fa8341652c0 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.904+0000 7fa833400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa83406d4a0 0x7fa83406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.904+0000 7fa833400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa83406d4a0 0x7fa83406d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55434/0 (socket says 172.21.15.99:55434) 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.904+0000 7fa833400700 1 -- 172.21.15.99:0/2976285332 learned_addr learned my addr 172.21.15.99:0/2976285332 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.905+0000 7fa832000700 1 --2- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8341652c0 0x7fa834165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.905+0000 7fa833400700 1 -- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8341652c0 msgr2=0x7fa834165740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.905+0000 7fa833400700 1 --2- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8341652c0 0x7fa834165740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.905+0000 7fa833400700 1 -- 172.21.15.99:0/2976285332 >> v2:172.21.15.99:3300/0 conn(0x7fa834144660 msgr2=0x7fa834162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.905+0000 7fa833400700 1 --2- 172.21.15.99:0/2976285332 >> v2:172.21.15.99:3300/0 conn(0x7fa834144660 0x7fa834162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.905+0000 7fa833400700 1 -- 172.21.15.99:0/2976285332 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834165d80 con 0x7fa83406d4a0 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.905+0000 7fa832000700 1 --2- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8341652c0 0x7fa834165740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:30.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7fa833400700 1 --2- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83406d4a0 0x7fa83406d8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8340702e0 tx=0x7fa824009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05f1e2bc7f250e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.902+0000 7f80eea00700 1 -- 172.21.15.99:0/3479663845 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f015beb0 con 0x7f80f015ae70 2024-03-29T09:09:30.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.904+0000 7f80eea00700 1 --2- 172.21.15.99:0/3479663845 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f015b250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009dc0 tx=0x7f80e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b11251c78f3a356 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7fa831600700 1 -- 172.21.15.99:0/2976285332 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400b870 con 0x7fa83406d4a0 2024-03-29T09:09:30.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7fa8f3e00700 1 -- start start 2024-03-29T09:09:30.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7fa831600700 1 -- 172.21.15.99:0/2976285332 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82400b9d0 con 0x7fa83406d4a0 2024-03-29T09:09:30.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7fa831600700 1 -- 172.21.15.99:0/2976285332 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400bd60 con 0x7fa83406d4a0 2024-03-29T09:09:30.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7f80ed600700 1 -- 172.21.15.99:0/3479663845 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e800e070 con 0x7f80f015ae70 2024-03-29T09:09:30.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7f80ed600700 1 -- 172.21.15.99:0/3479663845 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e800bbd0 con 0x7f80f015ae70 2024-03-29T09:09:30.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.906+0000 7f80ed600700 1 -- 172.21.15.99:0/3479663845 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8012430 con 0x7f80f015ae70 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.907+0000 7fa839c00700 1 -- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83406d4a0 msgr2=0x7fa83406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 --2- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83406d4a0 0x7fa83406d8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8340702e0 tx=0x7fa824009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 -- 172.21.15.99:0/2976285332 shutdown_connections 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 --2- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83406d4a0 0x7fa83406d8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 --2- 172.21.15.99:0/2976285332 >> v2:172.21.15.99:3300/0 conn(0x7fa834144660 0x7fa834162750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 --2- 172.21.15.99:0/2976285332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8341652c0 0x7fa834165740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 -- 172.21.15.99:0/2976285332 >> 172.21.15.99:0/2976285332 conn(0x7fa8340b9210 msgr2=0x7fa8340b9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 -- 172.21.15.99:0/2976285332 shutdown_connections 2024-03-29T09:09:30.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7fa839c00700 1 -- 172.21.15.99:0/2976285332 wait complete. 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7f80f5e00700 1 -- 172.21.15.99:0/3479663845 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 msgr2=0x7f80f015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.908+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3479663845 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f015b250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009dc0 tx=0x7f80e800b330 comp rx=0 tx=0).stop 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7f80f5e00700 1 -- 172.21.15.99:0/3479663845 shutdown_connections 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3479663845 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f015b250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7f80f5e00700 1 -- 172.21.15.99:0/3479663845 >> 172.21.15.99:0/3479663845 conn(0x7f80f0066540 msgr2=0x7f80f0066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f406ce60 0x7fa8f41faa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8f40b21e0 0x7fa8f4071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f41f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7f80f5e00700 1 -- 172.21.15.99:0/3479663845 shutdown_connections 2024-03-29T09:09:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f41fa010 con 0x7fa8f406ce60 2024-03-29T09:09:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f41f74e0 con 0x7fa8f4161ac0 2024-03-29T09:09:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f41f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7f22b1600700 1 -- 172.21.15.99:0/3885508810 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f22b41ffe40 con 0x7f22b40b3a00 2024-03-29T09:09:30.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f41f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55444/0 (socket says 172.21.15.99:55444) 2024-03-29T09:09:30.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7fa8f3400700 1 -- 172.21.15.99:0/2119677003 learned_addr learned my addr 172.21.15.99:0/2119677003 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.910+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f41f77c0 con 0x7fa8f40b21e0 2024-03-29T09:09:30.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.909+0000 7f80f5e00700 1 -- 172.21.15.99:0/3479663845 wait complete. 2024-03-29T09:09:30.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.911+0000 7fa8f2000700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8f40b21e0 0x7fa8f4071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.911+0000 7fa8f3400700 1 -- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8f40b21e0 msgr2=0x7fa8f4071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.911+0000 7fa8f3400700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8f40b21e0 0x7fa8f4071790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.911+0000 7fa8f3400700 1 -- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f406ce60 msgr2=0x7fa8f41faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.911+0000 7fa8f3400700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f406ce60 0x7fa8f41faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.911+0000 7fa8f3400700 1 -- 172.21.15.99:0/2119677003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0009550 con 0x7fa8f4161ac0 2024-03-29T09:09:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.911+0000 7fa8f3400700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f41f6a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400c410 tx=0x7fa8e400c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7fa8eaa00700 1 -- 172.21.15.99:0/2119677003 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40054e0 con 0x7fa8f4161ac0 2024-03-29T09:09:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7fa8eaa00700 1 -- 172.21.15.99:0/2119677003 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e40036a0 con 0x7fa8f4161ac0 2024-03-29T09:09:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7fa8eaa00700 1 -- 172.21.15.99:0/2119677003 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4003a10 con 0x7fa8f4161ac0 2024-03-29T09:09:30.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7fa839c00700 1 Processor -- start 2024-03-29T09:09:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7f80f5e00700 1 Processor -- start 2024-03-29T09:09:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7f80f5e00700 1 -- start start 2024-03-29T09:09:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f41fa260 con 0x7fa8f4161ac0 2024-03-29T09:09:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.912+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f4154010 con 0x7fa8f4161ac0 2024-03-29T09:09:30.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.913+0000 7fa839c00700 1 -- start start 2024-03-29T09:09:30.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.913+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f41541a0 con 0x7fa8f4161ac0 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.914+0000 7f80f5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f00b0b50 0x7f80f0070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.914+0000 7f80f5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f00b3e00 0x7f80f014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.914+0000 7f80f5e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f0150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.914+0000 7f80eea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f00b0b50 0x7f80f0070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.914+0000 7f80eea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f00b0b50 0x7f80f0070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55448/0 (socket says 172.21.15.99:55448) 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.914+0000 7f80eea00700 1 -- 172.21.15.99:0/3670504553 learned_addr learned my addr 172.21.15.99:0/3670504553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.914+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f014fe80 con 0x7f80f00b3e00 2024-03-29T09:09:30.917 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f014b290 con 0x7f80f00b0b50 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80ef400700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f0150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0070d70 con 0x7f80f015ae70 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80eea00700 1 -- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 msgr2=0x7f80f0150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80eea00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f0150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80eea00700 1 -- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f00b3e00 msgr2=0x7f80f014e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80eea00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f00b3e00 0x7f80f014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.915+0000 7f80eea00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e8009a70 con 0x7f80f00b0b50 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.916+0000 7f80ef400700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f0150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.916+0000 7f80eea00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f00b0b50 0x7f80f0070540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80e8014040 tx=0x7f80e8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.916+0000 7fa8eaa00700 1 -- 172.21.15.99:0/2119677003 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa8e4003b70 con 0x7fa8f4161ac0 2024-03-29T09:09:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.917+0000 7f80e7400700 1 -- 172.21.15.99:0/3670504553 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80098f0 con 0x7f80f00b0b50 2024-03-29T09:09:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.917+0000 7f80e7400700 1 -- 172.21.15.99:0/3670504553 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e8020070 con 0x7f80f00b0b50 2024-03-29T09:09:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.917+0000 7f80e7400700 1 -- 172.21.15.99:0/3670504553 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004280 con 0x7f80f00b0b50 2024-03-29T09:09:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.918+0000 7fa8eaa00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8d406ac90 0x7fa8d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.918+0000 7fa8eaa00700 1 -- 172.21.15.99:0/2119677003 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8e4088940 con 0x7fa8f4161ac0 2024-03-29T09:09:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.918+0000 7fa839c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83406d4a0 0x7fa83415d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.918+0000 7fa839c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa834144660 0x7fa834147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7fa8f2a00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8d406ac90 0x7fa8d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7fa832a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83406d4a0 0x7fa83415d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7fa832a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83406d4a0 0x7fa83415d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:33090/0 (socket says 172.21.15.99:33090) 2024-03-29T09:09:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7fa839c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8341652c0 0x7fa83415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7fa839c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83415c400 con 0x7fa834144660 2024-03-29T09:09:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7fa839c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83415e020 con 0x7fa8341652c0 2024-03-29T09:09:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7fa839c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83415e300 con 0x7fa83406d4a0 2024-03-29T09:09:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f0071050 con 0x7f80f00b0b50 2024-03-29T09:09:30.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.919+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f0150060 con 0x7f80f00b0b50 2024-03-29T09:09:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa8f2a00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8d406ac90 0x7fa8d406d150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec008e00 tx=0x7fa8ec008330 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8341652c0 0x7fa83415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8341652c0 0x7fa83415f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55458/0 (socket says 172.21.15.99:55458) 2024-03-29T09:09:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 -- 172.21.15.99:0/2020726674 learned_addr learned my addr 172.21.15.99:0/2020726674 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 -- 172.21.15.99:0/2020726674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83406d4a0 msgr2=0x7fa83415d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83406d4a0 0x7fa83415d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 -- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa834144660 msgr2=0x7fa834147950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-29T09:09:30.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa834144660 0x7fa834147950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 -- 172.21.15.99:0/2020726674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009550 con 0x7fa8341652c0 2024-03-29T09:09:30.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.920+0000 7fa833400700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8341652c0 0x7fa83415f0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa834161e10 tx=0x7fa824004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.921+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f0071570 con 0x7f80f00b0b50 2024-03-29T09:09:30.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.922+0000 7fa8eaa00700 1 -- 172.21.15.99:0/2119677003 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa8e40538d0 con 0x7fa8f4161ac0 2024-03-29T09:09:30.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.922+0000 7fa82aa00700 1 -- 172.21.15.99:0/2020726674 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824004440 con 0x7fa8341652c0 2024-03-29T09:09:30.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.922+0000 7fa82aa00700 1 -- 172.21.15.99:0/2020726674 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8240036a0 con 0x7fa8341652c0 2024-03-29T09:09:30.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.922+0000 7fa82aa00700 1 -- 172.21.15.99:0/2020726674 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824003a50 con 0x7fa8341652c0 2024-03-29T09:09:30.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.923+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83415e5e0 con 0x7fa8341652c0 2024-03-29T09:09:30.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.923+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83415eba0 con 0x7fa8341652c0 2024-03-29T09:09:30.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.924+0000 7f80e7400700 1 -- 172.21.15.99:0/3670504553 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f80e80043e0 con 0x7f80f00b0b50 2024-03-29T09:09:30.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.928+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83415edc0 con 0x7fa8341652c0 2024-03-29T09:09:30.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.930+0000 7fa82aa00700 1 -- 172.21.15.99:0/2020726674 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa824003bb0 con 0x7fa8341652c0 2024-03-29T09:09:30.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.931+0000 7fa82aa00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81406ac90 0x7fa81406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.931+0000 7fa82aa00700 1 -- 172.21.15.99:0/2020726674 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa824021030 con 0x7fa8341652c0 2024-03-29T09:09:30.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.932+0000 7fa832a00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81406ac90 0x7fa81406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.933+0000 7fa832a00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81406ac90 0x7fa81406d150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8340d73c0 tx=0x7fa82c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.932+0000 7f80e7400700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f80d006ad60 0x7f80d006d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.933+0000 7f80e7400700 1 -- 172.21.15.99:0/3670504553 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f80e8022070 con 0x7f80f00b0b50 2024-03-29T09:09:30.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.933+0000 7f80ee000700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f80d006ad60 0x7f80d006d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.933+0000 7f80e7400700 1 -- 172.21.15.99:0/3670504553 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f80e80b8580 con 0x7f80f00b0b50 2024-03-29T09:09:30.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.935+0000 7fa82aa00700 1 -- 172.21.15.99:0/2020726674 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa824055ba0 con 0x7fa8341652c0 2024-03-29T09:09:30.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.938+0000 7f80ee000700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f80d006ad60 0x7f80d006d220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80dc000f60 tx=0x7f80dc007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.970+0000 7f0d75200700 1 Processor -- start 2024-03-29T09:09:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.971+0000 7f0d75200700 1 -- start start 2024-03-29T09:09:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.973+0000 7f0d75200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d7014aff0 0x7f0d70070390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.973+0000 7f0d75200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d70071a50 0x7f0d700cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.973+0000 7f0d75200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700726e0 con 0x7f0d7006d5e0 2024-03-29T09:09:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.973+0000 7f0d75200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d70070a90 con 0x7f0d7014aff0 2024-03-29T09:09:30.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.973+0000 7f0d75200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70070d70 con 0x7f0d70071a50 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.975+0000 7f0d6e800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0d7006d5e0 0x7f0d700b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49710/0 (socket says 172.21.15.99:49710) 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.975+0000 7f0d6e800700 1 -- 172.21.15.99:0/2932506367 learned_addr learned my addr 172.21.15.99:0/2932506367 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.975+0000 7f0d6d400700 1 --2- 172.21.15.99:0/2932506367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d70071a50 0x7f0d700cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.976+0000 7f0d6d400700 1 -- 172.21.15.99:0/2932506367 >> v2:172.21.15.133:3300/0 conn(0x7f0d7014aff0 msgr2=0x7f0d70070390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.976+0000 7f0d6d400700 1 --2- 172.21.15.99:0/2932506367 >> v2:172.21.15.133:3300/0 conn(0x7f0d7014aff0 0x7f0d70070390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.976+0000 7f0d6d400700 1 -- 172.21.15.99:0/2932506367 >> v1:172.21.15.99:6789/0 conn(0x7f0d7006d5e0 legacy=0x7f0d700b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.977+0000 7f0d6d400700 1 -- 172.21.15.99:0/2932506367 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70071050 con 0x7f0d70071a50 2024-03-29T09:09:30.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.977+0000 7f0d6d400700 1 --2- 172.21.15.99:0/2932506367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d70071a50 0x7f0d700cc300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009840 tx=0x7f0d60009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0efa223e5405f27 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.978+0000 7f0d6ca00700 1 -- 172.21.15.99:0/2932506367 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60011830 con 0x7f0d70071a50 2024-03-29T09:09:30.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.978+0000 7f0d6ca00700 1 -- 172.21.15.99:0/2932506367 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d60011990 con 0x7f0d70071a50 2024-03-29T09:09:30.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.978+0000 7f0d6ca00700 1 -- 172.21.15.99:0/2932506367 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60011830 con 0x7f0d70071a50 2024-03-29T09:09:30.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.979+0000 7f0d75200700 1 -- 172.21.15.99:0/2932506367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d70071a50 msgr2=0x7f0d700cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.979+0000 7f0d75200700 1 --2- 172.21.15.99:0/2932506367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d70071a50 0x7f0d700cc300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009840 tx=0x7f0d60009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:30.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.980+0000 7f0d75200700 1 -- 172.21.15.99:0/2932506367 shutdown_connections 2024-03-29T09:09:30.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.980+0000 7f0d75200700 1 --2- 172.21.15.99:0/2932506367 >> v2:172.21.15.133:3300/0 conn(0x7f0d7014aff0 0x7f0d70070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.980+0000 7f0d75200700 1 --2- 172.21.15.99:0/2932506367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d70071a50 0x7f0d700cc300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.980+0000 7f0d75200700 1 -- 172.21.15.99:0/2932506367 >> 172.21.15.99:0/2932506367 conn(0x7f0d700b0ae0 msgr2=0x7f0d700b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:30.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.981+0000 7f0d75200700 1 -- 172.21.15.99:0/2932506367 shutdown_connections 2024-03-29T09:09:30.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.981+0000 7f0d75200700 1 -- 172.21.15.99:0/2932506367 wait complete. 2024-03-29T09:09:30.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.982+0000 7f7e70400700 1 Processor -- start 2024-03-29T09:09:30.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.983+0000 7f0d75200700 1 Processor -- start 2024-03-29T09:09:30.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.984+0000 7f0d75200700 1 -- start start 2024-03-29T09:09:30.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.982+0000 7f7e70400700 1 -- start start 2024-03-29T09:09:30.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.984+0000 7f7e70400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e6815a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.984+0000 7f7e70400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e68071010 con 0x7f7e68072330 2024-03-29T09:09:30.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.984+0000 7f7e70400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e680706e0 con 0x7f7e68157070 2024-03-29T09:09:30.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.984+0000 7f7e70400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6815a970 con 0x7f7e680ce3c0 2024-03-29T09:09:30.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.985+0000 7f7e6e400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e6815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.985+0000 7f7e6da00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7e68157070 0x7f7e68071c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:52106/0 (socket says 172.21.15.99:52106) 2024-03-29T09:09:30.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.985+0000 7f7e6da00700 1 -- 172.21.15.99:0/1888187777 learned_addr learned my addr 172.21.15.99:0/1888187777 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:30.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.985+0000 7f7e6d000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7e68072330 0x7f7e680cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49724/0 (socket says 172.21.15.99:49724) 2024-03-29T09:09:30.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.986+0000 7f7e67e00700 1 -- 172.21.15.99:0/1888187777 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251943220 0 0) 0x7f7e680706e0 con 0x7f7e68157070 2024-03-29T09:09:30.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.986+0000 7f7e67e00700 1 -- 172.21.15.99:0/1888187777 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e500035b0 con 0x7f7e68157070 2024-03-29T09:09:30.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.986+0000 7f7e6e400700 1 -- 172.21.15.99:0/1888187777 >> v1:172.21.15.133:6789/0 conn(0x7f7e68157070 legacy=0x7f7e68071c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f7e6e400700 1 -- 172.21.15.99:0/1888187777 >> v1:172.21.15.99:6789/0 conn(0x7f7e68072330 legacy=0x7f7e680cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.986+0000 7f0d75200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d7006d5e0 0x7f0d70136200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f0d75200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70071a50 0x7f0d701641b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f7e67e00700 1 -- 172.21.15.99:0/1888187777 <== mon.1 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716930417 0 0) 0x7f7e500035b0 con 0x7f7e68157070 2024-03-29T09:09:30.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f7e6e400700 1 -- 172.21.15.99:0/1888187777 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e680709c0 con 0x7f7e680ce3c0 2024-03-29T09:09:30.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f0d6de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d7006d5e0 0x7f0d70136200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f0d75200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d7014aff0 0x7f0d70162250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f7e6e400700 1 --2- 172.21.15.99:0/1888187777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e6815a390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0097f0 tx=0x7f7e5c009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a77adcf648bd678 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f7e67e00700 1 -- 172.21.15.99:0/1888187777 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54001670 con 0x7f7e680ce3c0 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f7e67e00700 1 -- 172.21.15.99:0/1888187777 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c004030 con 0x7f7e680ce3c0 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f7e67e00700 1 -- 172.21.15.99:0/1888187777 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e54001670 con 0x7f7e680ce3c0 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f0d6d400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70071a50 0x7f0d701641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f0d6de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d7006d5e0 0x7f0d70136200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53410/0 (socket says 172.21.15.99:53410) 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.987+0000 7f0d6d400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70071a50 0x7f0d701641b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55462/0 (socket says 172.21.15.99:55462) 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f0d6de00700 1 -- 172.21.15.99:0/1983045198 learned_addr learned my addr 172.21.15.99:0/1983045198 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d701349e0 con 0x7f0d7006d5e0 2024-03-29T09:09:30.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70135d70 con 0x7f0d70071a50 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70134700 con 0x7f0d7014aff0 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.988+0000 7f0d6e800700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d7014aff0 0x7f0d70162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f7e70400700 1 -- 172.21.15.99:0/1888187777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 msgr2=0x7f7e6815a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f7e70400700 1 --2- 172.21.15.99:0/1888187777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e6815a390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0097f0 tx=0x7f7e5c009b00 comp rx=0 tx=0).stop 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f7e70400700 1 -- 172.21.15.99:0/1888187777 shutdown_connections 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f7e70400700 1 --2- 172.21.15.99:0/1888187777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e6815a390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f7e70400700 1 -- 172.21.15.99:0/1888187777 >> 172.21.15.99:0/1888187777 conn(0x7f7e680b0bc0 msgr2=0x7f7e680b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f7e70400700 1 -- 172.21.15.99:0/1888187777 shutdown_connections 2024-03-29T09:09:30.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f7e70400700 1 -- 172.21.15.99:0/1888187777 wait complete. 2024-03-29T09:09:30.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f0d6e800700 1 -- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70071a50 msgr2=0x7f0d701641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f0d6e800700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70071a50 0x7f0d701641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f0d6e800700 1 -- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d7006d5e0 msgr2=0x7f0d70136200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f0d6e800700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d7006d5e0 0x7f0d70136200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f0d6e800700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600094a0 con 0x7f0d7014aff0 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f0d6de00700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d7006d5e0 0x7f0d70136200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.989+0000 7f0d6e800700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d7014aff0 0x7f0d70162250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c460 tx=0x7f0d5c00c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.990+0000 7f0d66000700 1 -- 172.21.15.99:0/1983045198 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c005d70 con 0x7f0d7014aff0 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.990+0000 7f0d66000700 1 -- 172.21.15.99:0/1983045198 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c00ce30 con 0x7f0d7014aff0 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.990+0000 7f0d66000700 1 -- 172.21.15.99:0/1983045198 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0072d0 con 0x7f0d7014aff0 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.990+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d70135720 con 0x7f0d7014aff0 2024-03-29T09:09:30.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.990+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d701646f0 con 0x7f0d7014aff0 2024-03-29T09:09:30.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.992+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d480052f0 con 0x7f0d7014aff0 2024-03-29T09:09:30.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.992+0000 7f7e70400700 1 Processor -- start 2024-03-29T09:09:30.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.993+0000 7f7e70400700 1 -- start start 2024-03-29T09:09:30.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.994+0000 7f7e70400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e68072330 0x7f7e68150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.995+0000 7f7e70400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e68151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.995+0000 7f7e70400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e68157070 0x7f7e68143190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.993+0000 7f0d66000700 1 -- 172.21.15.99:0/1983045198 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0d5c007430 con 0x7f0d7014aff0 2024-03-29T09:09:30.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.995+0000 7f0d66000700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0d5006ace0 0x7f0d5006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.995+0000 7f7e70400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68151ce0 con 0x7f7e68072330 2024-03-29T09:09:30.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.995+0000 7f7e70400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681524f0 con 0x7f7e68157070 2024-03-29T09:09:30.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7997e00700 1 Processor -- start 2024-03-29T09:09:30.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f0d6de00700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0d5006ace0 0x7f0d5006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7997e00700 1 -- start start 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e70400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e68141060 con 0x7f7e680ce3c0 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6d000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e68151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6d000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e68151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:33124/0 (socket says 172.21.15.99:33124) 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6d000700 1 -- 172.21.15.99:0/104701279 learned_addr learned my addr 172.21.15.99:0/104701279 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6e400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e68157070 0x7f7e68143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6da00700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e68072330 0x7f7e68150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6e400700 1 -- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 msgr2=0x7f7e68151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6e400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e68151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.996+0000 7f7e6e400700 1 -- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e68072330 msgr2=0x7f7e68150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.997+0000 7f0d66000700 1 -- 172.21.15.99:0/1983045198 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0d5c0885f0 con 0x7f0d7014aff0 2024-03-29T09:09:30.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.997+0000 7f0d6de00700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0d5006ace0 0x7f0d5006d1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d700b1d10 tx=0x7f0d6800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.997+0000 7f7e6e400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e68072330 0x7f7e68150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7e6e400700 1 -- 172.21.15.99:0/104701279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0094a0 con 0x7f7e68157070 2024-03-29T09:09:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7e6da00700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e68072330 0x7f7e68150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7e6e400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e68157070 0x7f7e68143190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c014000 tx=0x7f7e5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7997e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7998157890 0x7f79980cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7997e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79980cf560 0x7f79980b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7997e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79980b2180 0x7f7998160c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7997e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799815d670 con 0x7f7998157890 2024-03-29T09:09:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7997e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79980cf9f0 con 0x7f79980cf560 2024-03-29T09:09:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.998+0000 7f7997e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799815d860 con 0x7f79980b2180 2024-03-29T09:09:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7e65600700 1 -- 172.21.15.99:0/104701279 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c010830 con 0x7f7e68157070 2024-03-29T09:09:30.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7e65600700 1 -- 172.21.15.99:0/104701279 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c010990 con 0x7f7e68157070 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7996000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79980cf560 0x7f79980b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7996000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79980cf560 0x7f79980b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55488/0 (socket says 172.21.15.99:55488) 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7996000700 1 -- 172.21.15.99:0/419389658 learned_addr learned my addr 172.21.15.99:0/419389658 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7996000700 1 -- 172.21.15.99:0/419389658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79980b2180 msgr2=0x7f7998160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7996000700 1 --2- 172.21.15.99:0/419389658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79980b2180 0x7f7998160c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.000+0000 7f7996000700 1 -- 172.21.15.99:0/419389658 >> v2:172.21.15.99:3300/0 conn(0x7f7998157890 msgr2=0x7f79980cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.000+0000 7f7996000700 1 --2- 172.21.15.99:0/419389658 >> v2:172.21.15.99:3300/0 conn(0x7f7998157890 0x7f79980cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.000+0000 7f7996000700 1 -- 172.21.15.99:0/419389658 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799815db40 con 0x7f79980cf560 2024-03-29T09:09:31.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7e65600700 1 -- 172.21.15.99:0/104701279 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c010830 con 0x7f7e68157070 2024-03-29T09:09:31.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e68151f30 con 0x7f7e68157070 2024-03-29T09:09:31.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:30.999+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e68141300 con 0x7f7e68157070 2024-03-29T09:09:31.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.001+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e440052f0 con 0x7f7e68157070 2024-03-29T09:09:31.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.001+0000 7f7996000700 1 --2- 172.21.15.99:0/419389658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79980b1c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79880098f0 tx=0x7f7988009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4ec56a7bce57f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:31.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.002+0000 7f0d66000700 1 -- 172.21.15.99:0/1983045198 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0d5c053700 con 0x7f0d7014aff0 2024-03-29T09:09:31.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.003+0000 7f7e65600700 1 -- 172.21.15.99:0/104701279 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e5c010b00 con 0x7f7e68157070 2024-03-29T09:09:31.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.005+0000 7f7e65600700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e4806acf0 0x7f7e4806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.005+0000 7f7e65600700 1 -- 172.21.15.99:0/104701279 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7e5c08b4a0 con 0x7f7e68157070 2024-03-29T09:09:31.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.006+0000 7f7e6da00700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e4806acf0 0x7f7e4806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.006+0000 7f7e65600700 1 -- 172.21.15.99:0/104701279 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e5c056440 con 0x7f7e68157070 2024-03-29T09:09:31.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.007+0000 7f7e6da00700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e4806acf0 0x7f7e4806d1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e680cdaf0 tx=0x7f7e54008b30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:31.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.007+0000 7f7995600700 1 -- 172.21.15.99:0/419389658 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798800b860 con 0x7f79980cf560 2024-03-29T09:09:31.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.007+0000 7f7995600700 1 -- 172.21.15.99:0/419389658 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798800b9c0 con 0x7f79980cf560 2024-03-29T09:09:31.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.007+0000 7f7995600700 1 -- 172.21.15.99:0/419389658 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798800bd50 con 0x7f79980cf560 2024-03-29T09:09:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.011+0000 7f7997e00700 1 -- 172.21.15.99:0/419389658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 msgr2=0x7f79980b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.011+0000 7f7997e00700 1 --2- 172.21.15.99:0/419389658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79980b1c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79880098f0 tx=0x7f7988009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:31.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.013+0000 7f7997e00700 1 -- 172.21.15.99:0/419389658 shutdown_connections 2024-03-29T09:09:31.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.013+0000 7f7997e00700 1 --2- 172.21.15.99:0/419389658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79980b2180 0x7f7998160c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.013+0000 7f7997e00700 1 --2- 172.21.15.99:0/419389658 >> v2:172.21.15.99:3300/0 conn(0x7f7998157890 0x7f79980cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.013+0000 7f7997e00700 1 --2- 172.21.15.99:0/419389658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79980b1c40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.013+0000 7f7997e00700 1 -- 172.21.15.99:0/419389658 >> 172.21.15.99:0/419389658 conn(0x7f7998066540 msgr2=0x7f7998066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.013+0000 7f7997e00700 1 -- 172.21.15.99:0/419389658 shutdown_connections 2024-03-29T09:09:31.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.013+0000 7f7997e00700 1 -- 172.21.15.99:0/419389658 wait complete. 2024-03-29T09:09:31.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.016+0000 7f7997e00700 1 Processor -- start 2024-03-29T09:09:31.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.017+0000 7f7997e00700 1 -- start start 2024-03-29T09:09:31.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.018+0000 7f7997e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79980b2180 0x7f79981e8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.018+0000 7f7997e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79981e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.018+0000 7f7997e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79981538f0 0x7f7998141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.018+0000 7f7997e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79981e6f90 con 0x7f79980b2180 2024-03-29T09:09:31.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.018+0000 7f7997e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79981e7a40 con 0x7f79980cf560 2024-03-29T09:09:31.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.018+0000 7f7997e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79981e71a0 con 0x7f79981538f0 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79981e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79981e6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55498/0 (socket says 172.21.15.99:55498) 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 -- 172.21.15.99:0/315989669 learned_addr learned my addr 172.21.15.99:0/315989669 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 -- 172.21.15.99:0/315989669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79981538f0 msgr2=0x7f7998141100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79981538f0 0x7f7998141100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 -- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79980b2180 msgr2=0x7f79981e8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79980b2180 0x7f79981e8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.019+0000 7f7996000700 1 -- 172.21.15.99:0/315989669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7988009550 con 0x7f79980cf560 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.020+0000 7f7996000700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79981e6a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f79980ae7d0 tx=0x7f7988004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.020+0000 7f798ea00700 1 -- 172.21.15.99:0/315989669 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988004400 con 0x7f79980cf560 2024-03-29T09:09:31.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.020+0000 7f798ea00700 1 -- 172.21.15.99:0/315989669 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7988028070 con 0x7f79980cf560 2024-03-29T09:09:31.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.020+0000 7f798ea00700 1 -- 172.21.15.99:0/315989669 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988024a90 con 0x7f79980cf560 2024-03-29T09:09:31.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.020+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79981e7480 con 0x7f79980cf560 2024-03-29T09:09:31.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.020+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79981f3010 con 0x7f79980cf560 2024-03-29T09:09:31.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.023+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79700052f0 con 0x7f79980cf560 2024-03-29T09:09:31.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.025+0000 7f798ea00700 1 -- 172.21.15.99:0/315989669 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7988022070 con 0x7f79980cf560 2024-03-29T09:09:31.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.026+0000 7f798ea00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f797806ac90 0x7f797806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.027+0000 7f798ea00700 1 -- 172.21.15.99:0/315989669 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7988020060 con 0x7f79980cf560 2024-03-29T09:09:31.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.028+0000 7f7996a00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f797806ac90 0x7f797806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.029+0000 7f7996a00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f797806ac90 0x7f797806d150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f799000ab90 tx=0x7f7990009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:31.035 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.034+0000 7f798ea00700 1 -- 172.21.15.99:0/315989669 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f79880555f0 con 0x7f79980cf560 2024-03-29T09:09:31.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.309+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f04140d92d0 con 0x7f0414053790 2024-03-29T09:09:31.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.309+0000 7f040f400700 1 -- 172.21.15.99:0/3917646957 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f04140d92d0 con 0x7f0414053790 2024-03-29T09:09:31.312 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:31.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.314+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03fc06ac90 msgr2=0x7f03fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.314+0000 7f0422600700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03fc06ac90 0x7f03fc06d150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f04080014e0 tx=0x7f0408007a60 comp rx=0 tx=0).stop 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.314+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 msgr2=0x7f04140de150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.314+0000 7f0422600700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f04140de150 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0410000c00 tx=0x7f0410004750 comp rx=0 tx=0).stop 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.314+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 shutdown_connections 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.315+0000 7f0422600700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03fc06ac90 0x7f03fc06d150 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.315+0000 7f0422600700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f04140e32c0 0x7f04141849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.315+0000 7f0422600700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04140063d0 0x7f04140d9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.315+0000 7f0422600700 1 --2- 172.21.15.99:0/3917646957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0414053790 0x7f04140de150 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.315+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 >> 172.21.15.99:0/3917646957 conn(0x7f04140559c0 msgr2=0x7f04140dcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.317+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 shutdown_connections 2024-03-29T09:09:31.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.317+0000 7f0422600700 1 -- 172.21.15.99:0/3917646957 wait complete. 2024-03-29T09:09:31.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-03-29T09:09:31.347 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.369+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f80f0071570 con 0x7f80f00b0b50 2024-03-29T09:09:31.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.369+0000 7f80e7400700 1 -- 172.21.15.99:0/3670504553 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f80f0071570 con 0x7f80f00b0b50 2024-03-29T09:09:31.369 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:31.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.372+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f80d006ad60 msgr2=0x7f80d006d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f80d006ad60 0x7f80d006d220 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f80dc000f60 tx=0x7f80dc007a40 comp rx=0 tx=0).stop 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f00b0b50 msgr2=0x7f80f0070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f00b0b50 0x7f80f0070540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f80e8014040 tx=0x7f80e8004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 shutdown_connections 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f80d006ad60 0x7f80d006d220 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80f015ae70 0x7f80f0150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80f00b0b50 0x7f80f0070540 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 --2- 172.21.15.99:0/3670504553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80f00b3e00 0x7f80f014e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.373+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 >> 172.21.15.99:0/3670504553 conn(0x7f80f0066540 msgr2=0x7f80f00b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.374+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 shutdown_connections 2024-03-29T09:09:31.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.374+0000 7f80f5e00700 1 -- 172.21.15.99:0/3670504553 wait complete. 2024-03-29T09:09:31.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.382+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f22b41ffe40 con 0x7f22b40b3a00 2024-03-29T09:09:31.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.384+0000 7f22b1600700 1 -- 172.21.15.99:0/3885508810 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f22b41ffe40 con 0x7f22b40b3a00 2024-03-29T09:09:31.384 INFO:teuthology.orchestra.run.smithi099.stdout:55834574859 2024-03-29T09:09:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.383+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa8f41fa780 con 0x7fa8f4161ac0 2024-03-29T09:09:31.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.384+0000 7fa8eaa00700 1 -- 172.21.15.99:0/2119677003 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8f41fa780 con 0x7fa8f4161ac0 2024-03-29T09:09:31.384 INFO:teuthology.orchestra.run.smithi099.stdout:55834574860 2024-03-29T09:09:31.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.388+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f229406acf0 msgr2=0x7f229406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.388+0000 7f22bc600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f229406acf0 0x7f229406d1b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009440 tx=0x7f22ac01c040 comp rx=0 tx=0).stop 2024-03-29T09:09:31.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.388+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b40b3a00 msgr2=0x7f22b4202ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.388+0000 7f22bc600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b40b3a00 0x7f22b4202ee0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c430 tx=0x7f22a800c740 comp rx=0 tx=0).stop 2024-03-29T09:09:31.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8d406ac90 msgr2=0x7fa8d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8d406ac90 0x7fa8d406d150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec008e00 tx=0x7fa8ec008330 comp rx=0 tx=0).stop 2024-03-29T09:09:31.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 shutdown_connections 2024-03-29T09:09:31.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f229406acf0 0x7f229406d1b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22b40afb20 0x7f22b4200120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22b4070ab0 0x7f22b409c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 --2- 172.21.15.99:0/3885508810 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f22b40b3a00 0x7f22b4202ee0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 >> 172.21.15.99:0/3885508810 conn(0x7f22b4066540 msgr2=0x7f22b40703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 msgr2=0x7fa8f41f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f41f6a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400c410 tx=0x7fa8e400c7d0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 shutdown_connections 2024-03-29T09:09:31.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7f22bc600700 1 -- 172.21.15.99:0/3885508810 wait complete. 2024-03-29T09:09:31.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 shutdown_connections 2024-03-29T09:09:31.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa8d406ac90 0x7fa8d406d150 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec008e00 tx=0x7fa8ec008330 comp rx=0 tx=0).stop 2024-03-29T09:09:31.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8f4161ac0 0x7fa8f41f6a40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa8f406ce60 0x7fa8f41faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 --2- 172.21.15.99:0/2119677003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa8f40b21e0 0x7fa8f4071790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.389+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 >> 172.21.15.99:0/2119677003 conn(0x7fa8f4066960 msgr2=0x7fa8f4066d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.390+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 shutdown_connections 2024-03-29T09:09:31.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.390+0000 7fa8f3e00700 1 -- 172.21.15.99:0/2119677003 wait complete. 2024-03-29T09:09:31.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.396+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa834067730 con 0x7fa8341652c0 2024-03-29T09:09:31.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.397+0000 7fa82aa00700 1 -- 172.21.15.99:0/2020726674 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa82402b090 con 0x7fa8341652c0 2024-03-29T09:09:31.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-03-29T09:09:31.398 INFO:teuthology.orchestra.run.smithi099.stdout:55834574858 2024-03-29T09:09:31.398 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81406ac90 msgr2=0x7fa81406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81406ac90 0x7fa81406d150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8340d73c0 tx=0x7fa82c00d040 comp rx=0 tx=0).stop 2024-03-29T09:09:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8341652c0 msgr2=0x7fa83415f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8341652c0 0x7fa83415f0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa834161e10 tx=0x7fa824004750 comp rx=0 tx=0).stop 2024-03-29T09:09:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 shutdown_connections 2024-03-29T09:09:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81406ac90 0x7fa81406d150 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8341652c0 0x7fa83415f0c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83406d4a0 0x7fa83415d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 --2- 172.21.15.99:0/2020726674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa834144660 0x7fa834147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 >> 172.21.15.99:0/2020726674 conn(0x7fa8340b9210 msgr2=0x7fa834143d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.403+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 shutdown_connections 2024-03-29T09:09:31.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.404+0000 7fa839c00700 1 -- 172.21.15.99:0/2020726674 wait complete. 2024-03-29T09:09:31.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-03-29T09:09:31.415 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-03-29T09:09:31.419 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-03-29T09:09:31.426 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.446+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f79700055a0 con 0x7f79980cf560 2024-03-29T09:09:31.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.446+0000 7f798ea00700 1 -- 172.21.15.99:0/315989669 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7988058c10 con 0x7f79980cf560 2024-03-29T09:09:31.445 INFO:teuthology.orchestra.run.smithi099.stdout:55834574860 2024-03-29T09:09:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f797806ac90 msgr2=0x7f797806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f797806ac90 0x7f797806d150 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f799000ab90 tx=0x7f7990009250 comp rx=0 tx=0).stop 2024-03-29T09:09:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 msgr2=0x7f79981e6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79981e6a50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f79980ae7d0 tx=0x7f7988004750 comp rx=0 tx=0).stop 2024-03-29T09:09:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 shutdown_connections 2024-03-29T09:09:31.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f797806ac90 0x7f797806d150 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79981538f0 0x7f7998141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79980b2180 0x7f79981e8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 --2- 172.21.15.99:0/315989669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79980cf560 0x7f79981e6a50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 >> 172.21.15.99:0/315989669 conn(0x7f7998066540 msgr2=0x7f7998066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 shutdown_connections 2024-03-29T09:09:31.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.449+0000 7f7997e00700 1 -- 172.21.15.99:0/315989669 wait complete. 2024-03-29T09:09:31.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.467+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0d48005870 con 0x7f0d7014aff0 2024-03-29T09:09:31.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.467+0000 7f0d66000700 1 -- 172.21.15.99:0/1983045198 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d5c001120 con 0x7f0d7014aff0 2024-03-29T09:09:31.467 INFO:teuthology.orchestra.run.smithi099.stdout:55834574861 2024-03-29T09:09:31.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.7 2024-03-29T09:09:31.468 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.471+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0d5006ace0 msgr2=0x7f0d5006d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.471+0000 7f0d75200700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0d5006ace0 0x7f0d5006d1a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0d700b1d10 tx=0x7f0d6800b040 comp rx=0 tx=0).stop 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.471+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d7014aff0 msgr2=0x7f0d70162250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.471+0000 7f0d75200700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d7014aff0 0x7f0d70162250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c460 tx=0x7f0d5c00c770 comp rx=0 tx=0).stop 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.471+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 shutdown_connections 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.471+0000 7f0d75200700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0d5006ace0 0x7f0d5006d1a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.471+0000 7f0d75200700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d7014aff0 0x7f0d70162250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.472+0000 7f0d75200700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0d7006d5e0 0x7f0d70136200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.472+0000 7f0d75200700 1 --2- 172.21.15.99:0/1983045198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d70071a50 0x7f0d701641b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.472+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 >> 172.21.15.99:0/1983045198 conn(0x7f0d700b0ae0 msgr2=0x7f0d700afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.472+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 shutdown_connections 2024-03-29T09:09:31.472 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.472+0000 7f0d75200700 1 -- 172.21.15.99:0/1983045198 wait complete. 2024-03-29T09:09:31.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.474+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7e44005870 con 0x7f7e68157070 2024-03-29T09:09:31.474 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.475+0000 7f7e65600700 1 -- 172.21.15.99:0/104701279 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7e5c059a60 con 0x7f7e68157070 2024-03-29T09:09:31.474 INFO:teuthology.orchestra.run.smithi099.stdout:55834574860 2024-03-29T09:09:31.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e4806acf0 msgr2=0x7f7e4806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e4806acf0 0x7f7e4806d1b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7e680cdaf0 tx=0x7f7e54008b30 comp rx=0 tx=0).stop 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e68157070 msgr2=0x7f7e68143190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e68157070 0x7f7e68143190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c014000 tx=0x7f7e5c004750 comp rx=0 tx=0).stop 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 shutdown_connections 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e4806acf0 0x7f7e4806d1b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e68157070 0x7f7e68143190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e68072330 0x7f7e68150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 --2- 172.21.15.99:0/104701279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e680ce3c0 0x7f7e68151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.477+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 >> 172.21.15.99:0/104701279 conn(0x7f7e680b0bc0 msgr2=0x7f7e68150720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.478+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 shutdown_connections 2024-03-29T09:09:31.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.478+0000 7f7e70400700 1 -- 172.21.15.99:0/104701279 wait complete. 2024-03-29T09:09:31.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574861 for osd.5 2024-03-29T09:09:31.496 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.6 2024-03-29T09:09:31.499 DEBUG:teuthology.parallel:result is None 2024-03-29T09:09:31.499 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T09:09:31.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:09:31.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.730+0000 7f9f07800700 1 Processor -- start 2024-03-29T09:09:31.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.731+0000 7f9f07800700 1 -- start start 2024-03-29T09:09:31.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f07800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f001608e0 0x7f9f000d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f07800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f00160cc0 0x7f9f000ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f07800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f000cddc0 con 0x7f9f000d3a70 2024-03-29T09:09:31.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f07800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f000ce0a0 con 0x7f9f001608e0 2024-03-29T09:09:31.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f07800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000ce380 con 0x7f9f00160cc0 2024-03-29T09:09:31.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f04e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f001608e0 0x7f9f000d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9effe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f00160cc0 0x7f9f000ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f04e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f001608e0 0x7f9f000d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55522/0 (socket says 172.21.15.99:55522) 2024-03-29T09:09:31.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.732+0000 7f9f04e00700 1 -- 172.21.15.99:0/1105992975 learned_addr learned my addr 172.21.15.99:0/1105992975 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:31.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9eff400700 1 -- 172.21.15.99:0/1105992975 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252016265 0 0) 0x7f9f000cddc0 con 0x7f9f000d3a70 2024-03-29T09:09:31.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9eff400700 1 -- 172.21.15.99:0/1105992975 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee80035b0 con 0x7f9f000d3a70 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9effe00700 1 -- 172.21.15.99:0/1105992975 >> v2:172.21.15.133:3300/0 conn(0x7f9f001608e0 msgr2=0x7f9f000d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9effe00700 1 --2- 172.21.15.99:0/1105992975 >> v2:172.21.15.133:3300/0 conn(0x7f9f001608e0 0x7f9f000d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9effe00700 1 -- 172.21.15.99:0/1105992975 >> v1:172.21.15.133:6789/0 conn(0x7f9f000d3a70 legacy=0x7f9f000d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9effe00700 1 -- 172.21.15.99:0/1105992975 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000ce660 con 0x7f9f00160cc0 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9f04e00700 1 --2- 172.21.15.99:0/1105992975 >> v2:172.21.15.133:3300/0 conn(0x7f9f001608e0 0x7f9f000d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9effe00700 1 --2- 172.21.15.99:0/1105992975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f00160cc0 0x7f9f000ceb80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009840 tx=0x7f9eec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928219c16ccbd842 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.733+0000 7f9eff400700 1 -- 172.21.15.99:0/1105992975 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011830 con 0x7f9f00160cc0 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9eff400700 1 -- 172.21.15.99:0/1105992975 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec011990 con 0x7f9f00160cc0 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9eff400700 1 -- 172.21.15.99:0/1105992975 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eec011d20 con 0x7f9f00160cc0 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 -- 172.21.15.99:0/1105992975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f00160cc0 msgr2=0x7f9f000ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 --2- 172.21.15.99:0/1105992975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f00160cc0 0x7f9f000ceb80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009840 tx=0x7f9eec009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 -- 172.21.15.99:0/1105992975 shutdown_connections 2024-03-29T09:09:31.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 --2- 172.21.15.99:0/1105992975 >> v2:172.21.15.133:3300/0 conn(0x7f9f001608e0 0x7f9f000d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 --2- 172.21.15.99:0/1105992975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f00160cc0 0x7f9f000ceb80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 -- 172.21.15.99:0/1105992975 >> 172.21.15.99:0/1105992975 conn(0x7f9f0009c490 msgr2=0x7f9f000d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 -- 172.21.15.99:0/1105992975 shutdown_connections 2024-03-29T09:09:31.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.734+0000 7f9f07800700 1 -- 172.21.15.99:0/1105992975 wait complete. 2024-03-29T09:09:31.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.736+0000 7f9f07800700 1 Processor -- start 2024-03-29T09:09:31.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.736+0000 7f9f07800700 1 -- start start 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f07800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f000d3a70 0x7f9f000cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f07800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f001608e0 0x7f9f000b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f07800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00160cc0 0x7f9f001e7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f07800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b2960 con 0x7f9f00160cc0 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f07800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b2b90 con 0x7f9f001608e0 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f07800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b2e70 con 0x7f9f000d3a70 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f05800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00160cc0 0x7f9f001e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f05800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00160cc0 0x7f9f001e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53478/0 (socket says 172.21.15.99:53478) 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.737+0000 7f9f05800700 1 -- 172.21.15.99:0/3622131455 learned_addr learned my addr 172.21.15.99:0/3622131455 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f04e00700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f000d3a70 0x7f9f000cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9effe00700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f001608e0 0x7f9f000b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f05800700 1 -- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f000d3a70 msgr2=0x7f9f000cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f05800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f000d3a70 0x7f9f000cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f05800700 1 -- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f001608e0 msgr2=0x7f9f000b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f05800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f001608e0 0x7f9f000b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f05800700 1 -- 172.21.15.99:0/3622131455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009550 con 0x7f9f00160cc0 2024-03-29T09:09:31.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f05800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00160cc0 0x7f9f001e7090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800cc20 tx=0x7f9ef800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:31.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9efcc00700 1 -- 172.21.15.99:0/3622131455 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8003c00 con 0x7f9f00160cc0 2024-03-29T09:09:31.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9efcc00700 1 -- 172.21.15.99:0/3622131455 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8003d60 con 0x7f9f00160cc0 2024-03-29T09:09:31.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec0094a0 con 0x7f9f00160cc0 2024-03-29T09:09:31.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.738+0000 7f9efcc00700 1 -- 172.21.15.99:0/3622131455 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800aae0 con 0x7f9f00160cc0 2024-03-29T09:09:31.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.739+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f00154bd0 con 0x7f9f00160cc0 2024-03-29T09:09:31.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.739+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f00154df0 con 0x7f9f00160cc0 2024-03-29T09:09:31.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.740+0000 7f9efcc00700 1 -- 172.21.15.99:0/3622131455 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9ef800ac40 con 0x7f9f00160cc0 2024-03-29T09:09:31.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.741+0000 7f9efcc00700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee006ac90 0x7f9ee006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:31.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.741+0000 7f9efcc00700 1 -- 172.21.15.99:0/3622131455 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ef8089670 con 0x7f9f00160cc0 2024-03-29T09:09:31.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.742+0000 7f9f04e00700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee006ac90 0x7f9ee006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:31.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.742+0000 7f9f04e00700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee006ac90 0x7f9ee006d150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006d30 tx=0x7f9ef40092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:31.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.746+0000 7f9efcc00700 1 -- 172.21.15.99:0/3622131455 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9ef8054550 con 0x7f9f00160cc0 2024-03-29T09:09:31.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.944+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f000b22a0 con 0x7f9ee006ac90 2024-03-29T09:09:31.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.954+0000 7f9efcc00700 1 -- 172.21.15.99:0/3622131455 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345298 (secure 0 0 0) 0x7f9ef40099e0 con 0x7f9ee006ac90 2024-03-29T09:09:31.958 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee006ac90 msgr2=0x7f9ee006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee006ac90 0x7f9ee006d150 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006d30 tx=0x7f9ef40092c0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00160cc0 msgr2=0x7f9f001e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00160cc0 0x7f9f001e7090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800cc20 tx=0x7f9ef800cf30 comp rx=0 tx=0).stop 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 shutdown_connections 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee006ac90 0x7f9ee006d150 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f00160cc0 0x7f9f001e7090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f000d3a70 0x7f9f000cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 --2- 172.21.15.99:0/3622131455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f001608e0 0x7f9f000b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:31.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 >> 172.21.15.99:0/3622131455 conn(0x7f9f0009c490 msgr2=0x7f9f0015d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 shutdown_connections 2024-03-29T09:09:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:31.959+0000 7f9f07800700 1 -- 172.21.15.99:0/3622131455 wait complete. 2024-03-29T09:09:31.962 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:09:31.977 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-03-29T09:09:30.458109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219392,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755328,"statfs":{"total":773094113280,"available":772869455872,"internally_reserved":0,"allocated":3284992,"data_stored":1988228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004650"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534081+0000","last_change":"2024-03-29T09:08:57.419769+0000","last_active":"2024-03-29T09:09:01.534081+0000","last_peered":"2024-03-29T09:09:01.534081+0000","last_clean":"2024-03-29T09:09:01.534081+0000","last_became_active":"2024-03-29T09:08:57.419455+0000","last_became_peered":"2024-03-29T09:08:57.419455+0000","last_unstale":"2024-03-29T09:09:01.534081+0000","last_undegraded":"2024-03-29T09:09:01.534081+0000","last_fullsized":"2024-03-29T09:09:01.534081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530834+0000","last_change":"2024-03-29T09:08:57.427159+0000","last_active":"2024-03-29T09:09:01.530834+0000","last_peered":"2024-03-29T09:09:01.530834+0000","last_clean":"2024-03-29T09:09:01.530834+0000","last_became_active":"2024-03-29T09:08:57.426926+0000","last_became_peered":"2024-03-29T09:08:57.426926+0000","last_unstale":"2024-03-29T09:09:01.530834+0000","last_undegraded":"2024-03-29T09:09:01.530834+0000","last_fullsized":"2024-03-29T09:09:01.530834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:08:57.419169+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:08:57.418824+0000","last_became_peered":"2024-03-29T09:08:57.418824+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530961+0000","last_change":"2024-03-29T09:08:57.409236+0000","last_active":"2024-03-29T09:09:01.530961+0000","last_peered":"2024-03-29T09:09:01.530961+0000","last_clean":"2024-03-29T09:09:01.530961+0000","last_became_active":"2024-03-29T09:08:57.408981+0000","last_became_peered":"2024-03-29T09:08:57.408981+0000","last_unstale":"2024-03-29T09:09:01.530961+0000","last_undegraded":"2024-03-29T09:09:01.530961+0000","last_fullsized":"2024-03-29T09:09:01.530961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533948+0000","last_change":"2024-03-29T09:08:57.423360+0000","last_active":"2024-03-29T09:09:01.533948+0000","last_peered":"2024-03-29T09:09:01.533948+0000","last_clean":"2024-03-29T09:09:01.533948+0000","last_became_active":"2024-03-29T09:08:57.423088+0000","last_became_peered":"2024-03-29T09:08:57.423088+0000","last_unstale":"2024-03-29T09:09:01.533948+0000","last_undegraded":"2024-03-29T09:09:01.533948+0000","last_fullsized":"2024-03-29T09:09:01.533948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530887+0000","last_change":"2024-03-29T09:08:57.415603+0000","last_active":"2024-03-29T09:09:01.530887+0000","last_peered":"2024-03-29T09:09:01.530887+0000","last_clean":"2024-03-29T09:09:01.530887+0000","last_became_active":"2024-03-29T09:08:57.413227+0000","last_became_peered":"2024-03-29T09:08:57.413227+0000","last_unstale":"2024-03-29T09:09:01.530887+0000","last_undegraded":"2024-03-29T09:09:01.530887+0000","last_fullsized":"2024-03-29T09:09:01.530887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533929+0000","last_change":"2024-03-29T09:08:57.421465+0000","last_active":"2024-03-29T09:09:01.533929+0000","last_peered":"2024-03-29T09:09:01.533929+0000","last_clean":"2024-03-29T09:09:01.533929+0000","last_became_active":"2024-03-29T09:08:57.421189+0000","last_became_peered":"2024-03-29T09:08:57.421189+0000","last_unstale":"2024-03-29T09:09:01.533929+0000","last_undegraded":"2024-03-29T09:09:01.533929+0000","last_fullsized":"2024-03-29T09:09:01.533929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530711+0000","last_change":"2024-03-29T09:08:57.430900+0000","last_active":"2024-03-29T09:09:01.530711+0000","last_peered":"2024-03-29T09:09:01.530711+0000","last_clean":"2024-03-29T09:09:01.530711+0000","last_became_active":"2024-03-29T09:08:57.430636+0000","last_became_peered":"2024-03-29T09:08:57.430636+0000","last_unstale":"2024-03-29T09:09:01.530711+0000","last_undegraded":"2024-03-29T09:09:01.530711+0000","last_fullsized":"2024-03-29T09:09:01.530711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528748+0000","last_change":"2024-03-29T09:08:57.413603+0000","last_active":"2024-03-29T09:09:01.528748+0000","last_peered":"2024-03-29T09:09:01.528748+0000","last_clean":"2024-03-29T09:09:01.528748+0000","last_became_active":"2024-03-29T09:08:57.413373+0000","last_became_peered":"2024-03-29T09:08:57.413373+0000","last_unstale":"2024-03-29T09:09:01.528748+0000","last_undegraded":"2024-03-29T09:09:01.528748+0000","last_fullsized":"2024-03-29T09:09:01.528748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534071+0000","last_change":"2024-03-29T09:08:57.417036+0000","last_active":"2024-03-29T09:09:01.534071+0000","last_peered":"2024-03-29T09:09:01.534071+0000","last_clean":"2024-03-29T09:09:01.534071+0000","last_became_active":"2024-03-29T09:08:57.416762+0000","last_became_peered":"2024-03-29T09:08:57.416762+0000","last_unstale":"2024-03-29T09:09:01.534071+0000","last_undegraded":"2024-03-29T09:09:01.534071+0000","last_fullsized":"2024-03-29T09:09:01.534071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534055+0000","last_change":"2024-03-29T09:08:57.417500+0000","last_active":"2024-03-29T09:09:01.534055+0000","last_peered":"2024-03-29T09:09:01.534055+0000","last_clean":"2024-03-29T09:09:01.534055+0000","last_became_active":"2024-03-29T09:08:57.417187+0000","last_became_peered":"2024-03-29T09:08:57.417187+0000","last_unstale":"2024-03-29T09:09:01.534055+0000","last_undegraded":"2024-03-29T09:09:01.534055+0000","last_fullsized":"2024-03-29T09:09:01.534055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534022+0000","last_change":"2024-03-29T09:08:57.416133+0000","last_active":"2024-03-29T09:09:01.534022+0000","last_peered":"2024-03-29T09:09:01.534022+0000","last_clean":"2024-03-29T09:09:01.534022+0000","last_became_active":"2024-03-29T09:08:57.415878+0000","last_became_peered":"2024-03-29T09:08:57.415878+0000","last_unstale":"2024-03-29T09:09:01.534022+0000","last_undegraded":"2024-03-29T09:09:01.534022+0000","last_fullsized":"2024-03-29T09:09:01.534022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525479+0000","last_change":"2024-03-29T09:08:57.418824+0000","last_active":"2024-03-29T09:09:01.525479+0000","last_peered":"2024-03-29T09:09:01.525479+0000","last_clean":"2024-03-29T09:09:01.525479+0000","last_became_active":"2024-03-29T09:08:57.418534+0000","last_became_peered":"2024-03-29T09:08:57.418534+0000","last_unstale":"2024-03-29T09:09:01.525479+0000","last_undegraded":"2024-03-29T09:09:01.525479+0000","last_fullsized":"2024-03-29T09:09:01.525479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:17:11.207261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525458+0000","last_change":"2024-03-29T09:08:57.412768+0000","last_active":"2024-03-29T09:09:01.525458+0000","last_peered":"2024-03-29T09:09:01.525458+0000","last_clean":"2024-03-29T09:09:01.525458+0000","last_became_active":"2024-03-29T09:08:57.412077+0000","last_became_peered":"2024-03-29T09:08:57.412077+0000","last_unstale":"2024-03-29T09:09:01.525458+0000","last_undegraded":"2024-03-29T09:09:01.525458+0000","last_fullsized":"2024-03-29T09:09:01.525458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530745+0000","last_change":"2024-03-29T09:08:57.430532+0000","last_active":"2024-03-29T09:09:01.530745+0000","last_peered":"2024-03-29T09:09:01.530745+0000","last_clean":"2024-03-29T09:09:01.530745+0000","last_became_active":"2024-03-29T09:08:57.430163+0000","last_became_peered":"2024-03-29T09:08:57.430163+0000","last_unstale":"2024-03-29T09:09:01.530745+0000","last_undegraded":"2024-03-29T09:09:01.530745+0000","last_fullsized":"2024-03-29T09:09:01.530745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527616+0000","last_change":"2024-03-29T09:08:57.413899+0000","last_active":"2024-03-29T09:09:01.527616+0000","last_peered":"2024-03-29T09:09:01.527616+0000","last_clean":"2024-03-29T09:09:01.527616+0000","last_became_active":"2024-03-29T09:08:57.413637+0000","last_became_peered":"2024-03-29T09:08:57.413637+0000","last_unstale":"2024-03-29T09:09:01.527616+0000","last_undegraded":"2024-03-29T09:09:01.527616+0000","last_fullsized":"2024-03-29T09:09:01.527616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:32.639943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527925+0000","last_change":"2024-03-29T09:08:57.418000+0000","last_active":"2024-03-29T09:09:01.527925+0000","last_peered":"2024-03-29T09:09:01.527925+0000","last_clean":"2024-03-29T09:09:01.527925+0000","last_became_active":"2024-03-29T09:08:57.417649+0000","last_became_peered":"2024-03-29T09:08:57.417649+0000","last_unstale":"2024-03-29T09:09:01.527925+0000","last_undegraded":"2024-03-29T09:09:01.527925+0000","last_fullsized":"2024-03-29T09:09:01.527925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:06.479973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527377+0000","last_change":"2024-03-29T09:08:57.430547+0000","last_active":"2024-03-29T09:09:01.527377+0000","last_peered":"2024-03-29T09:09:01.527377+0000","last_clean":"2024-03-29T09:09:01.527377+0000","last_became_active":"2024-03-29T09:08:57.430013+0000","last_became_peered":"2024-03-29T09:08:57.430013+0000","last_unstale":"2024-03-29T09:09:01.527377+0000","last_undegraded":"2024-03-29T09:09:01.527377+0000","last_fullsized":"2024-03-29T09:09:01.527377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527878+0000","last_change":"2024-03-29T09:08:57.419008+0000","last_active":"2024-03-29T09:09:01.527878+0000","last_peered":"2024-03-29T09:09:01.527878+0000","last_clean":"2024-03-29T09:09:01.527878+0000","last_became_active":"2024-03-29T09:08:57.418723+0000","last_became_peered":"2024-03-29T09:08:57.418723+0000","last_unstale":"2024-03-29T09:09:01.527878+0000","last_undegraded":"2024-03-29T09:09:01.527878+0000","last_fullsized":"2024-03-29T09:09:01.527878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:06.268267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527289+0000","last_change":"2024-03-29T09:09:01.448984+0000","last_active":"2024-03-29T09:09:01.527289+0000","last_peered":"2024-03-29T09:09:01.527289+0000","last_clean":"2024-03-29T09:09:01.527289+0000","last_became_active":"2024-03-29T09:09:01.446825+0000","last_became_peered":"2024-03-29T09:09:01.446825+0000","last_unstale":"2024-03-29T09:09:01.527289+0000","last_undegraded":"2024-03-29T09:09:01.527289+0000","last_fullsized":"2024-03-29T09:09:01.527289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:55.385731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.425281+0000","last_change":"2024-03-29T09:08:56.898980+0000","last_active":"2024-03-29T09:09:01.425281+0000","last_peered":"2024-03-29T09:09:01.425281+0000","last_clean":"2024-03-29T09:09:01.425281+0000","last_became_active":"2024-03-29T09:08:56.898646+0000","last_became_peered":"2024-03-29T09:08:56.898646+0000","last_unstale":"2024-03-29T09:09:01.425281+0000","last_undegraded":"2024-03-29T09:09:01.425281+0000","last_fullsized":"2024-03-29T09:09:01.425281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530854+0000","last_change":"2024-03-29T09:08:57.420574+0000","last_active":"2024-03-29T09:09:01.530854+0000","last_peered":"2024-03-29T09:09:01.530854+0000","last_clean":"2024-03-29T09:09:01.530854+0000","last_became_active":"2024-03-29T09:08:57.420318+0000","last_became_peered":"2024-03-29T09:08:57.420318+0000","last_unstale":"2024-03-29T09:09:01.530854+0000","last_undegraded":"2024-03-29T09:09:01.530854+0000","last_fullsized":"2024-03-29T09:09:01.530854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530783+0000","last_change":"2024-03-29T09:09:01.444440+0000","last_active":"2024-03-29T09:09:01.530783+0000","last_peered":"2024-03-29T09:09:01.530783+0000","last_clean":"2024-03-29T09:09:01.530783+0000","last_became_active":"2024-03-29T09:09:01.444026+0000","last_became_peered":"2024-03-29T09:09:01.444026+0000","last_unstale":"2024-03-29T09:09:01.530783+0000","last_undegraded":"2024-03-29T09:09:01.530783+0000","last_fullsized":"2024-03-29T09:09:01.530783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.421739+0000","last_change":"2024-03-29T09:08:56.908349+0000","last_active":"2024-03-29T09:09:01.421739+0000","last_peered":"2024-03-29T09:09:01.421739+0000","last_clean":"2024-03-29T09:09:01.421739+0000","last_became_active":"2024-03-29T09:08:56.908070+0000","last_became_peered":"2024-03-29T09:08:56.908070+0000","last_unstale":"2024-03-29T09:09:01.421739+0000","last_undegraded":"2024-03-29T09:09:01.421739+0000","last_fullsized":"2024-03-29T09:09:01.421739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530920+0000","last_change":"2024-03-29T09:08:57.422633+0000","last_active":"2024-03-29T09:09:01.530920+0000","last_peered":"2024-03-29T09:09:01.530920+0000","last_clean":"2024-03-29T09:09:01.530920+0000","last_became_active":"2024-03-29T09:08:57.422298+0000","last_became_peered":"2024-03-29T09:08:57.422298+0000","last_unstale":"2024-03-29T09:09:01.530920+0000","last_undegraded":"2024-03-29T09:09:01.530920+0000","last_fullsized":"2024-03-29T09:09:01.530920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530925+0000","last_change":"2024-03-29T09:09:01.435391+0000","last_active":"2024-03-29T09:09:01.530925+0000","last_peered":"2024-03-29T09:09:01.530925+0000","last_clean":"2024-03-29T09:09:01.530925+0000","last_became_active":"2024-03-29T09:09:01.435087+0000","last_became_peered":"2024-03-29T09:09:01.435087+0000","last_unstale":"2024-03-29T09:09:01.530925+0000","last_undegraded":"2024-03-29T09:09:01.530925+0000","last_fullsized":"2024-03-29T09:09:01.530925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424816+0000","last_change":"2024-03-29T09:08:56.910805+0000","last_active":"2024-03-29T09:09:01.424816+0000","last_peered":"2024-03-29T09:09:01.424816+0000","last_clean":"2024-03-29T09:09:01.424816+0000","last_became_active":"2024-03-29T09:08:56.908213+0000","last_became_peered":"2024-03-29T09:08:56.908213+0000","last_unstale":"2024-03-29T09:09:01.424816+0000","last_undegraded":"2024-03-29T09:09:01.424816+0000","last_fullsized":"2024-03-29T09:09:01.424816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528733+0000","last_change":"2024-03-29T09:08:57.420307+0000","last_active":"2024-03-29T09:09:01.528733+0000","last_peered":"2024-03-29T09:09:01.528733+0000","last_clean":"2024-03-29T09:09:01.528733+0000","last_became_active":"2024-03-29T09:08:57.420033+0000","last_became_peered":"2024-03-29T09:08:57.420033+0000","last_unstale":"2024-03-29T09:09:01.528733+0000","last_undegraded":"2024-03-29T09:09:01.528733+0000","last_fullsized":"2024-03-29T09:09:01.528733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530866+0000","last_change":"2024-03-29T09:09:01.441108+0000","last_active":"2024-03-29T09:09:01.530866+0000","last_peered":"2024-03-29T09:09:01.530866+0000","last_clean":"2024-03-29T09:09:01.530866+0000","last_became_active":"2024-03-29T09:09:01.440455+0000","last_became_peered":"2024-03-29T09:09:01.440455+0000","last_unstale":"2024-03-29T09:09:01.530866+0000","last_undegraded":"2024-03-29T09:09:01.530866+0000","last_fullsized":"2024-03-29T09:09:01.530866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424962+0000","last_change":"2024-03-29T09:08:56.908246+0000","last_active":"2024-03-29T09:09:01.424962+0000","last_peered":"2024-03-29T09:09:01.424962+0000","last_clean":"2024-03-29T09:09:01.424962+0000","last_became_active":"2024-03-29T09:08:56.907815+0000","last_became_peered":"2024-03-29T09:08:56.907815+0000","last_unstale":"2024-03-29T09:09:01.424962+0000","last_undegraded":"2024-03-29T09:09:01.424962+0000","last_fullsized":"2024-03-29T09:09:01.424962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527840+0000","last_change":"2024-03-29T09:08:57.420229+0000","last_active":"2024-03-29T09:09:01.527840+0000","last_peered":"2024-03-29T09:09:01.527840+0000","last_clean":"2024-03-29T09:09:01.527840+0000","last_became_active":"2024-03-29T09:08:57.419940+0000","last_became_peered":"2024-03-29T09:08:57.419940+0000","last_unstale":"2024-03-29T09:09:01.527840+0000","last_undegraded":"2024-03-29T09:09:01.527840+0000","last_fullsized":"2024-03-29T09:09:01.527840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:19.867399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527346+0000","last_change":"2024-03-29T09:09:01.447861+0000","last_active":"2024-03-29T09:09:01.527346+0000","last_peered":"2024-03-29T09:09:01.527346+0000","last_clean":"2024-03-29T09:09:01.527346+0000","last_became_active":"2024-03-29T09:09:01.447337+0000","last_became_peered":"2024-03-29T09:09:01.447337+0000","last_unstale":"2024-03-29T09:09:01.527346+0000","last_undegraded":"2024-03-29T09:09:01.527346+0000","last_fullsized":"2024-03-29T09:09:01.527346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:19.879877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419991+0000","last_change":"2024-03-29T09:08:56.908710+0000","last_active":"2024-03-29T09:09:01.419991+0000","last_peered":"2024-03-29T09:09:01.419991+0000","last_clean":"2024-03-29T09:09:01.419991+0000","last_became_active":"2024-03-29T09:08:56.906213+0000","last_became_peered":"2024-03-29T09:08:56.906213+0000","last_unstale":"2024-03-29T09:09:01.419991+0000","last_undegraded":"2024-03-29T09:09:01.419991+0000","last_fullsized":"2024-03-29T09:09:01.419991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528714+0000","last_change":"2024-03-29T09:08:57.423981+0000","last_active":"2024-03-29T09:09:01.528714+0000","last_peered":"2024-03-29T09:09:01.528714+0000","last_clean":"2024-03-29T09:09:01.528714+0000","last_became_active":"2024-03-29T09:08:57.423667+0000","last_became_peered":"2024-03-29T09:08:57.423667+0000","last_unstale":"2024-03-29T09:09:01.528714+0000","last_undegraded":"2024-03-29T09:09:01.528714+0000","last_fullsized":"2024-03-29T09:09:01.528714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527487+0000","last_change":"2024-03-29T09:09:01.455365+0000","last_active":"2024-03-29T09:09:01.527487+0000","last_peered":"2024-03-29T09:09:01.527487+0000","last_clean":"2024-03-29T09:09:01.527487+0000","last_became_active":"2024-03-29T09:09:01.453949+0000","last_became_peered":"2024-03-29T09:09:01.453949+0000","last_unstale":"2024-03-29T09:09:01.527487+0000","last_undegraded":"2024-03-29T09:09:01.527487+0000","last_fullsized":"2024-03-29T09:09:01.527487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:00.449212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414280+0000","last_change":"2024-03-29T09:08:56.912593+0000","last_active":"2024-03-29T09:09:01.414280+0000","last_peered":"2024-03-29T09:09:01.414280+0000","last_clean":"2024-03-29T09:09:01.414280+0000","last_became_active":"2024-03-29T09:08:56.912401+0000","last_became_peered":"2024-03-29T09:08:56.912401+0000","last_unstale":"2024-03-29T09:09:01.414280+0000","last_undegraded":"2024-03-29T09:09:01.414280+0000","last_fullsized":"2024-03-29T09:09:01.414280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:27.805156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527468+0000","last_change":"2024-03-29T09:08:57.421182+0000","last_active":"2024-03-29T09:09:01.527468+0000","last_peered":"2024-03-29T09:09:01.527468+0000","last_clean":"2024-03-29T09:09:01.527468+0000","last_became_active":"2024-03-29T09:08:57.420980+0000","last_became_peered":"2024-03-29T09:08:57.420980+0000","last_unstale":"2024-03-29T09:09:01.527468+0000","last_undegraded":"2024-03-29T09:09:01.527468+0000","last_fullsized":"2024-03-29T09:09:01.527468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527895+0000","last_change":"2024-03-29T09:09:01.433710+0000","last_active":"2024-03-29T09:09:01.527895+0000","last_peered":"2024-03-29T09:09:01.527895+0000","last_clean":"2024-03-29T09:09:01.527895+0000","last_became_active":"2024-03-29T09:09:01.433161+0000","last_became_peered":"2024-03-29T09:09:01.433161+0000","last_unstale":"2024-03-29T09:09:01.527895+0000","last_undegraded":"2024-03-29T09:09:01.527895+0000","last_fullsized":"2024-03-29T09:09:01.527895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:07:30.745379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429199+0000","last_change":"2024-03-29T09:08:56.911770+0000","last_active":"2024-03-29T09:09:01.429199+0000","last_peered":"2024-03-29T09:09:01.429199+0000","last_clean":"2024-03-29T09:09:01.429199+0000","last_became_active":"2024-03-29T09:08:56.911493+0000","last_became_peered":"2024-03-29T09:08:56.911493+0000","last_unstale":"2024-03-29T09:09:01.429199+0000","last_undegraded":"2024-03-29T09:09:01.429199+0000","last_fullsized":"2024-03-29T09:09:01.429199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530721+0000","last_change":"2024-03-29T09:08:57.426875+0000","last_active":"2024-03-29T09:09:01.530721+0000","last_peered":"2024-03-29T09:09:01.530721+0000","last_clean":"2024-03-29T09:09:01.530721+0000","last_became_active":"2024-03-29T09:08:57.426586+0000","last_became_peered":"2024-03-29T09:08:57.426586+0000","last_unstale":"2024-03-29T09:09:01.530721+0000","last_undegraded":"2024-03-29T09:09:01.530721+0000","last_fullsized":"2024-03-29T09:09:01.530721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527529+0000","last_change":"2024-03-29T09:09:01.440649+0000","last_active":"2024-03-29T09:09:01.527529+0000","last_peered":"2024-03-29T09:09:01.527529+0000","last_clean":"2024-03-29T09:09:01.527529+0000","last_became_active":"2024-03-29T09:09:01.440252+0000","last_became_peered":"2024-03-29T09:09:01.440252+0000","last_unstale":"2024-03-29T09:09:01.527529+0000","last_undegraded":"2024-03-29T09:09:01.527529+0000","last_fullsized":"2024-03-29T09:09:01.527529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414326+0000","last_change":"2024-03-29T09:08:56.908431+0000","last_active":"2024-03-29T09:09:01.414326+0000","last_peered":"2024-03-29T09:09:01.414326+0000","last_clean":"2024-03-29T09:09:01.414326+0000","last_became_active":"2024-03-29T09:08:56.906823+0000","last_became_peered":"2024-03-29T09:08:56.906823+0000","last_unstale":"2024-03-29T09:09:01.414326+0000","last_undegraded":"2024-03-29T09:09:01.414326+0000","last_fullsized":"2024-03-29T09:09:01.414326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:24.070569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527421+0000","last_change":"2024-03-29T09:08:57.410009+0000","last_active":"2024-03-29T09:09:01.527421+0000","last_peered":"2024-03-29T09:09:01.527421+0000","last_clean":"2024-03-29T09:09:01.527421+0000","last_became_active":"2024-03-29T09:08:57.409632+0000","last_became_peered":"2024-03-29T09:08:57.409632+0000","last_unstale":"2024-03-29T09:09:01.527421+0000","last_undegraded":"2024-03-29T09:09:01.527421+0000","last_fullsized":"2024-03-29T09:09:01.527421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527438+0000","last_change":"2024-03-29T09:09:01.447901+0000","last_active":"2024-03-29T09:09:01.527438+0000","last_peered":"2024-03-29T09:09:01.527438+0000","last_clean":"2024-03-29T09:09:01.527438+0000","last_became_active":"2024-03-29T09:09:01.447482+0000","last_became_peered":"2024-03-29T09:09:01.447482+0000","last_unstale":"2024-03-29T09:09:01.527438+0000","last_undegraded":"2024-03-29T09:09:01.527438+0000","last_fullsized":"2024-03-29T09:09:01.527438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.426470+0000","last_change":"2024-03-29T09:08:56.909506+0000","last_active":"2024-03-29T09:09:01.426470+0000","last_peered":"2024-03-29T09:09:01.426470+0000","last_clean":"2024-03-29T09:09:01.426470+0000","last_became_active":"2024-03-29T09:08:56.909183+0000","last_became_peered":"2024-03-29T09:08:56.909183+0000","last_unstale":"2024-03-29T09:09:01.426470+0000","last_undegraded":"2024-03-29T09:09:01.426470+0000","last_fullsized":"2024-03-29T09:09:01.426470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:04:15.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531038+0000","last_change":"2024-03-29T09:08:57.413715+0000","last_active":"2024-03-29T09:09:01.531038+0000","last_peered":"2024-03-29T09:09:01.531038+0000","last_clean":"2024-03-29T09:09:01.531038+0000","last_became_active":"2024-03-29T09:08:57.413145+0000","last_became_peered":"2024-03-29T09:08:57.413145+0000","last_unstale":"2024-03-29T09:09:01.531038+0000","last_undegraded":"2024-03-29T09:09:01.531038+0000","last_fullsized":"2024-03-29T09:09:01.531038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527274+0000","last_change":"2024-03-29T09:09:01.444258+0000","last_active":"2024-03-29T09:09:01.527274+0000","last_peered":"2024-03-29T09:09:01.527274+0000","last_clean":"2024-03-29T09:09:01.527274+0000","last_became_active":"2024-03-29T09:09:01.443829+0000","last_became_peered":"2024-03-29T09:09:01.443829+0000","last_unstale":"2024-03-29T09:09:01.527274+0000","last_undegraded":"2024-03-29T09:09:01.527274+0000","last_fullsized":"2024-03-29T09:09:01.527274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:29:01.967027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424436+0000","last_change":"2024-03-29T09:08:56.909671+0000","last_active":"2024-03-29T09:09:01.424436+0000","last_peered":"2024-03-29T09:09:01.424436+0000","last_clean":"2024-03-29T09:09:01.424436+0000","last_became_active":"2024-03-29T09:08:56.909426+0000","last_became_peered":"2024-03-29T09:08:56.909426+0000","last_unstale":"2024-03-29T09:09:01.424436+0000","last_undegraded":"2024-03-29T09:09:01.424436+0000","last_fullsized":"2024-03-29T09:09:01.424436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530931+0000","last_change":"2024-03-29T09:08:57.422965+0000","last_active":"2024-03-29T09:09:01.530931+0000","last_peered":"2024-03-29T09:09:01.530931+0000","last_clean":"2024-03-29T09:09:01.530931+0000","last_became_active":"2024-03-29T09:08:57.422689+0000","last_became_peered":"2024-03-29T09:08:57.422689+0000","last_unstale":"2024-03-29T09:09:01.530931+0000","last_undegraded":"2024-03-29T09:09:01.530931+0000","last_fullsized":"2024-03-29T09:09:01.530931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530907+0000","last_change":"2024-03-29T09:09:01.447399+0000","last_active":"2024-03-29T09:09:01.530907+0000","last_peered":"2024-03-29T09:09:01.530907+0000","last_clean":"2024-03-29T09:09:01.530907+0000","last_became_active":"2024-03-29T09:09:01.446880+0000","last_became_peered":"2024-03-29T09:09:01.446880+0000","last_unstale":"2024-03-29T09:09:01.530907+0000","last_undegraded":"2024-03-29T09:09:01.530907+0000","last_fullsized":"2024-03-29T09:09:01.530907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:49:27.133997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.433574+0000","last_change":"2024-03-29T09:08:56.908399+0000","last_active":"2024-03-29T09:09:01.433574+0000","last_peered":"2024-03-29T09:09:01.433574+0000","last_clean":"2024-03-29T09:09:01.433574+0000","last_became_active":"2024-03-29T09:08:56.906866+0000","last_became_peered":"2024-03-29T09:08:56.906866+0000","last_unstale":"2024-03-29T09:09:01.433574+0000","last_undegraded":"2024-03-29T09:09:01.433574+0000","last_fullsized":"2024-03-29T09:09:01.433574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527738+0000","last_change":"2024-03-29T09:08:57.423735+0000","last_active":"2024-03-29T09:09:01.527738+0000","last_peered":"2024-03-29T09:09:01.527738+0000","last_clean":"2024-03-29T09:09:01.527738+0000","last_became_active":"2024-03-29T09:08:57.423405+0000","last_became_peered":"2024-03-29T09:08:57.423405+0000","last_unstale":"2024-03-29T09:09:01.527738+0000","last_undegraded":"2024-03-29T09:09:01.527738+0000","last_fullsized":"2024-03-29T09:09:01.527738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527448+0000","last_change":"2024-03-29T09:09:01.442875+0000","last_active":"2024-03-29T09:09:01.527448+0000","last_peered":"2024-03-29T09:09:01.527448+0000","last_clean":"2024-03-29T09:09:01.527448+0000","last_became_active":"2024-03-29T09:09:01.442426+0000","last_became_peered":"2024-03-29T09:09:01.442426+0000","last_unstale":"2024-03-29T09:09:01.527448+0000","last_undegraded":"2024-03-29T09:09:01.527448+0000","last_fullsized":"2024-03-29T09:09:01.527448+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T09:09:15.577021+0000","last_change":"2024-03-29T09:08:56.895791+0000","last_active":"2024-03-29T09:09:15.577021+0000","last_peered":"2024-03-29T09:09:15.577021+0000","last_clean":"2024-03-29T09:09:15.577021+0000","last_became_active":"2024-03-29T09:08:56.894997+0000","last_became_peered":"2024-03-29T09:08:56.894997+0000","last_unstale":"2024-03-29T09:09:15.577021+0000","last_undegraded":"2024-03-29T09:09:15.577021+0000","last_fullsized":"2024-03-29T09:09:15.577021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534030+0000","last_change":"2024-03-29T09:08:57.427194+0000","last_active":"2024-03-29T09:09:01.534030+0000","last_peered":"2024-03-29T09:09:01.534030+0000","last_clean":"2024-03-29T09:09:01.534030+0000","last_became_active":"2024-03-29T09:08:57.425015+0000","last_became_peered":"2024-03-29T09:08:57.425015+0000","last_unstale":"2024-03-29T09:09:01.534030+0000","last_undegraded":"2024-03-29T09:09:01.534030+0000","last_fullsized":"2024-03-29T09:09:01.534030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527755+0000","last_change":"2024-03-29T09:09:01.442768+0000","last_active":"2024-03-29T09:09:01.527755+0000","last_peered":"2024-03-29T09:09:01.527755+0000","last_clean":"2024-03-29T09:09:01.527755+0000","last_became_active":"2024-03-29T09:09:01.442373+0000","last_became_peered":"2024-03-29T09:09:01.442373+0000","last_unstale":"2024-03-29T09:09:01.527755+0000","last_undegraded":"2024-03-29T09:09:01.527755+0000","last_fullsized":"2024-03-29T09:09:01.527755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:05:41.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.432870+0000","last_change":"2024-03-29T09:08:56.912783+0000","last_active":"2024-03-29T09:09:01.432870+0000","last_peered":"2024-03-29T09:09:01.432870+0000","last_clean":"2024-03-29T09:09:01.432870+0000","last_became_active":"2024-03-29T09:08:56.912488+0000","last_became_peered":"2024-03-29T09:08:56.912488+0000","last_unstale":"2024-03-29T09:09:01.432870+0000","last_undegraded":"2024-03-29T09:09:01.432870+0000","last_fullsized":"2024-03-29T09:09:01.432870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525440+0000","last_change":"2024-03-29T09:08:57.417913+0000","last_active":"2024-03-29T09:09:01.525440+0000","last_peered":"2024-03-29T09:09:01.525440+0000","last_clean":"2024-03-29T09:09:01.525440+0000","last_became_active":"2024-03-29T09:08:57.416399+0000","last_became_peered":"2024-03-29T09:08:57.416399+0000","last_unstale":"2024-03-29T09:09:01.525440+0000","last_undegraded":"2024-03-29T09:09:01.525440+0000","last_fullsized":"2024-03-29T09:09:01.525440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525543+0000","last_change":"2024-03-29T09:09:01.448636+0000","last_active":"2024-03-29T09:09:01.525543+0000","last_peered":"2024-03-29T09:09:01.525543+0000","last_clean":"2024-03-29T09:09:01.525543+0000","last_became_active":"2024-03-29T09:09:01.448098+0000","last_became_peered":"2024-03-29T09:09:01.448098+0000","last_unstale":"2024-03-29T09:09:01.525543+0000","last_undegraded":"2024-03-29T09:09:01.525543+0000","last_fullsized":"2024-03-29T09:09:01.525543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414449+0000","last_change":"2024-03-29T09:08:56.906000+0000","last_active":"2024-03-29T09:09:01.414449+0000","last_peered":"2024-03-29T09:09:01.414449+0000","last_clean":"2024-03-29T09:09:01.414449+0000","last_became_active":"2024-03-29T09:08:56.905743+0000","last_became_peered":"2024-03-29T09:08:56.905743+0000","last_unstale":"2024-03-29T09:09:01.414449+0000","last_undegraded":"2024-03-29T09:09:01.414449+0000","last_fullsized":"2024-03-29T09:09:01.414449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527722+0000","last_change":"2024-03-29T09:08:57.429076+0000","last_active":"2024-03-29T09:09:01.527722+0000","last_peered":"2024-03-29T09:09:01.527722+0000","last_clean":"2024-03-29T09:09:01.527722+0000","last_became_active":"2024-03-29T09:08:57.426637+0000","last_became_peered":"2024-03-29T09:08:57.426637+0000","last_unstale":"2024-03-29T09:09:01.527722+0000","last_undegraded":"2024-03-29T09:09:01.527722+0000","last_fullsized":"2024-03-29T09:09:01.527722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:48.952540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530885+0000","last_change":"2024-03-29T09:09:01.437381+0000","last_active":"2024-03-29T09:09:01.530885+0000","last_peered":"2024-03-29T09:09:01.530885+0000","last_clean":"2024-03-29T09:09:01.530885+0000","last_became_active":"2024-03-29T09:09:01.436950+0000","last_became_peered":"2024-03-29T09:09:01.436950+0000","last_unstale":"2024-03-29T09:09:01.530885+0000","last_undegraded":"2024-03-29T09:09:01.530885+0000","last_fullsized":"2024-03-29T09:09:01.530885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.356624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422703+0000","last_change":"2024-03-29T09:08:56.915220+0000","last_active":"2024-03-29T09:09:01.422703+0000","last_peered":"2024-03-29T09:09:01.422703+0000","last_clean":"2024-03-29T09:09:01.422703+0000","last_became_active":"2024-03-29T09:08:56.914954+0000","last_became_peered":"2024-03-29T09:08:56.914954+0000","last_unstale":"2024-03-29T09:09:01.422703+0000","last_undegraded":"2024-03-29T09:09:01.422703+0000","last_fullsized":"2024-03-29T09:09:01.422703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:19:55.114280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528690+0000","last_change":"2024-03-29T09:08:57.423950+0000","last_active":"2024-03-29T09:09:01.528690+0000","last_peered":"2024-03-29T09:09:01.528690+0000","last_clean":"2024-03-29T09:09:01.528690+0000","last_became_active":"2024-03-29T09:08:57.423703+0000","last_became_peered":"2024-03-29T09:08:57.423703+0000","last_unstale":"2024-03-29T09:09:01.528690+0000","last_undegraded":"2024-03-29T09:09:01.528690+0000","last_fullsized":"2024-03-29T09:09:01.528690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:28.288403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530874+0000","last_change":"2024-03-29T09:09:01.440631+0000","last_active":"2024-03-29T09:09:01.530874+0000","last_peered":"2024-03-29T09:09:01.530874+0000","last_clean":"2024-03-29T09:09:01.530874+0000","last_became_active":"2024-03-29T09:09:01.440159+0000","last_became_peered":"2024-03-29T09:09:01.440159+0000","last_unstale":"2024-03-29T09:09:01.530874+0000","last_undegraded":"2024-03-29T09:09:01.530874+0000","last_fullsized":"2024-03-29T09:09:01.530874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.423297+0000","last_change":"2024-03-29T09:08:56.904028+0000","last_active":"2024-03-29T09:09:01.423297+0000","last_peered":"2024-03-29T09:09:01.423297+0000","last_clean":"2024-03-29T09:09:01.423297+0000","last_became_active":"2024-03-29T09:08:56.903620+0000","last_became_peered":"2024-03-29T09:08:56.903620+0000","last_unstale":"2024-03-29T09:09:01.423297+0000","last_undegraded":"2024-03-29T09:09:01.423297+0000","last_fullsized":"2024-03-29T09:09:01.423297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527655+0000","last_change":"2024-03-29T09:08:57.414230+0000","last_active":"2024-03-29T09:09:01.527655+0000","last_peered":"2024-03-29T09:09:01.527655+0000","last_clean":"2024-03-29T09:09:01.527655+0000","last_became_active":"2024-03-29T09:08:57.413731+0000","last_became_peered":"2024-03-29T09:08:57.413731+0000","last_unstale":"2024-03-29T09:09:01.527655+0000","last_undegraded":"2024-03-29T09:09:01.527655+0000","last_fullsized":"2024-03-29T09:09:01.527655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528703+0000","last_change":"2024-03-29T09:09:01.435579+0000","last_active":"2024-03-29T09:09:01.528703+0000","last_peered":"2024-03-29T09:09:01.528703+0000","last_clean":"2024-03-29T09:09:01.528703+0000","last_became_active":"2024-03-29T09:09:01.433461+0000","last_became_peered":"2024-03-29T09:09:01.433461+0000","last_unstale":"2024-03-29T09:09:01.528703+0000","last_undegraded":"2024-03-29T09:09:01.528703+0000","last_fullsized":"2024-03-29T09:09:01.528703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:17.513175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422681+0000","last_change":"2024-03-29T09:08:56.915230+0000","last_active":"2024-03-29T09:09:01.422681+0000","last_peered":"2024-03-29T09:09:01.422681+0000","last_clean":"2024-03-29T09:09:01.422681+0000","last_became_active":"2024-03-29T09:08:56.914943+0000","last_became_peered":"2024-03-29T09:08:56.914943+0000","last_unstale":"2024-03-29T09:09:01.422681+0000","last_undegraded":"2024-03-29T09:09:01.422681+0000","last_fullsized":"2024-03-29T09:09:01.422681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531054+0000","last_change":"2024-03-29T09:08:57.425607+0000","last_active":"2024-03-29T09:09:01.531054+0000","last_peered":"2024-03-29T09:09:01.531054+0000","last_clean":"2024-03-29T09:09:01.531054+0000","last_became_active":"2024-03-29T09:08:57.424221+0000","last_became_peered":"2024-03-29T09:08:57.424221+0000","last_unstale":"2024-03-29T09:09:01.531054+0000","last_undegraded":"2024-03-29T09:09:01.531054+0000","last_fullsized":"2024-03-29T09:09:01.531054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525430+0000","last_change":"2024-03-29T09:09:01.447292+0000","last_active":"2024-03-29T09:09:01.525430+0000","last_peered":"2024-03-29T09:09:01.525430+0000","last_clean":"2024-03-29T09:09:01.525430+0000","last_became_active":"2024-03-29T09:09:01.446779+0000","last_became_peered":"2024-03-29T09:09:01.446779+0000","last_unstale":"2024-03-29T09:09:01.525430+0000","last_undegraded":"2024-03-29T09:09:01.525430+0000","last_fullsized":"2024-03-29T09:09:01.525430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:10.002579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419894+0000","last_change":"2024-03-29T09:08:56.905021+0000","last_active":"2024-03-29T09:09:01.419894+0000","last_peered":"2024-03-29T09:09:01.419894+0000","last_clean":"2024-03-29T09:09:01.419894+0000","last_became_active":"2024-03-29T09:08:56.903363+0000","last_became_peered":"2024-03-29T09:08:56.903363+0000","last_unstale":"2024-03-29T09:09:01.419894+0000","last_undegraded":"2024-03-29T09:09:01.419894+0000","last_fullsized":"2024-03-29T09:09:01.419894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530999+0000","last_change":"2024-03-29T09:08:57.427940+0000","last_active":"2024-03-29T09:09:01.530999+0000","last_peered":"2024-03-29T09:09:01.530999+0000","last_clean":"2024-03-29T09:09:01.530999+0000","last_became_active":"2024-03-29T09:08:57.427293+0000","last_became_peered":"2024-03-29T09:08:57.427293+0000","last_unstale":"2024-03-29T09:09:01.530999+0000","last_undegraded":"2024-03-29T09:09:01.530999+0000","last_fullsized":"2024-03-29T09:09:01.530999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525532+0000","last_change":"2024-03-29T09:09:01.451319+0000","last_active":"2024-03-29T09:09:01.525532+0000","last_peered":"2024-03-29T09:09:01.525532+0000","last_clean":"2024-03-29T09:09:01.525532+0000","last_became_active":"2024-03-29T09:09:01.450880+0000","last_became_peered":"2024-03-29T09:09:01.450880+0000","last_unstale":"2024-03-29T09:09:01.525532+0000","last_undegraded":"2024-03-29T09:09:01.525532+0000","last_fullsized":"2024-03-29T09:09:01.525532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414329+0000","last_change":"2024-03-29T09:08:56.909170+0000","last_active":"2024-03-29T09:09:01.414329+0000","last_peered":"2024-03-29T09:09:01.414329+0000","last_clean":"2024-03-29T09:09:01.414329+0000","last_became_active":"2024-03-29T09:08:56.908866+0000","last_became_peered":"2024-03-29T09:08:56.908866+0000","last_unstale":"2024-03-29T09:09:01.414329+0000","last_undegraded":"2024-03-29T09:09:01.414329+0000","last_fullsized":"2024-03-29T09:09:01.414329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:58.589863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533977+0000","last_change":"2024-03-29T09:08:57.415898+0000","last_active":"2024-03-29T09:09:01.533977+0000","last_peered":"2024-03-29T09:09:01.533977+0000","last_clean":"2024-03-29T09:09:01.533977+0000","last_became_active":"2024-03-29T09:08:57.415653+0000","last_became_peered":"2024-03-29T09:08:57.415653+0000","last_unstale":"2024-03-29T09:09:01.533977+0000","last_undegraded":"2024-03-29T09:09:01.533977+0000","last_fullsized":"2024-03-29T09:09:01.533977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525566+0000","last_change":"2024-03-29T09:09:01.451066+0000","last_active":"2024-03-29T09:09:01.525566+0000","last_peered":"2024-03-29T09:09:01.525566+0000","last_clean":"2024-03-29T09:09:01.525566+0000","last_became_active":"2024-03-29T09:09:01.450444+0000","last_became_peered":"2024-03-29T09:09:01.450444+0000","last_unstale":"2024-03-29T09:09:01.525566+0000","last_undegraded":"2024-03-29T09:09:01.525566+0000","last_fullsized":"2024-03-29T09:09:01.525566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530697+0000","last_change":"2024-03-29T09:08:57.426320+0000","last_active":"2024-03-29T09:09:01.530697+0000","last_peered":"2024-03-29T09:09:01.530697+0000","last_clean":"2024-03-29T09:09:01.530697+0000","last_became_active":"2024-03-29T09:08:57.426071+0000","last_became_peered":"2024-03-29T09:08:57.426071+0000","last_unstale":"2024-03-29T09:09:01.530697+0000","last_undegraded":"2024-03-29T09:09:01.530697+0000","last_fullsized":"2024-03-29T09:09:01.530697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.428903+0000","last_change":"2024-03-29T09:08:56.906294+0000","last_active":"2024-03-29T09:09:01.428903+0000","last_peered":"2024-03-29T09:09:01.428903+0000","last_clean":"2024-03-29T09:09:01.428903+0000","last_became_active":"2024-03-29T09:08:56.905993+0000","last_became_peered":"2024-03-29T09:08:56.905993+0000","last_unstale":"2024-03-29T09:09:01.428903+0000","last_undegraded":"2024-03-29T09:09:01.428903+0000","last_fullsized":"2024-03-29T09:09:01.428903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527859+0000","last_change":"2024-03-29T09:09:01.435004+0000","last_active":"2024-03-29T09:09:01.527859+0000","last_peered":"2024-03-29T09:09:01.527859+0000","last_clean":"2024-03-29T09:09:01.527859+0000","last_became_active":"2024-03-29T09:09:01.434628+0000","last_became_peered":"2024-03-29T09:09:01.434628+0000","last_unstale":"2024-03-29T09:09:01.527859+0000","last_undegraded":"2024-03-29T09:09:01.527859+0000","last_fullsized":"2024-03-29T09:09:01.527859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:05.380910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527458+0000","last_change":"2024-03-29T09:08:57.411609+0000","last_active":"2024-03-29T09:09:01.527458+0000","last_peered":"2024-03-29T09:09:01.527458+0000","last_clean":"2024-03-29T09:09:01.527458+0000","last_became_active":"2024-03-29T09:08:57.411313+0000","last_became_peered":"2024-03-29T09:08:57.411313+0000","last_unstale":"2024-03-29T09:09:01.527458+0000","last_undegraded":"2024-03-29T09:09:01.527458+0000","last_fullsized":"2024-03-29T09:09:01.527458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414265+0000","last_change":"2024-03-29T09:08:56.900590+0000","last_active":"2024-03-29T09:09:01.414265+0000","last_peered":"2024-03-29T09:09:01.414265+0000","last_clean":"2024-03-29T09:09:01.414265+0000","last_became_active":"2024-03-29T09:08:56.900329+0000","last_became_peered":"2024-03-29T09:08:56.900329+0000","last_unstale":"2024-03-29T09:09:01.414265+0000","last_undegraded":"2024-03-29T09:09:01.414265+0000","last_fullsized":"2024-03-29T09:09:01.414265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:56.593036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420635+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:09:01.420635+0000","last_peered":"2024-03-29T09:09:01.420635+0000","last_clean":"2024-03-29T09:09:01.420635+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:09:01.420635+0000","last_undegraded":"2024-03-29T09:09:01.420635+0000","last_fullsized":"2024-03-29T09:09:01.420635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530864+0000","last_change":"2024-03-29T09:09:01.438759+0000","last_active":"2024-03-29T09:09:01.530864+0000","last_peered":"2024-03-29T09:09:01.530864+0000","last_clean":"2024-03-29T09:09:01.530864+0000","last_became_active":"2024-03-29T09:09:01.438275+0000","last_became_peered":"2024-03-29T09:09:01.438275+0000","last_unstale":"2024-03-29T09:09:01.530864+0000","last_undegraded":"2024-03-29T09:09:01.530864+0000","last_fullsized":"2024-03-29T09:09:01.530864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:02:29.227678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525418+0000","last_change":"2024-03-29T09:08:57.418469+0000","last_active":"2024-03-29T09:09:01.525418+0000","last_peered":"2024-03-29T09:09:01.525418+0000","last_clean":"2024-03-29T09:09:01.525418+0000","last_became_active":"2024-03-29T09:08:57.416011+0000","last_became_peered":"2024-03-29T09:08:57.416011+0000","last_unstale":"2024-03-29T09:09:01.525418+0000","last_undegraded":"2024-03-29T09:09:01.525418+0000","last_fullsized":"2024-03-29T09:09:01.525418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414118+0000","last_change":"2024-03-29T09:08:56.900382+0000","last_active":"2024-03-29T09:09:01.414118+0000","last_peered":"2024-03-29T09:09:01.414118+0000","last_clean":"2024-03-29T09:09:01.414118+0000","last_became_active":"2024-03-29T09:08:56.899884+0000","last_became_peered":"2024-03-29T09:08:56.899884+0000","last_unstale":"2024-03-29T09:09:01.414118+0000","last_undegraded":"2024-03-29T09:09:01.414118+0000","last_fullsized":"2024-03-29T09:09:01.414118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:51.548290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525469+0000","last_change":"2024-03-29T09:09:01.447790+0000","last_active":"2024-03-29T09:09:01.525469+0000","last_peered":"2024-03-29T09:09:01.525469+0000","last_clean":"2024-03-29T09:09:01.525469+0000","last_became_active":"2024-03-29T09:09:01.447254+0000","last_became_peered":"2024-03-29T09:09:01.447254+0000","last_unstale":"2024-03-29T09:09:01.525469+0000","last_undegraded":"2024-03-29T09:09:01.525469+0000","last_fullsized":"2024-03-29T09:09:01.525469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:27.709797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.415102+0000","last_change":"2024-03-29T09:08:56.904347+0000","last_active":"2024-03-29T09:09:01.415102+0000","last_peered":"2024-03-29T09:09:01.415102+0000","last_clean":"2024-03-29T09:09:01.415102+0000","last_became_active":"2024-03-29T09:08:56.903387+0000","last_became_peered":"2024-03-29T09:08:56.903387+0000","last_unstale":"2024-03-29T09:09:01.415102+0000","last_undegraded":"2024-03-29T09:09:01.415102+0000","last_fullsized":"2024-03-29T09:09:01.415102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525510+0000","last_change":"2024-03-29T09:08:57.419559+0000","last_active":"2024-03-29T09:09:01.525510+0000","last_peered":"2024-03-29T09:09:01.525510+0000","last_clean":"2024-03-29T09:09:01.525510+0000","last_became_active":"2024-03-29T09:08:57.419295+0000","last_became_peered":"2024-03-29T09:08:57.419295+0000","last_unstale":"2024-03-29T09:09:01.525510+0000","last_undegraded":"2024-03-29T09:09:01.525510+0000","last_fullsized":"2024-03-29T09:09:01.525510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527366+0000","last_change":"2024-03-29T09:09:01.433109+0000","last_active":"2024-03-29T09:09:01.527366+0000","last_peered":"2024-03-29T09:09:01.527366+0000","last_clean":"2024-03-29T09:09:01.527366+0000","last_became_active":"2024-03-29T09:09:01.432662+0000","last_became_peered":"2024-03-29T09:09:01.432662+0000","last_unstale":"2024-03-29T09:09:01.527366+0000","last_undegraded":"2024-03-29T09:09:01.527366+0000","last_fullsized":"2024-03-29T09:09:01.527366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422867+0000","last_change":"2024-03-29T09:08:56.912181+0000","last_active":"2024-03-29T09:09:01.422867+0000","last_peered":"2024-03-29T09:09:01.422867+0000","last_clean":"2024-03-29T09:09:01.422867+0000","last_became_active":"2024-03-29T09:08:56.911842+0000","last_became_peered":"2024-03-29T09:08:56.911842+0000","last_unstale":"2024-03-29T09:09:01.422867+0000","last_undegraded":"2024-03-29T09:09:01.422867+0000","last_fullsized":"2024-03-29T09:09:01.422867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527556+0000","last_change":"2024-03-29T09:08:57.423784+0000","last_active":"2024-03-29T09:09:01.527556+0000","last_peered":"2024-03-29T09:09:01.527556+0000","last_clean":"2024-03-29T09:09:01.527556+0000","last_became_active":"2024-03-29T09:08:57.423442+0000","last_became_peered":"2024-03-29T09:08:57.423442+0000","last_unstale":"2024-03-29T09:09:01.527556+0000","last_undegraded":"2024-03-29T09:09:01.527556+0000","last_fullsized":"2024-03-29T09:09:01.527556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:09:01.451554+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:09:01.450348+0000","last_became_peered":"2024-03-29T09:09:01.450348+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414924+0000","last_change":"2024-03-29T09:08:56.911357+0000","last_active":"2024-03-29T09:09:01.414924+0000","last_peered":"2024-03-29T09:09:01.414924+0000","last_clean":"2024-03-29T09:09:01.414924+0000","last_became_active":"2024-03-29T09:08:56.910692+0000","last_became_peered":"2024-03-29T09:08:56.910692+0000","last_unstale":"2024-03-29T09:09:01.414924+0000","last_undegraded":"2024-03-29T09:09:01.414924+0000","last_fullsized":"2024-03-29T09:09:01.414924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:32.927312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527540+0000","last_change":"2024-03-29T09:08:57.415718+0000","last_active":"2024-03-29T09:09:01.527540+0000","last_peered":"2024-03-29T09:09:01.527540+0000","last_clean":"2024-03-29T09:09:01.527540+0000","last_became_active":"2024-03-29T09:08:57.415442+0000","last_became_peered":"2024-03-29T09:08:57.415442+0000","last_unstale":"2024-03-29T09:09:01.527540+0000","last_undegraded":"2024-03-29T09:09:01.527540+0000","last_fullsized":"2024-03-29T09:09:01.527540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:44.638689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531018+0000","last_change":"2024-03-29T09:09:01.444447+0000","last_active":"2024-03-29T09:09:01.531018+0000","last_peered":"2024-03-29T09:09:01.531018+0000","last_clean":"2024-03-29T09:09:01.531018+0000","last_became_active":"2024-03-29T09:09:01.443976+0000","last_became_peered":"2024-03-29T09:09:01.443976+0000","last_unstale":"2024-03-29T09:09:01.531018+0000","last_undegraded":"2024-03-29T09:09:01.531018+0000","last_fullsized":"2024-03-29T09:09:01.531018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422915+0000","last_change":"2024-03-29T09:08:56.904777+0000","last_active":"2024-03-29T09:09:01.422915+0000","last_peered":"2024-03-29T09:09:01.422915+0000","last_clean":"2024-03-29T09:09:01.422915+0000","last_became_active":"2024-03-29T09:08:56.904503+0000","last_became_peered":"2024-03-29T09:08:56.904503+0000","last_unstale":"2024-03-29T09:09:01.422915+0000","last_undegraded":"2024-03-29T09:09:01.422915+0000","last_fullsized":"2024-03-29T09:09:01.422915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527647+0000","last_change":"2024-03-29T09:08:57.409679+0000","last_active":"2024-03-29T09:09:01.527647+0000","last_peered":"2024-03-29T09:09:01.527647+0000","last_clean":"2024-03-29T09:09:01.527647+0000","last_became_active":"2024-03-29T09:08:57.409346+0000","last_became_peered":"2024-03-29T09:08:57.409346+0000","last_unstale":"2024-03-29T09:09:01.527647+0000","last_undegraded":"2024-03-29T09:09:01.527647+0000","last_fullsized":"2024-03-29T09:09:01.527647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527392+0000","last_change":"2024-03-29T09:09:01.451808+0000","last_active":"2024-03-29T09:09:01.527392+0000","last_peered":"2024-03-29T09:09:01.527392+0000","last_clean":"2024-03-29T09:09:01.527392+0000","last_became_active":"2024-03-29T09:09:01.451262+0000","last_became_peered":"2024-03-29T09:09:01.451262+0000","last_unstale":"2024-03-29T09:09:01.527392+0000","last_undegraded":"2024-03-29T09:09:01.527392+0000","last_fullsized":"2024-03-29T09:09:01.527392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:25.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422122+0000","last_change":"2024-03-29T09:08:56.912120+0000","last_active":"2024-03-29T09:09:01.422122+0000","last_peered":"2024-03-29T09:09:01.422122+0000","last_clean":"2024-03-29T09:09:01.422122+0000","last_became_active":"2024-03-29T09:08:56.911834+0000","last_became_peered":"2024-03-29T09:08:56.911834+0000","last_unstale":"2024-03-29T09:09:01.422122+0000","last_undegraded":"2024-03-29T09:09:01.422122+0000","last_fullsized":"2024-03-29T09:09:01.422122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527406+0000","last_change":"2024-03-29T09:08:57.413619+0000","last_active":"2024-03-29T09:09:01.527406+0000","last_peered":"2024-03-29T09:09:01.527406+0000","last_clean":"2024-03-29T09:09:01.527406+0000","last_became_active":"2024-03-29T09:08:57.409612+0000","last_became_peered":"2024-03-29T09:08:57.409612+0000","last_unstale":"2024-03-29T09:09:01.527406+0000","last_undegraded":"2024-03-29T09:09:01.527406+0000","last_fullsized":"2024-03-29T09:09:01.527406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533989+0000","last_change":"2024-03-29T09:09:01.436295+0000","last_active":"2024-03-29T09:09:01.533989+0000","last_peered":"2024-03-29T09:09:01.533989+0000","last_clean":"2024-03-29T09:09:01.533989+0000","last_became_active":"2024-03-29T09:09:01.435972+0000","last_became_peered":"2024-03-29T09:09:01.435972+0000","last_unstale":"2024-03-29T09:09:01.533989+0000","last_undegraded":"2024-03-29T09:09:01.533989+0000","last_fullsized":"2024-03-29T09:09:01.533989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429481+0000","last_change":"2024-03-29T09:08:56.907160+0000","last_active":"2024-03-29T09:09:01.429481+0000","last_peered":"2024-03-29T09:09:01.429481+0000","last_clean":"2024-03-29T09:09:01.429481+0000","last_became_active":"2024-03-29T09:08:56.906801+0000","last_became_peered":"2024-03-29T09:08:56.906801+0000","last_unstale":"2024-03-29T09:09:01.429481+0000","last_undegraded":"2024-03-29T09:09:01.429481+0000","last_fullsized":"2024-03-29T09:09:01.429481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527323+0000","last_change":"2024-03-29T09:08:57.425389+0000","last_active":"2024-03-29T09:09:01.527323+0000","last_peered":"2024-03-29T09:09:01.527323+0000","last_clean":"2024-03-29T09:09:01.527323+0000","last_became_active":"2024-03-29T09:08:57.425127+0000","last_became_peered":"2024-03-29T09:08:57.425127+0000","last_unstale":"2024-03-29T09:09:01.527323+0000","last_undegraded":"2024-03-29T09:09:01.527323+0000","last_fullsized":"2024-03-29T09:09:01.527323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527793+0000","last_change":"2024-03-29T09:09:01.432980+0000","last_active":"2024-03-29T09:09:01.527793+0000","last_peered":"2024-03-29T09:09:01.527793+0000","last_clean":"2024-03-29T09:09:01.527793+0000","last_became_active":"2024-03-29T09:09:01.432597+0000","last_became_peered":"2024-03-29T09:09:01.432597+0000","last_unstale":"2024-03-29T09:09:01.527793+0000","last_undegraded":"2024-03-29T09:09:01.527793+0000","last_fullsized":"2024-03-29T09:09:01.527793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:52.143266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414584+0000","last_change":"2024-03-29T09:08:56.901765+0000","last_active":"2024-03-29T09:09:01.414584+0000","last_peered":"2024-03-29T09:09:01.414584+0000","last_clean":"2024-03-29T09:09:01.414584+0000","last_became_active":"2024-03-29T09:08:56.901513+0000","last_became_peered":"2024-03-29T09:08:56.901513+0000","last_unstale":"2024-03-29T09:09:01.414584+0000","last_undegraded":"2024-03-29T09:09:01.414584+0000","last_fullsized":"2024-03-29T09:09:01.414584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:55.597965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527664+0000","last_change":"2024-03-29T09:08:57.406785+0000","last_active":"2024-03-29T09:09:01.527664+0000","last_peered":"2024-03-29T09:09:01.527664+0000","last_clean":"2024-03-29T09:09:01.527664+0000","last_became_active":"2024-03-29T09:08:57.406261+0000","last_became_peered":"2024-03-29T09:08:57.406261+0000","last_unstale":"2024-03-29T09:09:01.527664+0000","last_undegraded":"2024-03-29T09:09:01.527664+0000","last_fullsized":"2024-03-29T09:09:01.527664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:09:43.625475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530845+0000","last_change":"2024-03-29T09:09:01.445228+0000","last_active":"2024-03-29T09:09:01.530845+0000","last_peered":"2024-03-29T09:09:01.530845+0000","last_clean":"2024-03-29T09:09:01.530845+0000","last_became_active":"2024-03-29T09:09:01.444917+0000","last_became_peered":"2024-03-29T09:09:01.444917+0000","last_unstale":"2024-03-29T09:09:01.530845+0000","last_undegraded":"2024-03-29T09:09:01.530845+0000","last_fullsized":"2024-03-29T09:09:01.530845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.431916+0000","last_change":"2024-03-29T09:08:56.914015+0000","last_active":"2024-03-29T09:09:01.431916+0000","last_peered":"2024-03-29T09:09:01.431916+0000","last_clean":"2024-03-29T09:09:01.431916+0000","last_became_active":"2024-03-29T09:08:56.913673+0000","last_became_peered":"2024-03-29T09:08:56.913673+0000","last_unstale":"2024-03-29T09:09:01.431916+0000","last_undegraded":"2024-03-29T09:09:01.431916+0000","last_fullsized":"2024-03-29T09:09:01.431916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527811+0000","last_change":"2024-03-29T09:08:57.410999+0000","last_active":"2024-03-29T09:09:01.527811+0000","last_peered":"2024-03-29T09:09:01.527811+0000","last_clean":"2024-03-29T09:09:01.527811+0000","last_became_active":"2024-03-29T09:08:57.410742+0000","last_became_peered":"2024-03-29T09:08:57.410742+0000","last_unstale":"2024-03-29T09:09:01.527811+0000","last_undegraded":"2024-03-29T09:09:01.527811+0000","last_fullsized":"2024-03-29T09:09:01.527811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:24.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530767+0000","last_change":"2024-03-29T09:09:01.434584+0000","last_active":"2024-03-29T09:09:01.530767+0000","last_peered":"2024-03-29T09:09:01.530767+0000","last_clean":"2024-03-29T09:09:01.530767+0000","last_became_active":"2024-03-29T09:09:01.434233+0000","last_became_peered":"2024-03-29T09:09:01.434233+0000","last_unstale":"2024-03-29T09:09:01.530767+0000","last_undegraded":"2024-03-29T09:09:01.530767+0000","last_fullsized":"2024-03-29T09:09:01.530767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:18:14.395090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420352+0000","last_change":"2024-03-29T09:08:56.908753+0000","last_active":"2024-03-29T09:09:01.420352+0000","last_peered":"2024-03-29T09:09:01.420352+0000","last_clean":"2024-03-29T09:09:01.420352+0000","last_became_active":"2024-03-29T09:08:56.908505+0000","last_became_peered":"2024-03-29T09:08:56.908505+0000","last_unstale":"2024-03-29T09:09:01.420352+0000","last_undegraded":"2024-03-29T09:09:01.420352+0000","last_fullsized":"2024-03-29T09:09:01.420352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533999+0000","last_change":"2024-03-29T09:08:57.418006+0000","last_active":"2024-03-29T09:09:01.533999+0000","last_peered":"2024-03-29T09:09:01.533999+0000","last_clean":"2024-03-29T09:09:01.533999+0000","last_became_active":"2024-03-29T09:08:57.417644+0000","last_became_peered":"2024-03-29T09:08:57.417644+0000","last_unstale":"2024-03-29T09:09:01.533999+0000","last_undegraded":"2024-03-29T09:09:01.533999+0000","last_fullsized":"2024-03-29T09:09:01.533999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527566+0000","last_change":"2024-03-29T09:09:01.431602+0000","last_active":"2024-03-29T09:09:01.527566+0000","last_peered":"2024-03-29T09:09:01.527566+0000","last_clean":"2024-03-29T09:09:01.527566+0000","last_became_active":"2024-03-29T09:09:01.430422+0000","last_became_peered":"2024-03-29T09:09:01.430422+0000","last_unstale":"2024-03-29T09:09:01.527566+0000","last_undegraded":"2024-03-29T09:09:01.527566+0000","last_fullsized":"2024-03-29T09:09:01.527566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:21:33.289630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429605+0000","last_change":"2024-03-29T09:08:56.907602+0000","last_active":"2024-03-29T09:09:01.429605+0000","last_peered":"2024-03-29T09:09:01.429605+0000","last_clean":"2024-03-29T09:09:01.429605+0000","last_became_active":"2024-03-29T09:08:56.907321+0000","last_became_peered":"2024-03-29T09:08:56.907321+0000","last_unstale":"2024-03-29T09:09:01.429605+0000","last_undegraded":"2024-03-29T09:09:01.429605+0000","last_fullsized":"2024-03-29T09:09:01.429605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534106+0000","last_change":"2024-03-29T09:08:57.419001+0000","last_active":"2024-03-29T09:09:01.534106+0000","last_peered":"2024-03-29T09:09:01.534106+0000","last_clean":"2024-03-29T09:09:01.534106+0000","last_became_active":"2024-03-29T09:08:57.418769+0000","last_became_peered":"2024-03-29T09:08:57.418769+0000","last_unstale":"2024-03-29T09:09:01.534106+0000","last_undegraded":"2024-03-29T09:09:01.534106+0000","last_fullsized":"2024-03-29T09:09:01.534106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530742+0000","last_change":"2024-03-29T09:08:57.408417+0000","last_active":"2024-03-29T09:09:01.530742+0000","last_peered":"2024-03-29T09:09:01.530742+0000","last_clean":"2024-03-29T09:09:01.530742+0000","last_became_active":"2024-03-29T09:08:57.408131+0000","last_became_peered":"2024-03-29T09:08:57.408131+0000","last_unstale":"2024-03-29T09:09:01.530742+0000","last_undegraded":"2024-03-29T09:09:01.530742+0000","last_fullsized":"2024-03-29T09:09:01.530742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527306+0000","last_change":"2024-03-29T09:08:57.418220+0000","last_active":"2024-03-29T09:09:01.527306+0000","last_peered":"2024-03-29T09:09:01.527306+0000","last_clean":"2024-03-29T09:09:01.527306+0000","last_became_active":"2024-03-29T09:08:57.417900+0000","last_became_peered":"2024-03-29T09:08:57.417900+0000","last_unstale":"2024-03-29T09:09:01.527306+0000","last_undegraded":"2024-03-29T09:09:01.527306+0000","last_fullsized":"2024-03-29T09:09:01.527306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527708+0000","last_change":"2024-03-29T09:08:57.424647+0000","last_active":"2024-03-29T09:09:01.527708+0000","last_peered":"2024-03-29T09:09:01.527708+0000","last_clean":"2024-03-29T09:09:01.527708+0000","last_became_active":"2024-03-29T09:08:57.424404+0000","last_became_peered":"2024-03-29T09:08:57.424404+0000","last_unstale":"2024-03-29T09:09:01.527708+0000","last_undegraded":"2024-03-29T09:09:01.527708+0000","last_fullsized":"2024-03-29T09:09:01.527708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:06:29.651626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530787+0000","last_change":"2024-03-29T09:08:57.419276+0000","last_active":"2024-03-29T09:09:01.530787+0000","last_peered":"2024-03-29T09:09:01.530787+0000","last_clean":"2024-03-29T09:09:01.530787+0000","last_became_active":"2024-03-29T09:08:57.418974+0000","last_became_peered":"2024-03-29T09:08:57.418974+0000","last_unstale":"2024-03-29T09:09:01.530787+0000","last_undegraded":"2024-03-29T09:09:01.530787+0000","last_fullsized":"2024-03-29T09:09:01.530787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525521+0000","last_change":"2024-03-29T09:08:57.429156+0000","last_active":"2024-03-29T09:09:01.525521+0000","last_peered":"2024-03-29T09:09:01.525521+0000","last_clean":"2024-03-29T09:09:01.525521+0000","last_became_active":"2024-03-29T09:08:57.428873+0000","last_became_peered":"2024-03-29T09:08:57.428873+0000","last_unstale":"2024-03-29T09:09:01.525521+0000","last_undegraded":"2024-03-29T09:09:01.525521+0000","last_fullsized":"2024-03-29T09:09:01.525521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525556+0000","last_change":"2024-03-29T09:08:57.423650+0000","last_active":"2024-03-29T09:09:01.525556+0000","last_peered":"2024-03-29T09:09:01.525556+0000","last_clean":"2024-03-29T09:09:01.525556+0000","last_became_active":"2024-03-29T09:08:57.423385+0000","last_became_peered":"2024-03-29T09:08:57.423385+0000","last_unstale":"2024-03-29T09:09:01.525556+0000","last_undegraded":"2024-03-29T09:09:01.525556+0000","last_fullsized":"2024-03-29T09:09:01.525556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531076+0000","last_change":"2024-03-29T09:08:57.413866+0000","last_active":"2024-03-29T09:09:01.531076+0000","last_peered":"2024-03-29T09:09:01.531076+0000","last_clean":"2024-03-29T09:09:01.531076+0000","last_became_active":"2024-03-29T09:08:57.413483+0000","last_became_peered":"2024-03-29T09:08:57.413483+0000","last_unstale":"2024-03-29T09:09:01.531076+0000","last_undegraded":"2024-03-29T09:09:01.531076+0000","last_fullsized":"2024-03-29T09:09:01.531076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530897+0000","last_change":"2024-03-29T09:08:57.422855+0000","last_active":"2024-03-29T09:09:01.530897+0000","last_peered":"2024-03-29T09:09:01.530897+0000","last_clean":"2024-03-29T09:09:01.530897+0000","last_became_active":"2024-03-29T09:08:57.422350+0000","last_became_peered":"2024-03-29T09:08:57.422350+0000","last_unstale":"2024-03-29T09:09:01.530897+0000","last_undegraded":"2024-03-29T09:09:01.530897+0000","last_fullsized":"2024-03-29T09:09:01.530897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527637+0000","last_change":"2024-03-29T09:08:57.427109+0000","last_active":"2024-03-29T09:09:01.527637+0000","last_peered":"2024-03-29T09:09:01.527637+0000","last_clean":"2024-03-29T09:09:01.527637+0000","last_became_active":"2024-03-29T09:08:57.424525+0000","last_became_peered":"2024-03-29T09:08:57.424525+0000","last_unstale":"2024-03-29T09:09:01.527637+0000","last_undegraded":"2024-03-29T09:09:01.527637+0000","last_fullsized":"2024-03-29T09:09:01.527637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:52.774904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530755+0000","last_change":"2024-03-29T09:08:57.416922+0000","last_active":"2024-03-29T09:09:01.530755+0000","last_peered":"2024-03-29T09:09:01.530755+0000","last_clean":"2024-03-29T09:09:01.530755+0000","last_became_active":"2024-03-29T09:08:57.415900+0000","last_became_peered":"2024-03-29T09:08:57.415900+0000","last_unstale":"2024-03-29T09:09:01.530755+0000","last_undegraded":"2024-03-29T09:09:01.530755+0000","last_fullsized":"2024-03-29T09:09:01.530755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528724+0000","last_change":"2024-03-29T09:08:57.416177+0000","last_active":"2024-03-29T09:09:01.528724+0000","last_peered":"2024-03-29T09:09:01.528724+0000","last_clean":"2024-03-29T09:09:01.528724+0000","last_became_active":"2024-03-29T09:08:57.415844+0000","last_became_peered":"2024-03-29T09:08:57.415844+0000","last_unstale":"2024-03-29T09:09:01.528724+0000","last_undegraded":"2024-03-29T09:09:01.528724+0000","last_fullsized":"2024-03-29T09:09:01.528724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527479+0000","last_change":"2024-03-29T09:08:57.411579+0000","last_active":"2024-03-29T09:09:01.527479+0000","last_peered":"2024-03-29T09:09:01.527479+0000","last_clean":"2024-03-29T09:09:01.527479+0000","last_became_active":"2024-03-29T09:08:57.411297+0000","last_became_peered":"2024-03-29T09:08:57.411297+0000","last_unstale":"2024-03-29T09:09:01.527479+0000","last_undegraded":"2024-03-29T09:09:01.527479+0000","last_fullsized":"2024-03-29T09:09:01.527479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:13.990570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534013+0000","last_change":"2024-03-29T09:08:57.417300+0000","last_active":"2024-03-29T09:09:01.534013+0000","last_peered":"2024-03-29T09:09:01.534013+0000","last_clean":"2024-03-29T09:09:01.534013+0000","last_became_active":"2024-03-29T09:08:57.416733+0000","last_became_peered":"2024-03-29T09:08:57.416733+0000","last_unstale":"2024-03-29T09:09:01.534013+0000","last_undegraded":"2024-03-29T09:09:01.534013+0000","last_fullsized":"2024-03-29T09:09:01.534013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:51.863125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":690226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:09:31.982 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:09:32.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.208+0000 7f7d7d600700 1 Processor -- start 2024-03-29T09:09:32.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.208+0000 7f7d7d600700 1 -- start start 2024-03-29T09:09:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d7d600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7d781608e0 0x7f7d780d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d7d600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d780d1b20 0x7f7d780ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d7d600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d780cddc0 con 0x7f7d78160cc0 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d7d600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d780ce0a0 con 0x7f7d781608e0 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d7d600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d780ce380 con 0x7f7d780d1b20 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d76a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7d781608e0 0x7f7d780d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d76000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7d78160cc0 0x7f7d780d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49730/0 (socket says 172.21.15.99:49730) 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.209+0000 7f7d76000700 1 -- 172.21.15.99:0/1219827535 learned_addr learned my addr 172.21.15.99:0/1219827535 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d77400700 1 --2- 172.21.15.99:0/1219827535 >> v2:172.21.15.133:3300/0 conn(0x7f7d780d1b20 0x7f7d780ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d76a00700 1 -- 172.21.15.99:0/1219827535 >> v2:172.21.15.133:3300/0 conn(0x7f7d780d1b20 msgr2=0x7f7d780ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:32.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d76a00700 1 --2- 172.21.15.99:0/1219827535 >> v2:172.21.15.133:3300/0 conn(0x7f7d780d1b20 0x7f7d780ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d76a00700 1 -- 172.21.15.99:0/1219827535 >> v1:172.21.15.99:6789/0 conn(0x7f7d78160cc0 legacy=0x7f7d780d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d76a00700 1 -- 172.21.15.99:0/1219827535 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780ce660 con 0x7f7d781608e0 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d75600700 1 -- 172.21.15.99:0/1219827535 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313303174 0 0) 0x7f7d780cddc0 con 0x7f7d78160cc0 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d76a00700 1 --2- 172.21.15.99:0/1219827535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d781608e0 0x7f7d780d0150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d7809cb70 tx=0x7f7d68009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73ec49756c1c792 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d75600700 1 -- 172.21.15.99:0/1219827535 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68011830 con 0x7f7d781608e0 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d75600700 1 -- 172.21.15.99:0/1219827535 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d68011990 con 0x7f7d781608e0 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.210+0000 7f7d75600700 1 -- 172.21.15.99:0/1219827535 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68011d20 con 0x7f7d781608e0 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 -- 172.21.15.99:0/1219827535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d781608e0 msgr2=0x7f7d780d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 --2- 172.21.15.99:0/1219827535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d781608e0 0x7f7d780d0150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7d7809cb70 tx=0x7f7d68009b20 comp rx=0 tx=0).stop 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 -- 172.21.15.99:0/1219827535 shutdown_connections 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 --2- 172.21.15.99:0/1219827535 >> v2:172.21.15.133:3300/0 conn(0x7f7d780d1b20 0x7f7d780ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 --2- 172.21.15.99:0/1219827535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d781608e0 0x7f7d780d0150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 -- 172.21.15.99:0/1219827535 >> 172.21.15.99:0/1219827535 conn(0x7f7d7809c490 msgr2=0x7f7d780d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:32.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 -- 172.21.15.99:0/1219827535 shutdown_connections 2024-03-29T09:09:32.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.211+0000 7f7d7d600700 1 -- 172.21.15.99:0/1219827535 wait complete. 2024-03-29T09:09:32.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.213+0000 7f7d7d600700 1 Processor -- start 2024-03-29T09:09:32.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.213+0000 7f7d7d600700 1 -- start start 2024-03-29T09:09:32.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d7d600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d780d1b20 0x7f7d781559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d7d600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d78160cc0 0x7f7d78156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d7d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d78156e80 0x7f7d7815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d7d600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d78157310 con 0x7f7d78156e80 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d7d600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d781f43d0 con 0x7f7d780d1b20 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d7d600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7815c370 con 0x7f7d78160cc0 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d76a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d780d1b20 0x7f7d781559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d77400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d78156e80 0x7f7d7815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d76a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d780d1b20 0x7f7d781559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:55550/0 (socket says 172.21.15.99:55550) 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d76a00700 1 -- 172.21.15.99:0/2254846985 learned_addr learned my addr 172.21.15.99:0/2254846985 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.214+0000 7f7d76000700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d78160cc0 0x7f7d78156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d77400700 1 -- 172.21.15.99:0/2254846985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d78160cc0 msgr2=0x7f7d78156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d77400700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d78160cc0 0x7f7d78156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d77400700 1 -- 172.21.15.99:0/2254846985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d780d1b20 msgr2=0x7f7d781559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d77400700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d780d1b20 0x7f7d781559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d77400700 1 -- 172.21.15.99:0/2254846985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680094a0 con 0x7f7d78156e80 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d77400700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d78156e80 0x7f7d7815ad20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d70007d90 tx=0x7f7d7000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:32.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d6ea00700 1 -- 172.21.15.99:0/2254846985 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70018070 con 0x7f7d78156e80 2024-03-29T09:09:32.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d6ea00700 1 -- 172.21.15.99:0/2254846985 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7000eb30 con 0x7f7d78156e80 2024-03-29T09:09:32.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7815bad0 con 0x7f7d78156e80 2024-03-29T09:09:32.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d6ea00700 1 -- 172.21.15.99:0/2254846985 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70003a20 con 0x7f7d78156e80 2024-03-29T09:09:32.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.215+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7815c090 con 0x7f7d78156e80 2024-03-29T09:09:32.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.216+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d781f4540 con 0x7f7d78156e80 2024-03-29T09:09:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.217+0000 7f7d6ea00700 1 -- 172.21.15.99:0/2254846985 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7d70014420 con 0x7f7d78156e80 2024-03-29T09:09:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.218+0000 7f7d6ea00700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7d5806ac90 0x7f7d5806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.218+0000 7f7d6ea00700 1 -- 172.21.15.99:0/2254846985 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7d70089590 con 0x7f7d78156e80 2024-03-29T09:09:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.218+0000 7f7d76a00700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7d5806ac90 0x7f7d5806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.219+0000 7f7d76a00700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7d5806ac90 0x7f7d5806d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009fd0 tx=0x7f7d68008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:32.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.222+0000 7f7d6ea00700 1 -- 172.21.15.99:0/2254846985 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7d700543e0 con 0x7f7d78156e80 2024-03-29T09:09:32.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.420+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d780b22a0 con 0x7f7d5806ac90 2024-03-29T09:09:32.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.430+0000 7f7d6ea00700 1 -- 172.21.15.99:0/2254846985 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345298 (secure 0 0 0) 0x7f7d68011630 con 0x7f7d5806ac90 2024-03-29T09:09:32.434 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7d5806ac90 msgr2=0x7f7d5806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7d5806ac90 0x7f7d5806d150 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009fd0 tx=0x7f7d68008030 comp rx=0 tx=0).stop 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d78156e80 msgr2=0x7f7d7815ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d78156e80 0x7f7d7815ad20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7d70007d90 tx=0x7f7d7000e3c0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 shutdown_connections 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7d5806ac90 0x7f7d5806d150 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7d78156e80 0x7f7d7815ad20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d780d1b20 0x7f7d781559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 --2- 172.21.15.99:0/2254846985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d78160cc0 0x7f7d78156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 >> 172.21.15.99:0/2254846985 conn(0x7f7d7809c490 msgr2=0x7f7d780b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:32.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 shutdown_connections 2024-03-29T09:09:32.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.435+0000 7f7d7d600700 1 -- 172.21.15.99:0/2254846985 wait complete. 2024-03-29T09:09:32.438 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:09:32.452 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-03-29T09:09:30.458109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594962,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219392,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755328,"statfs":{"total":773094113280,"available":772869455872,"internally_reserved":0,"allocated":3284992,"data_stored":1988228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004650"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534081+0000","last_change":"2024-03-29T09:08:57.419769+0000","last_active":"2024-03-29T09:09:01.534081+0000","last_peered":"2024-03-29T09:09:01.534081+0000","last_clean":"2024-03-29T09:09:01.534081+0000","last_became_active":"2024-03-29T09:08:57.419455+0000","last_became_peered":"2024-03-29T09:08:57.419455+0000","last_unstale":"2024-03-29T09:09:01.534081+0000","last_undegraded":"2024-03-29T09:09:01.534081+0000","last_fullsized":"2024-03-29T09:09:01.534081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530834+0000","last_change":"2024-03-29T09:08:57.427159+0000","last_active":"2024-03-29T09:09:01.530834+0000","last_peered":"2024-03-29T09:09:01.530834+0000","last_clean":"2024-03-29T09:09:01.530834+0000","last_became_active":"2024-03-29T09:08:57.426926+0000","last_became_peered":"2024-03-29T09:08:57.426926+0000","last_unstale":"2024-03-29T09:09:01.530834+0000","last_undegraded":"2024-03-29T09:09:01.530834+0000","last_fullsized":"2024-03-29T09:09:01.530834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:58.444421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:08:57.419169+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:08:57.418824+0000","last_became_peered":"2024-03-29T09:08:57.418824+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530961+0000","last_change":"2024-03-29T09:08:57.409236+0000","last_active":"2024-03-29T09:09:01.530961+0000","last_peered":"2024-03-29T09:09:01.530961+0000","last_clean":"2024-03-29T09:09:01.530961+0000","last_became_active":"2024-03-29T09:08:57.408981+0000","last_became_peered":"2024-03-29T09:08:57.408981+0000","last_unstale":"2024-03-29T09:09:01.530961+0000","last_undegraded":"2024-03-29T09:09:01.530961+0000","last_fullsized":"2024-03-29T09:09:01.530961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533948+0000","last_change":"2024-03-29T09:08:57.423360+0000","last_active":"2024-03-29T09:09:01.533948+0000","last_peered":"2024-03-29T09:09:01.533948+0000","last_clean":"2024-03-29T09:09:01.533948+0000","last_became_active":"2024-03-29T09:08:57.423088+0000","last_became_peered":"2024-03-29T09:08:57.423088+0000","last_unstale":"2024-03-29T09:09:01.533948+0000","last_undegraded":"2024-03-29T09:09:01.533948+0000","last_fullsized":"2024-03-29T09:09:01.533948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530887+0000","last_change":"2024-03-29T09:08:57.415603+0000","last_active":"2024-03-29T09:09:01.530887+0000","last_peered":"2024-03-29T09:09:01.530887+0000","last_clean":"2024-03-29T09:09:01.530887+0000","last_became_active":"2024-03-29T09:08:57.413227+0000","last_became_peered":"2024-03-29T09:08:57.413227+0000","last_unstale":"2024-03-29T09:09:01.530887+0000","last_undegraded":"2024-03-29T09:09:01.530887+0000","last_fullsized":"2024-03-29T09:09:01.530887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533929+0000","last_change":"2024-03-29T09:08:57.421465+0000","last_active":"2024-03-29T09:09:01.533929+0000","last_peered":"2024-03-29T09:09:01.533929+0000","last_clean":"2024-03-29T09:09:01.533929+0000","last_became_active":"2024-03-29T09:08:57.421189+0000","last_became_peered":"2024-03-29T09:08:57.421189+0000","last_unstale":"2024-03-29T09:09:01.533929+0000","last_undegraded":"2024-03-29T09:09:01.533929+0000","last_fullsized":"2024-03-29T09:09:01.533929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530711+0000","last_change":"2024-03-29T09:08:57.430900+0000","last_active":"2024-03-29T09:09:01.530711+0000","last_peered":"2024-03-29T09:09:01.530711+0000","last_clean":"2024-03-29T09:09:01.530711+0000","last_became_active":"2024-03-29T09:08:57.430636+0000","last_became_peered":"2024-03-29T09:08:57.430636+0000","last_unstale":"2024-03-29T09:09:01.530711+0000","last_undegraded":"2024-03-29T09:09:01.530711+0000","last_fullsized":"2024-03-29T09:09:01.530711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528748+0000","last_change":"2024-03-29T09:08:57.413603+0000","last_active":"2024-03-29T09:09:01.528748+0000","last_peered":"2024-03-29T09:09:01.528748+0000","last_clean":"2024-03-29T09:09:01.528748+0000","last_became_active":"2024-03-29T09:08:57.413373+0000","last_became_peered":"2024-03-29T09:08:57.413373+0000","last_unstale":"2024-03-29T09:09:01.528748+0000","last_undegraded":"2024-03-29T09:09:01.528748+0000","last_fullsized":"2024-03-29T09:09:01.528748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534071+0000","last_change":"2024-03-29T09:08:57.417036+0000","last_active":"2024-03-29T09:09:01.534071+0000","last_peered":"2024-03-29T09:09:01.534071+0000","last_clean":"2024-03-29T09:09:01.534071+0000","last_became_active":"2024-03-29T09:08:57.416762+0000","last_became_peered":"2024-03-29T09:08:57.416762+0000","last_unstale":"2024-03-29T09:09:01.534071+0000","last_undegraded":"2024-03-29T09:09:01.534071+0000","last_fullsized":"2024-03-29T09:09:01.534071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534055+0000","last_change":"2024-03-29T09:08:57.417500+0000","last_active":"2024-03-29T09:09:01.534055+0000","last_peered":"2024-03-29T09:09:01.534055+0000","last_clean":"2024-03-29T09:09:01.534055+0000","last_became_active":"2024-03-29T09:08:57.417187+0000","last_became_peered":"2024-03-29T09:08:57.417187+0000","last_unstale":"2024-03-29T09:09:01.534055+0000","last_undegraded":"2024-03-29T09:09:01.534055+0000","last_fullsized":"2024-03-29T09:09:01.534055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534022+0000","last_change":"2024-03-29T09:08:57.416133+0000","last_active":"2024-03-29T09:09:01.534022+0000","last_peered":"2024-03-29T09:09:01.534022+0000","last_clean":"2024-03-29T09:09:01.534022+0000","last_became_active":"2024-03-29T09:08:57.415878+0000","last_became_peered":"2024-03-29T09:08:57.415878+0000","last_unstale":"2024-03-29T09:09:01.534022+0000","last_undegraded":"2024-03-29T09:09:01.534022+0000","last_fullsized":"2024-03-29T09:09:01.534022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525479+0000","last_change":"2024-03-29T09:08:57.418824+0000","last_active":"2024-03-29T09:09:01.525479+0000","last_peered":"2024-03-29T09:09:01.525479+0000","last_clean":"2024-03-29T09:09:01.525479+0000","last_became_active":"2024-03-29T09:08:57.418534+0000","last_became_peered":"2024-03-29T09:08:57.418534+0000","last_unstale":"2024-03-29T09:09:01.525479+0000","last_undegraded":"2024-03-29T09:09:01.525479+0000","last_fullsized":"2024-03-29T09:09:01.525479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:17:11.207261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525458+0000","last_change":"2024-03-29T09:08:57.412768+0000","last_active":"2024-03-29T09:09:01.525458+0000","last_peered":"2024-03-29T09:09:01.525458+0000","last_clean":"2024-03-29T09:09:01.525458+0000","last_became_active":"2024-03-29T09:08:57.412077+0000","last_became_peered":"2024-03-29T09:08:57.412077+0000","last_unstale":"2024-03-29T09:09:01.525458+0000","last_undegraded":"2024-03-29T09:09:01.525458+0000","last_fullsized":"2024-03-29T09:09:01.525458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:33.935343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530745+0000","last_change":"2024-03-29T09:08:57.430532+0000","last_active":"2024-03-29T09:09:01.530745+0000","last_peered":"2024-03-29T09:09:01.530745+0000","last_clean":"2024-03-29T09:09:01.530745+0000","last_became_active":"2024-03-29T09:08:57.430163+0000","last_became_peered":"2024-03-29T09:08:57.430163+0000","last_unstale":"2024-03-29T09:09:01.530745+0000","last_undegraded":"2024-03-29T09:09:01.530745+0000","last_fullsized":"2024-03-29T09:09:01.530745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527616+0000","last_change":"2024-03-29T09:08:57.413899+0000","last_active":"2024-03-29T09:09:01.527616+0000","last_peered":"2024-03-29T09:09:01.527616+0000","last_clean":"2024-03-29T09:09:01.527616+0000","last_became_active":"2024-03-29T09:08:57.413637+0000","last_became_peered":"2024-03-29T09:08:57.413637+0000","last_unstale":"2024-03-29T09:09:01.527616+0000","last_undegraded":"2024-03-29T09:09:01.527616+0000","last_fullsized":"2024-03-29T09:09:01.527616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:32.639943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527925+0000","last_change":"2024-03-29T09:08:57.418000+0000","last_active":"2024-03-29T09:09:01.527925+0000","last_peered":"2024-03-29T09:09:01.527925+0000","last_clean":"2024-03-29T09:09:01.527925+0000","last_became_active":"2024-03-29T09:08:57.417649+0000","last_became_peered":"2024-03-29T09:08:57.417649+0000","last_unstale":"2024-03-29T09:09:01.527925+0000","last_undegraded":"2024-03-29T09:09:01.527925+0000","last_fullsized":"2024-03-29T09:09:01.527925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:06.479973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527377+0000","last_change":"2024-03-29T09:08:57.430547+0000","last_active":"2024-03-29T09:09:01.527377+0000","last_peered":"2024-03-29T09:09:01.527377+0000","last_clean":"2024-03-29T09:09:01.527377+0000","last_became_active":"2024-03-29T09:08:57.430013+0000","last_became_peered":"2024-03-29T09:08:57.430013+0000","last_unstale":"2024-03-29T09:09:01.527377+0000","last_undegraded":"2024-03-29T09:09:01.527377+0000","last_fullsized":"2024-03-29T09:09:01.527377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527878+0000","last_change":"2024-03-29T09:08:57.419008+0000","last_active":"2024-03-29T09:09:01.527878+0000","last_peered":"2024-03-29T09:09:01.527878+0000","last_clean":"2024-03-29T09:09:01.527878+0000","last_became_active":"2024-03-29T09:08:57.418723+0000","last_became_peered":"2024-03-29T09:08:57.418723+0000","last_unstale":"2024-03-29T09:09:01.527878+0000","last_undegraded":"2024-03-29T09:09:01.527878+0000","last_fullsized":"2024-03-29T09:09:01.527878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:06.268267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527289+0000","last_change":"2024-03-29T09:09:01.448984+0000","last_active":"2024-03-29T09:09:01.527289+0000","last_peered":"2024-03-29T09:09:01.527289+0000","last_clean":"2024-03-29T09:09:01.527289+0000","last_became_active":"2024-03-29T09:09:01.446825+0000","last_became_peered":"2024-03-29T09:09:01.446825+0000","last_unstale":"2024-03-29T09:09:01.527289+0000","last_undegraded":"2024-03-29T09:09:01.527289+0000","last_fullsized":"2024-03-29T09:09:01.527289+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:55.385731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.425281+0000","last_change":"2024-03-29T09:08:56.898980+0000","last_active":"2024-03-29T09:09:01.425281+0000","last_peered":"2024-03-29T09:09:01.425281+0000","last_clean":"2024-03-29T09:09:01.425281+0000","last_became_active":"2024-03-29T09:08:56.898646+0000","last_became_peered":"2024-03-29T09:08:56.898646+0000","last_unstale":"2024-03-29T09:09:01.425281+0000","last_undegraded":"2024-03-29T09:09:01.425281+0000","last_fullsized":"2024-03-29T09:09:01.425281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530854+0000","last_change":"2024-03-29T09:08:57.420574+0000","last_active":"2024-03-29T09:09:01.530854+0000","last_peered":"2024-03-29T09:09:01.530854+0000","last_clean":"2024-03-29T09:09:01.530854+0000","last_became_active":"2024-03-29T09:08:57.420318+0000","last_became_peered":"2024-03-29T09:08:57.420318+0000","last_unstale":"2024-03-29T09:09:01.530854+0000","last_undegraded":"2024-03-29T09:09:01.530854+0000","last_fullsized":"2024-03-29T09:09:01.530854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:21.872230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530783+0000","last_change":"2024-03-29T09:09:01.444440+0000","last_active":"2024-03-29T09:09:01.530783+0000","last_peered":"2024-03-29T09:09:01.530783+0000","last_clean":"2024-03-29T09:09:01.530783+0000","last_became_active":"2024-03-29T09:09:01.444026+0000","last_became_peered":"2024-03-29T09:09:01.444026+0000","last_unstale":"2024-03-29T09:09:01.530783+0000","last_undegraded":"2024-03-29T09:09:01.530783+0000","last_fullsized":"2024-03-29T09:09:01.530783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.421739+0000","last_change":"2024-03-29T09:08:56.908349+0000","last_active":"2024-03-29T09:09:01.421739+0000","last_peered":"2024-03-29T09:09:01.421739+0000","last_clean":"2024-03-29T09:09:01.421739+0000","last_became_active":"2024-03-29T09:08:56.908070+0000","last_became_peered":"2024-03-29T09:08:56.908070+0000","last_unstale":"2024-03-29T09:09:01.421739+0000","last_undegraded":"2024-03-29T09:09:01.421739+0000","last_fullsized":"2024-03-29T09:09:01.421739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530920+0000","last_change":"2024-03-29T09:08:57.422633+0000","last_active":"2024-03-29T09:09:01.530920+0000","last_peered":"2024-03-29T09:09:01.530920+0000","last_clean":"2024-03-29T09:09:01.530920+0000","last_became_active":"2024-03-29T09:08:57.422298+0000","last_became_peered":"2024-03-29T09:08:57.422298+0000","last_unstale":"2024-03-29T09:09:01.530920+0000","last_undegraded":"2024-03-29T09:09:01.530920+0000","last_fullsized":"2024-03-29T09:09:01.530920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:45:59.812541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530925+0000","last_change":"2024-03-29T09:09:01.435391+0000","last_active":"2024-03-29T09:09:01.530925+0000","last_peered":"2024-03-29T09:09:01.530925+0000","last_clean":"2024-03-29T09:09:01.530925+0000","last_became_active":"2024-03-29T09:09:01.435087+0000","last_became_peered":"2024-03-29T09:09:01.435087+0000","last_unstale":"2024-03-29T09:09:01.530925+0000","last_undegraded":"2024-03-29T09:09:01.530925+0000","last_fullsized":"2024-03-29T09:09:01.530925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424816+0000","last_change":"2024-03-29T09:08:56.910805+0000","last_active":"2024-03-29T09:09:01.424816+0000","last_peered":"2024-03-29T09:09:01.424816+0000","last_clean":"2024-03-29T09:09:01.424816+0000","last_became_active":"2024-03-29T09:08:56.908213+0000","last_became_peered":"2024-03-29T09:08:56.908213+0000","last_unstale":"2024-03-29T09:09:01.424816+0000","last_undegraded":"2024-03-29T09:09:01.424816+0000","last_fullsized":"2024-03-29T09:09:01.424816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:45.121451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528733+0000","last_change":"2024-03-29T09:08:57.420307+0000","last_active":"2024-03-29T09:09:01.528733+0000","last_peered":"2024-03-29T09:09:01.528733+0000","last_clean":"2024-03-29T09:09:01.528733+0000","last_became_active":"2024-03-29T09:08:57.420033+0000","last_became_peered":"2024-03-29T09:08:57.420033+0000","last_unstale":"2024-03-29T09:09:01.528733+0000","last_undegraded":"2024-03-29T09:09:01.528733+0000","last_fullsized":"2024-03-29T09:09:01.528733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530866+0000","last_change":"2024-03-29T09:09:01.441108+0000","last_active":"2024-03-29T09:09:01.530866+0000","last_peered":"2024-03-29T09:09:01.530866+0000","last_clean":"2024-03-29T09:09:01.530866+0000","last_became_active":"2024-03-29T09:09:01.440455+0000","last_became_peered":"2024-03-29T09:09:01.440455+0000","last_unstale":"2024-03-29T09:09:01.530866+0000","last_undegraded":"2024-03-29T09:09:01.530866+0000","last_fullsized":"2024-03-29T09:09:01.530866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424962+0000","last_change":"2024-03-29T09:08:56.908246+0000","last_active":"2024-03-29T09:09:01.424962+0000","last_peered":"2024-03-29T09:09:01.424962+0000","last_clean":"2024-03-29T09:09:01.424962+0000","last_became_active":"2024-03-29T09:08:56.907815+0000","last_became_peered":"2024-03-29T09:08:56.907815+0000","last_unstale":"2024-03-29T09:09:01.424962+0000","last_undegraded":"2024-03-29T09:09:01.424962+0000","last_fullsized":"2024-03-29T09:09:01.424962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527840+0000","last_change":"2024-03-29T09:08:57.420229+0000","last_active":"2024-03-29T09:09:01.527840+0000","last_peered":"2024-03-29T09:09:01.527840+0000","last_clean":"2024-03-29T09:09:01.527840+0000","last_became_active":"2024-03-29T09:08:57.419940+0000","last_became_peered":"2024-03-29T09:08:57.419940+0000","last_unstale":"2024-03-29T09:09:01.527840+0000","last_undegraded":"2024-03-29T09:09:01.527840+0000","last_fullsized":"2024-03-29T09:09:01.527840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:19.867399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527346+0000","last_change":"2024-03-29T09:09:01.447861+0000","last_active":"2024-03-29T09:09:01.527346+0000","last_peered":"2024-03-29T09:09:01.527346+0000","last_clean":"2024-03-29T09:09:01.527346+0000","last_became_active":"2024-03-29T09:09:01.447337+0000","last_became_peered":"2024-03-29T09:09:01.447337+0000","last_unstale":"2024-03-29T09:09:01.527346+0000","last_undegraded":"2024-03-29T09:09:01.527346+0000","last_fullsized":"2024-03-29T09:09:01.527346+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:19.879877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419991+0000","last_change":"2024-03-29T09:08:56.908710+0000","last_active":"2024-03-29T09:09:01.419991+0000","last_peered":"2024-03-29T09:09:01.419991+0000","last_clean":"2024-03-29T09:09:01.419991+0000","last_became_active":"2024-03-29T09:08:56.906213+0000","last_became_peered":"2024-03-29T09:08:56.906213+0000","last_unstale":"2024-03-29T09:09:01.419991+0000","last_undegraded":"2024-03-29T09:09:01.419991+0000","last_fullsized":"2024-03-29T09:09:01.419991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528714+0000","last_change":"2024-03-29T09:08:57.423981+0000","last_active":"2024-03-29T09:09:01.528714+0000","last_peered":"2024-03-29T09:09:01.528714+0000","last_clean":"2024-03-29T09:09:01.528714+0000","last_became_active":"2024-03-29T09:08:57.423667+0000","last_became_peered":"2024-03-29T09:08:57.423667+0000","last_unstale":"2024-03-29T09:09:01.528714+0000","last_undegraded":"2024-03-29T09:09:01.528714+0000","last_fullsized":"2024-03-29T09:09:01.528714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527487+0000","last_change":"2024-03-29T09:09:01.455365+0000","last_active":"2024-03-29T09:09:01.527487+0000","last_peered":"2024-03-29T09:09:01.527487+0000","last_clean":"2024-03-29T09:09:01.527487+0000","last_became_active":"2024-03-29T09:09:01.453949+0000","last_became_peered":"2024-03-29T09:09:01.453949+0000","last_unstale":"2024-03-29T09:09:01.527487+0000","last_undegraded":"2024-03-29T09:09:01.527487+0000","last_fullsized":"2024-03-29T09:09:01.527487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:59:00.449212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414280+0000","last_change":"2024-03-29T09:08:56.912593+0000","last_active":"2024-03-29T09:09:01.414280+0000","last_peered":"2024-03-29T09:09:01.414280+0000","last_clean":"2024-03-29T09:09:01.414280+0000","last_became_active":"2024-03-29T09:08:56.912401+0000","last_became_peered":"2024-03-29T09:08:56.912401+0000","last_unstale":"2024-03-29T09:09:01.414280+0000","last_undegraded":"2024-03-29T09:09:01.414280+0000","last_fullsized":"2024-03-29T09:09:01.414280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:27.805156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527468+0000","last_change":"2024-03-29T09:08:57.421182+0000","last_active":"2024-03-29T09:09:01.527468+0000","last_peered":"2024-03-29T09:09:01.527468+0000","last_clean":"2024-03-29T09:09:01.527468+0000","last_became_active":"2024-03-29T09:08:57.420980+0000","last_became_peered":"2024-03-29T09:08:57.420980+0000","last_unstale":"2024-03-29T09:09:01.527468+0000","last_undegraded":"2024-03-29T09:09:01.527468+0000","last_fullsized":"2024-03-29T09:09:01.527468+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527895+0000","last_change":"2024-03-29T09:09:01.433710+0000","last_active":"2024-03-29T09:09:01.527895+0000","last_peered":"2024-03-29T09:09:01.527895+0000","last_clean":"2024-03-29T09:09:01.527895+0000","last_became_active":"2024-03-29T09:09:01.433161+0000","last_became_peered":"2024-03-29T09:09:01.433161+0000","last_unstale":"2024-03-29T09:09:01.527895+0000","last_undegraded":"2024-03-29T09:09:01.527895+0000","last_fullsized":"2024-03-29T09:09:01.527895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:07:30.745379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429199+0000","last_change":"2024-03-29T09:08:56.911770+0000","last_active":"2024-03-29T09:09:01.429199+0000","last_peered":"2024-03-29T09:09:01.429199+0000","last_clean":"2024-03-29T09:09:01.429199+0000","last_became_active":"2024-03-29T09:08:56.911493+0000","last_became_peered":"2024-03-29T09:08:56.911493+0000","last_unstale":"2024-03-29T09:09:01.429199+0000","last_undegraded":"2024-03-29T09:09:01.429199+0000","last_fullsized":"2024-03-29T09:09:01.429199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530721+0000","last_change":"2024-03-29T09:08:57.426875+0000","last_active":"2024-03-29T09:09:01.530721+0000","last_peered":"2024-03-29T09:09:01.530721+0000","last_clean":"2024-03-29T09:09:01.530721+0000","last_became_active":"2024-03-29T09:08:57.426586+0000","last_became_peered":"2024-03-29T09:08:57.426586+0000","last_unstale":"2024-03-29T09:09:01.530721+0000","last_undegraded":"2024-03-29T09:09:01.530721+0000","last_fullsized":"2024-03-29T09:09:01.530721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527529+0000","last_change":"2024-03-29T09:09:01.440649+0000","last_active":"2024-03-29T09:09:01.527529+0000","last_peered":"2024-03-29T09:09:01.527529+0000","last_clean":"2024-03-29T09:09:01.527529+0000","last_became_active":"2024-03-29T09:09:01.440252+0000","last_became_peered":"2024-03-29T09:09:01.440252+0000","last_unstale":"2024-03-29T09:09:01.527529+0000","last_undegraded":"2024-03-29T09:09:01.527529+0000","last_fullsized":"2024-03-29T09:09:01.527529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414326+0000","last_change":"2024-03-29T09:08:56.908431+0000","last_active":"2024-03-29T09:09:01.414326+0000","last_peered":"2024-03-29T09:09:01.414326+0000","last_clean":"2024-03-29T09:09:01.414326+0000","last_became_active":"2024-03-29T09:08:56.906823+0000","last_became_peered":"2024-03-29T09:08:56.906823+0000","last_unstale":"2024-03-29T09:09:01.414326+0000","last_undegraded":"2024-03-29T09:09:01.414326+0000","last_fullsized":"2024-03-29T09:09:01.414326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:24.070569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527421+0000","last_change":"2024-03-29T09:08:57.410009+0000","last_active":"2024-03-29T09:09:01.527421+0000","last_peered":"2024-03-29T09:09:01.527421+0000","last_clean":"2024-03-29T09:09:01.527421+0000","last_became_active":"2024-03-29T09:08:57.409632+0000","last_became_peered":"2024-03-29T09:08:57.409632+0000","last_unstale":"2024-03-29T09:09:01.527421+0000","last_undegraded":"2024-03-29T09:09:01.527421+0000","last_fullsized":"2024-03-29T09:09:01.527421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527438+0000","last_change":"2024-03-29T09:09:01.447901+0000","last_active":"2024-03-29T09:09:01.527438+0000","last_peered":"2024-03-29T09:09:01.527438+0000","last_clean":"2024-03-29T09:09:01.527438+0000","last_became_active":"2024-03-29T09:09:01.447482+0000","last_became_peered":"2024-03-29T09:09:01.447482+0000","last_unstale":"2024-03-29T09:09:01.527438+0000","last_undegraded":"2024-03-29T09:09:01.527438+0000","last_fullsized":"2024-03-29T09:09:01.527438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.426470+0000","last_change":"2024-03-29T09:08:56.909506+0000","last_active":"2024-03-29T09:09:01.426470+0000","last_peered":"2024-03-29T09:09:01.426470+0000","last_clean":"2024-03-29T09:09:01.426470+0000","last_became_active":"2024-03-29T09:08:56.909183+0000","last_became_peered":"2024-03-29T09:08:56.909183+0000","last_unstale":"2024-03-29T09:09:01.426470+0000","last_undegraded":"2024-03-29T09:09:01.426470+0000","last_fullsized":"2024-03-29T09:09:01.426470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:04:15.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531038+0000","last_change":"2024-03-29T09:08:57.413715+0000","last_active":"2024-03-29T09:09:01.531038+0000","last_peered":"2024-03-29T09:09:01.531038+0000","last_clean":"2024-03-29T09:09:01.531038+0000","last_became_active":"2024-03-29T09:08:57.413145+0000","last_became_peered":"2024-03-29T09:08:57.413145+0000","last_unstale":"2024-03-29T09:09:01.531038+0000","last_undegraded":"2024-03-29T09:09:01.531038+0000","last_fullsized":"2024-03-29T09:09:01.531038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527274+0000","last_change":"2024-03-29T09:09:01.444258+0000","last_active":"2024-03-29T09:09:01.527274+0000","last_peered":"2024-03-29T09:09:01.527274+0000","last_clean":"2024-03-29T09:09:01.527274+0000","last_became_active":"2024-03-29T09:09:01.443829+0000","last_became_peered":"2024-03-29T09:09:01.443829+0000","last_unstale":"2024-03-29T09:09:01.527274+0000","last_undegraded":"2024-03-29T09:09:01.527274+0000","last_fullsized":"2024-03-29T09:09:01.527274+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:29:01.967027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.424436+0000","last_change":"2024-03-29T09:08:56.909671+0000","last_active":"2024-03-29T09:09:01.424436+0000","last_peered":"2024-03-29T09:09:01.424436+0000","last_clean":"2024-03-29T09:09:01.424436+0000","last_became_active":"2024-03-29T09:08:56.909426+0000","last_became_peered":"2024-03-29T09:08:56.909426+0000","last_unstale":"2024-03-29T09:09:01.424436+0000","last_undegraded":"2024-03-29T09:09:01.424436+0000","last_fullsized":"2024-03-29T09:09:01.424436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530931+0000","last_change":"2024-03-29T09:08:57.422965+0000","last_active":"2024-03-29T09:09:01.530931+0000","last_peered":"2024-03-29T09:09:01.530931+0000","last_clean":"2024-03-29T09:09:01.530931+0000","last_became_active":"2024-03-29T09:08:57.422689+0000","last_became_peered":"2024-03-29T09:08:57.422689+0000","last_unstale":"2024-03-29T09:09:01.530931+0000","last_undegraded":"2024-03-29T09:09:01.530931+0000","last_fullsized":"2024-03-29T09:09:01.530931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530907+0000","last_change":"2024-03-29T09:09:01.447399+0000","last_active":"2024-03-29T09:09:01.530907+0000","last_peered":"2024-03-29T09:09:01.530907+0000","last_clean":"2024-03-29T09:09:01.530907+0000","last_became_active":"2024-03-29T09:09:01.446880+0000","last_became_peered":"2024-03-29T09:09:01.446880+0000","last_unstale":"2024-03-29T09:09:01.530907+0000","last_undegraded":"2024-03-29T09:09:01.530907+0000","last_fullsized":"2024-03-29T09:09:01.530907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:49:27.133997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.433574+0000","last_change":"2024-03-29T09:08:56.908399+0000","last_active":"2024-03-29T09:09:01.433574+0000","last_peered":"2024-03-29T09:09:01.433574+0000","last_clean":"2024-03-29T09:09:01.433574+0000","last_became_active":"2024-03-29T09:08:56.906866+0000","last_became_peered":"2024-03-29T09:08:56.906866+0000","last_unstale":"2024-03-29T09:09:01.433574+0000","last_undegraded":"2024-03-29T09:09:01.433574+0000","last_fullsized":"2024-03-29T09:09:01.433574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527738+0000","last_change":"2024-03-29T09:08:57.423735+0000","last_active":"2024-03-29T09:09:01.527738+0000","last_peered":"2024-03-29T09:09:01.527738+0000","last_clean":"2024-03-29T09:09:01.527738+0000","last_became_active":"2024-03-29T09:08:57.423405+0000","last_became_peered":"2024-03-29T09:08:57.423405+0000","last_unstale":"2024-03-29T09:09:01.527738+0000","last_undegraded":"2024-03-29T09:09:01.527738+0000","last_fullsized":"2024-03-29T09:09:01.527738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:33.393767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527448+0000","last_change":"2024-03-29T09:09:01.442875+0000","last_active":"2024-03-29T09:09:01.527448+0000","last_peered":"2024-03-29T09:09:01.527448+0000","last_clean":"2024-03-29T09:09:01.527448+0000","last_became_active":"2024-03-29T09:09:01.442426+0000","last_became_peered":"2024-03-29T09:09:01.442426+0000","last_unstale":"2024-03-29T09:09:01.527448+0000","last_undegraded":"2024-03-29T09:09:01.527448+0000","last_fullsized":"2024-03-29T09:09:01.527448+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-03-29T09:09:15.577021+0000","last_change":"2024-03-29T09:08:56.895791+0000","last_active":"2024-03-29T09:09:15.577021+0000","last_peered":"2024-03-29T09:09:15.577021+0000","last_clean":"2024-03-29T09:09:15.577021+0000","last_became_active":"2024-03-29T09:08:56.894997+0000","last_became_peered":"2024-03-29T09:08:56.894997+0000","last_unstale":"2024-03-29T09:09:15.577021+0000","last_undegraded":"2024-03-29T09:09:15.577021+0000","last_fullsized":"2024-03-29T09:09:15.577021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534030+0000","last_change":"2024-03-29T09:08:57.427194+0000","last_active":"2024-03-29T09:09:01.534030+0000","last_peered":"2024-03-29T09:09:01.534030+0000","last_clean":"2024-03-29T09:09:01.534030+0000","last_became_active":"2024-03-29T09:08:57.425015+0000","last_became_peered":"2024-03-29T09:08:57.425015+0000","last_unstale":"2024-03-29T09:09:01.534030+0000","last_undegraded":"2024-03-29T09:09:01.534030+0000","last_fullsized":"2024-03-29T09:09:01.534030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:49.705445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527755+0000","last_change":"2024-03-29T09:09:01.442768+0000","last_active":"2024-03-29T09:09:01.527755+0000","last_peered":"2024-03-29T09:09:01.527755+0000","last_clean":"2024-03-29T09:09:01.527755+0000","last_became_active":"2024-03-29T09:09:01.442373+0000","last_became_peered":"2024-03-29T09:09:01.442373+0000","last_unstale":"2024-03-29T09:09:01.527755+0000","last_undegraded":"2024-03-29T09:09:01.527755+0000","last_fullsized":"2024-03-29T09:09:01.527755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:05:41.144719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.432870+0000","last_change":"2024-03-29T09:08:56.912783+0000","last_active":"2024-03-29T09:09:01.432870+0000","last_peered":"2024-03-29T09:09:01.432870+0000","last_clean":"2024-03-29T09:09:01.432870+0000","last_became_active":"2024-03-29T09:08:56.912488+0000","last_became_peered":"2024-03-29T09:08:56.912488+0000","last_unstale":"2024-03-29T09:09:01.432870+0000","last_undegraded":"2024-03-29T09:09:01.432870+0000","last_fullsized":"2024-03-29T09:09:01.432870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525440+0000","last_change":"2024-03-29T09:08:57.417913+0000","last_active":"2024-03-29T09:09:01.525440+0000","last_peered":"2024-03-29T09:09:01.525440+0000","last_clean":"2024-03-29T09:09:01.525440+0000","last_became_active":"2024-03-29T09:08:57.416399+0000","last_became_peered":"2024-03-29T09:08:57.416399+0000","last_unstale":"2024-03-29T09:09:01.525440+0000","last_undegraded":"2024-03-29T09:09:01.525440+0000","last_fullsized":"2024-03-29T09:09:01.525440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525543+0000","last_change":"2024-03-29T09:09:01.448636+0000","last_active":"2024-03-29T09:09:01.525543+0000","last_peered":"2024-03-29T09:09:01.525543+0000","last_clean":"2024-03-29T09:09:01.525543+0000","last_became_active":"2024-03-29T09:09:01.448098+0000","last_became_peered":"2024-03-29T09:09:01.448098+0000","last_unstale":"2024-03-29T09:09:01.525543+0000","last_undegraded":"2024-03-29T09:09:01.525543+0000","last_fullsized":"2024-03-29T09:09:01.525543+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414449+0000","last_change":"2024-03-29T09:08:56.906000+0000","last_active":"2024-03-29T09:09:01.414449+0000","last_peered":"2024-03-29T09:09:01.414449+0000","last_clean":"2024-03-29T09:09:01.414449+0000","last_became_active":"2024-03-29T09:08:56.905743+0000","last_became_peered":"2024-03-29T09:08:56.905743+0000","last_unstale":"2024-03-29T09:09:01.414449+0000","last_undegraded":"2024-03-29T09:09:01.414449+0000","last_fullsized":"2024-03-29T09:09:01.414449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:18.722602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527722+0000","last_change":"2024-03-29T09:08:57.429076+0000","last_active":"2024-03-29T09:09:01.527722+0000","last_peered":"2024-03-29T09:09:01.527722+0000","last_clean":"2024-03-29T09:09:01.527722+0000","last_became_active":"2024-03-29T09:08:57.426637+0000","last_became_peered":"2024-03-29T09:08:57.426637+0000","last_unstale":"2024-03-29T09:09:01.527722+0000","last_undegraded":"2024-03-29T09:09:01.527722+0000","last_fullsized":"2024-03-29T09:09:01.527722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:48.952540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530885+0000","last_change":"2024-03-29T09:09:01.437381+0000","last_active":"2024-03-29T09:09:01.530885+0000","last_peered":"2024-03-29T09:09:01.530885+0000","last_clean":"2024-03-29T09:09:01.530885+0000","last_became_active":"2024-03-29T09:09:01.436950+0000","last_became_peered":"2024-03-29T09:09:01.436950+0000","last_unstale":"2024-03-29T09:09:01.530885+0000","last_undegraded":"2024-03-29T09:09:01.530885+0000","last_fullsized":"2024-03-29T09:09:01.530885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.356624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422703+0000","last_change":"2024-03-29T09:08:56.915220+0000","last_active":"2024-03-29T09:09:01.422703+0000","last_peered":"2024-03-29T09:09:01.422703+0000","last_clean":"2024-03-29T09:09:01.422703+0000","last_became_active":"2024-03-29T09:08:56.914954+0000","last_became_peered":"2024-03-29T09:08:56.914954+0000","last_unstale":"2024-03-29T09:09:01.422703+0000","last_undegraded":"2024-03-29T09:09:01.422703+0000","last_fullsized":"2024-03-29T09:09:01.422703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:19:55.114280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528690+0000","last_change":"2024-03-29T09:08:57.423950+0000","last_active":"2024-03-29T09:09:01.528690+0000","last_peered":"2024-03-29T09:09:01.528690+0000","last_clean":"2024-03-29T09:09:01.528690+0000","last_became_active":"2024-03-29T09:08:57.423703+0000","last_became_peered":"2024-03-29T09:08:57.423703+0000","last_unstale":"2024-03-29T09:09:01.528690+0000","last_undegraded":"2024-03-29T09:09:01.528690+0000","last_fullsized":"2024-03-29T09:09:01.528690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:28.288403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530874+0000","last_change":"2024-03-29T09:09:01.440631+0000","last_active":"2024-03-29T09:09:01.530874+0000","last_peered":"2024-03-29T09:09:01.530874+0000","last_clean":"2024-03-29T09:09:01.530874+0000","last_became_active":"2024-03-29T09:09:01.440159+0000","last_became_peered":"2024-03-29T09:09:01.440159+0000","last_unstale":"2024-03-29T09:09:01.530874+0000","last_undegraded":"2024-03-29T09:09:01.530874+0000","last_fullsized":"2024-03-29T09:09:01.530874+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.423297+0000","last_change":"2024-03-29T09:08:56.904028+0000","last_active":"2024-03-29T09:09:01.423297+0000","last_peered":"2024-03-29T09:09:01.423297+0000","last_clean":"2024-03-29T09:09:01.423297+0000","last_became_active":"2024-03-29T09:08:56.903620+0000","last_became_peered":"2024-03-29T09:08:56.903620+0000","last_unstale":"2024-03-29T09:09:01.423297+0000","last_undegraded":"2024-03-29T09:09:01.423297+0000","last_fullsized":"2024-03-29T09:09:01.423297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527655+0000","last_change":"2024-03-29T09:08:57.414230+0000","last_active":"2024-03-29T09:09:01.527655+0000","last_peered":"2024-03-29T09:09:01.527655+0000","last_clean":"2024-03-29T09:09:01.527655+0000","last_became_active":"2024-03-29T09:08:57.413731+0000","last_became_peered":"2024-03-29T09:08:57.413731+0000","last_unstale":"2024-03-29T09:09:01.527655+0000","last_undegraded":"2024-03-29T09:09:01.527655+0000","last_fullsized":"2024-03-29T09:09:01.527655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:03.521556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528703+0000","last_change":"2024-03-29T09:09:01.435579+0000","last_active":"2024-03-29T09:09:01.528703+0000","last_peered":"2024-03-29T09:09:01.528703+0000","last_clean":"2024-03-29T09:09:01.528703+0000","last_became_active":"2024-03-29T09:09:01.433461+0000","last_became_peered":"2024-03-29T09:09:01.433461+0000","last_unstale":"2024-03-29T09:09:01.528703+0000","last_undegraded":"2024-03-29T09:09:01.528703+0000","last_fullsized":"2024-03-29T09:09:01.528703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:17.513175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422681+0000","last_change":"2024-03-29T09:08:56.915230+0000","last_active":"2024-03-29T09:09:01.422681+0000","last_peered":"2024-03-29T09:09:01.422681+0000","last_clean":"2024-03-29T09:09:01.422681+0000","last_became_active":"2024-03-29T09:08:56.914943+0000","last_became_peered":"2024-03-29T09:08:56.914943+0000","last_unstale":"2024-03-29T09:09:01.422681+0000","last_undegraded":"2024-03-29T09:09:01.422681+0000","last_fullsized":"2024-03-29T09:09:01.422681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:19.251442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531054+0000","last_change":"2024-03-29T09:08:57.425607+0000","last_active":"2024-03-29T09:09:01.531054+0000","last_peered":"2024-03-29T09:09:01.531054+0000","last_clean":"2024-03-29T09:09:01.531054+0000","last_became_active":"2024-03-29T09:08:57.424221+0000","last_became_peered":"2024-03-29T09:08:57.424221+0000","last_unstale":"2024-03-29T09:09:01.531054+0000","last_undegraded":"2024-03-29T09:09:01.531054+0000","last_fullsized":"2024-03-29T09:09:01.531054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:56.605995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525430+0000","last_change":"2024-03-29T09:09:01.447292+0000","last_active":"2024-03-29T09:09:01.525430+0000","last_peered":"2024-03-29T09:09:01.525430+0000","last_clean":"2024-03-29T09:09:01.525430+0000","last_became_active":"2024-03-29T09:09:01.446779+0000","last_became_peered":"2024-03-29T09:09:01.446779+0000","last_unstale":"2024-03-29T09:09:01.525430+0000","last_undegraded":"2024-03-29T09:09:01.525430+0000","last_fullsized":"2024-03-29T09:09:01.525430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:10.002579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.419894+0000","last_change":"2024-03-29T09:08:56.905021+0000","last_active":"2024-03-29T09:09:01.419894+0000","last_peered":"2024-03-29T09:09:01.419894+0000","last_clean":"2024-03-29T09:09:01.419894+0000","last_became_active":"2024-03-29T09:08:56.903363+0000","last_became_peered":"2024-03-29T09:08:56.903363+0000","last_unstale":"2024-03-29T09:09:01.419894+0000","last_undegraded":"2024-03-29T09:09:01.419894+0000","last_fullsized":"2024-03-29T09:09:01.419894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530999+0000","last_change":"2024-03-29T09:08:57.427940+0000","last_active":"2024-03-29T09:09:01.530999+0000","last_peered":"2024-03-29T09:09:01.530999+0000","last_clean":"2024-03-29T09:09:01.530999+0000","last_became_active":"2024-03-29T09:08:57.427293+0000","last_became_peered":"2024-03-29T09:08:57.427293+0000","last_unstale":"2024-03-29T09:09:01.530999+0000","last_undegraded":"2024-03-29T09:09:01.530999+0000","last_fullsized":"2024-03-29T09:09:01.530999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525532+0000","last_change":"2024-03-29T09:09:01.451319+0000","last_active":"2024-03-29T09:09:01.525532+0000","last_peered":"2024-03-29T09:09:01.525532+0000","last_clean":"2024-03-29T09:09:01.525532+0000","last_became_active":"2024-03-29T09:09:01.450880+0000","last_became_peered":"2024-03-29T09:09:01.450880+0000","last_unstale":"2024-03-29T09:09:01.525532+0000","last_undegraded":"2024-03-29T09:09:01.525532+0000","last_fullsized":"2024-03-29T09:09:01.525532+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.475145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414329+0000","last_change":"2024-03-29T09:08:56.909170+0000","last_active":"2024-03-29T09:09:01.414329+0000","last_peered":"2024-03-29T09:09:01.414329+0000","last_clean":"2024-03-29T09:09:01.414329+0000","last_became_active":"2024-03-29T09:08:56.908866+0000","last_became_peered":"2024-03-29T09:08:56.908866+0000","last_unstale":"2024-03-29T09:09:01.414329+0000","last_undegraded":"2024-03-29T09:09:01.414329+0000","last_fullsized":"2024-03-29T09:09:01.414329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:58.589863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533977+0000","last_change":"2024-03-29T09:08:57.415898+0000","last_active":"2024-03-29T09:09:01.533977+0000","last_peered":"2024-03-29T09:09:01.533977+0000","last_clean":"2024-03-29T09:09:01.533977+0000","last_became_active":"2024-03-29T09:08:57.415653+0000","last_became_peered":"2024-03-29T09:08:57.415653+0000","last_unstale":"2024-03-29T09:09:01.533977+0000","last_undegraded":"2024-03-29T09:09:01.533977+0000","last_fullsized":"2024-03-29T09:09:01.533977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525566+0000","last_change":"2024-03-29T09:09:01.451066+0000","last_active":"2024-03-29T09:09:01.525566+0000","last_peered":"2024-03-29T09:09:01.525566+0000","last_clean":"2024-03-29T09:09:01.525566+0000","last_became_active":"2024-03-29T09:09:01.450444+0000","last_became_peered":"2024-03-29T09:09:01.450444+0000","last_unstale":"2024-03-29T09:09:01.525566+0000","last_undegraded":"2024-03-29T09:09:01.525566+0000","last_fullsized":"2024-03-29T09:09:01.525566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:11:24.412218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530697+0000","last_change":"2024-03-29T09:08:57.426320+0000","last_active":"2024-03-29T09:09:01.530697+0000","last_peered":"2024-03-29T09:09:01.530697+0000","last_clean":"2024-03-29T09:09:01.530697+0000","last_became_active":"2024-03-29T09:08:57.426071+0000","last_became_peered":"2024-03-29T09:08:57.426071+0000","last_unstale":"2024-03-29T09:09:01.530697+0000","last_undegraded":"2024-03-29T09:09:01.530697+0000","last_fullsized":"2024-03-29T09:09:01.530697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.428903+0000","last_change":"2024-03-29T09:08:56.906294+0000","last_active":"2024-03-29T09:09:01.428903+0000","last_peered":"2024-03-29T09:09:01.428903+0000","last_clean":"2024-03-29T09:09:01.428903+0000","last_became_active":"2024-03-29T09:08:56.905993+0000","last_became_peered":"2024-03-29T09:08:56.905993+0000","last_unstale":"2024-03-29T09:09:01.428903+0000","last_undegraded":"2024-03-29T09:09:01.428903+0000","last_fullsized":"2024-03-29T09:09:01.428903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:40.264120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527859+0000","last_change":"2024-03-29T09:09:01.435004+0000","last_active":"2024-03-29T09:09:01.527859+0000","last_peered":"2024-03-29T09:09:01.527859+0000","last_clean":"2024-03-29T09:09:01.527859+0000","last_became_active":"2024-03-29T09:09:01.434628+0000","last_became_peered":"2024-03-29T09:09:01.434628+0000","last_unstale":"2024-03-29T09:09:01.527859+0000","last_undegraded":"2024-03-29T09:09:01.527859+0000","last_fullsized":"2024-03-29T09:09:01.527859+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:05.380910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527458+0000","last_change":"2024-03-29T09:08:57.411609+0000","last_active":"2024-03-29T09:09:01.527458+0000","last_peered":"2024-03-29T09:09:01.527458+0000","last_clean":"2024-03-29T09:09:01.527458+0000","last_became_active":"2024-03-29T09:08:57.411313+0000","last_became_peered":"2024-03-29T09:08:57.411313+0000","last_unstale":"2024-03-29T09:09:01.527458+0000","last_undegraded":"2024-03-29T09:09:01.527458+0000","last_fullsized":"2024-03-29T09:09:01.527458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414265+0000","last_change":"2024-03-29T09:08:56.900590+0000","last_active":"2024-03-29T09:09:01.414265+0000","last_peered":"2024-03-29T09:09:01.414265+0000","last_clean":"2024-03-29T09:09:01.414265+0000","last_became_active":"2024-03-29T09:08:56.900329+0000","last_became_peered":"2024-03-29T09:08:56.900329+0000","last_unstale":"2024-03-29T09:09:01.414265+0000","last_undegraded":"2024-03-29T09:09:01.414265+0000","last_fullsized":"2024-03-29T09:09:01.414265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:56.593036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420635+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:09:01.420635+0000","last_peered":"2024-03-29T09:09:01.420635+0000","last_clean":"2024-03-29T09:09:01.420635+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:09:01.420635+0000","last_undegraded":"2024-03-29T09:09:01.420635+0000","last_fullsized":"2024-03-29T09:09:01.420635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530864+0000","last_change":"2024-03-29T09:09:01.438759+0000","last_active":"2024-03-29T09:09:01.530864+0000","last_peered":"2024-03-29T09:09:01.530864+0000","last_clean":"2024-03-29T09:09:01.530864+0000","last_became_active":"2024-03-29T09:09:01.438275+0000","last_became_peered":"2024-03-29T09:09:01.438275+0000","last_unstale":"2024-03-29T09:09:01.530864+0000","last_undegraded":"2024-03-29T09:09:01.530864+0000","last_fullsized":"2024-03-29T09:09:01.530864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:02:29.227678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525418+0000","last_change":"2024-03-29T09:08:57.418469+0000","last_active":"2024-03-29T09:09:01.525418+0000","last_peered":"2024-03-29T09:09:01.525418+0000","last_clean":"2024-03-29T09:09:01.525418+0000","last_became_active":"2024-03-29T09:08:57.416011+0000","last_became_peered":"2024-03-29T09:08:57.416011+0000","last_unstale":"2024-03-29T09:09:01.525418+0000","last_undegraded":"2024-03-29T09:09:01.525418+0000","last_fullsized":"2024-03-29T09:09:01.525418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:46:59.597893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414118+0000","last_change":"2024-03-29T09:08:56.900382+0000","last_active":"2024-03-29T09:09:01.414118+0000","last_peered":"2024-03-29T09:09:01.414118+0000","last_clean":"2024-03-29T09:09:01.414118+0000","last_became_active":"2024-03-29T09:08:56.899884+0000","last_became_peered":"2024-03-29T09:08:56.899884+0000","last_unstale":"2024-03-29T09:09:01.414118+0000","last_undegraded":"2024-03-29T09:09:01.414118+0000","last_fullsized":"2024-03-29T09:09:01.414118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:51.548290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525469+0000","last_change":"2024-03-29T09:09:01.447790+0000","last_active":"2024-03-29T09:09:01.525469+0000","last_peered":"2024-03-29T09:09:01.525469+0000","last_clean":"2024-03-29T09:09:01.525469+0000","last_became_active":"2024-03-29T09:09:01.447254+0000","last_became_peered":"2024-03-29T09:09:01.447254+0000","last_unstale":"2024-03-29T09:09:01.525469+0000","last_undegraded":"2024-03-29T09:09:01.525469+0000","last_fullsized":"2024-03-29T09:09:01.525469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:27.709797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.415102+0000","last_change":"2024-03-29T09:08:56.904347+0000","last_active":"2024-03-29T09:09:01.415102+0000","last_peered":"2024-03-29T09:09:01.415102+0000","last_clean":"2024-03-29T09:09:01.415102+0000","last_became_active":"2024-03-29T09:08:56.903387+0000","last_became_peered":"2024-03-29T09:08:56.903387+0000","last_unstale":"2024-03-29T09:09:01.415102+0000","last_undegraded":"2024-03-29T09:09:01.415102+0000","last_fullsized":"2024-03-29T09:09:01.415102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525510+0000","last_change":"2024-03-29T09:08:57.419559+0000","last_active":"2024-03-29T09:09:01.525510+0000","last_peered":"2024-03-29T09:09:01.525510+0000","last_clean":"2024-03-29T09:09:01.525510+0000","last_became_active":"2024-03-29T09:08:57.419295+0000","last_became_peered":"2024-03-29T09:08:57.419295+0000","last_unstale":"2024-03-29T09:09:01.525510+0000","last_undegraded":"2024-03-29T09:09:01.525510+0000","last_fullsized":"2024-03-29T09:09:01.525510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527366+0000","last_change":"2024-03-29T09:09:01.433109+0000","last_active":"2024-03-29T09:09:01.527366+0000","last_peered":"2024-03-29T09:09:01.527366+0000","last_clean":"2024-03-29T09:09:01.527366+0000","last_became_active":"2024-03-29T09:09:01.432662+0000","last_became_peered":"2024-03-29T09:09:01.432662+0000","last_unstale":"2024-03-29T09:09:01.527366+0000","last_undegraded":"2024-03-29T09:09:01.527366+0000","last_fullsized":"2024-03-29T09:09:01.527366+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:41.465124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422867+0000","last_change":"2024-03-29T09:08:56.912181+0000","last_active":"2024-03-29T09:09:01.422867+0000","last_peered":"2024-03-29T09:09:01.422867+0000","last_clean":"2024-03-29T09:09:01.422867+0000","last_became_active":"2024-03-29T09:08:56.911842+0000","last_became_peered":"2024-03-29T09:08:56.911842+0000","last_unstale":"2024-03-29T09:09:01.422867+0000","last_undegraded":"2024-03-29T09:09:01.422867+0000","last_fullsized":"2024-03-29T09:09:01.422867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527556+0000","last_change":"2024-03-29T09:08:57.423784+0000","last_active":"2024-03-29T09:09:01.527556+0000","last_peered":"2024-03-29T09:09:01.527556+0000","last_clean":"2024-03-29T09:09:01.527556+0000","last_became_active":"2024-03-29T09:08:57.423442+0000","last_became_peered":"2024-03-29T09:08:57.423442+0000","last_unstale":"2024-03-29T09:09:01.527556+0000","last_undegraded":"2024-03-29T09:09:01.527556+0000","last_fullsized":"2024-03-29T09:09:01.527556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530823+0000","last_change":"2024-03-29T09:09:01.451554+0000","last_active":"2024-03-29T09:09:01.530823+0000","last_peered":"2024-03-29T09:09:01.530823+0000","last_clean":"2024-03-29T09:09:01.530823+0000","last_became_active":"2024-03-29T09:09:01.450348+0000","last_became_peered":"2024-03-29T09:09:01.450348+0000","last_unstale":"2024-03-29T09:09:01.530823+0000","last_undegraded":"2024-03-29T09:09:01.530823+0000","last_fullsized":"2024-03-29T09:09:01.530823+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:38.930018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414924+0000","last_change":"2024-03-29T09:08:56.911357+0000","last_active":"2024-03-29T09:09:01.414924+0000","last_peered":"2024-03-29T09:09:01.414924+0000","last_clean":"2024-03-29T09:09:01.414924+0000","last_became_active":"2024-03-29T09:08:56.910692+0000","last_became_peered":"2024-03-29T09:08:56.910692+0000","last_unstale":"2024-03-29T09:09:01.414924+0000","last_undegraded":"2024-03-29T09:09:01.414924+0000","last_fullsized":"2024-03-29T09:09:01.414924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:02:32.927312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527540+0000","last_change":"2024-03-29T09:08:57.415718+0000","last_active":"2024-03-29T09:09:01.527540+0000","last_peered":"2024-03-29T09:09:01.527540+0000","last_clean":"2024-03-29T09:09:01.527540+0000","last_became_active":"2024-03-29T09:08:57.415442+0000","last_became_peered":"2024-03-29T09:08:57.415442+0000","last_unstale":"2024-03-29T09:09:01.527540+0000","last_undegraded":"2024-03-29T09:09:01.527540+0000","last_fullsized":"2024-03-29T09:09:01.527540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:44.638689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531018+0000","last_change":"2024-03-29T09:09:01.444447+0000","last_active":"2024-03-29T09:09:01.531018+0000","last_peered":"2024-03-29T09:09:01.531018+0000","last_clean":"2024-03-29T09:09:01.531018+0000","last_became_active":"2024-03-29T09:09:01.443976+0000","last_became_peered":"2024-03-29T09:09:01.443976+0000","last_unstale":"2024-03-29T09:09:01.531018+0000","last_undegraded":"2024-03-29T09:09:01.531018+0000","last_fullsized":"2024-03-29T09:09:01.531018+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:19.415693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422915+0000","last_change":"2024-03-29T09:08:56.904777+0000","last_active":"2024-03-29T09:09:01.422915+0000","last_peered":"2024-03-29T09:09:01.422915+0000","last_clean":"2024-03-29T09:09:01.422915+0000","last_became_active":"2024-03-29T09:08:56.904503+0000","last_became_peered":"2024-03-29T09:08:56.904503+0000","last_unstale":"2024-03-29T09:09:01.422915+0000","last_undegraded":"2024-03-29T09:09:01.422915+0000","last_fullsized":"2024-03-29T09:09:01.422915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527647+0000","last_change":"2024-03-29T09:08:57.409679+0000","last_active":"2024-03-29T09:09:01.527647+0000","last_peered":"2024-03-29T09:09:01.527647+0000","last_clean":"2024-03-29T09:09:01.527647+0000","last_became_active":"2024-03-29T09:08:57.409346+0000","last_became_peered":"2024-03-29T09:08:57.409346+0000","last_unstale":"2024-03-29T09:09:01.527647+0000","last_undegraded":"2024-03-29T09:09:01.527647+0000","last_fullsized":"2024-03-29T09:09:01.527647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527392+0000","last_change":"2024-03-29T09:09:01.451808+0000","last_active":"2024-03-29T09:09:01.527392+0000","last_peered":"2024-03-29T09:09:01.527392+0000","last_clean":"2024-03-29T09:09:01.527392+0000","last_became_active":"2024-03-29T09:09:01.451262+0000","last_became_peered":"2024-03-29T09:09:01.451262+0000","last_unstale":"2024-03-29T09:09:01.527392+0000","last_undegraded":"2024-03-29T09:09:01.527392+0000","last_fullsized":"2024-03-29T09:09:01.527392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:37:25.394835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.422122+0000","last_change":"2024-03-29T09:08:56.912120+0000","last_active":"2024-03-29T09:09:01.422122+0000","last_peered":"2024-03-29T09:09:01.422122+0000","last_clean":"2024-03-29T09:09:01.422122+0000","last_became_active":"2024-03-29T09:08:56.911834+0000","last_became_peered":"2024-03-29T09:08:56.911834+0000","last_unstale":"2024-03-29T09:09:01.422122+0000","last_undegraded":"2024-03-29T09:09:01.422122+0000","last_fullsized":"2024-03-29T09:09:01.422122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:06.853615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527406+0000","last_change":"2024-03-29T09:08:57.413619+0000","last_active":"2024-03-29T09:09:01.527406+0000","last_peered":"2024-03-29T09:09:01.527406+0000","last_clean":"2024-03-29T09:09:01.527406+0000","last_became_active":"2024-03-29T09:08:57.409612+0000","last_became_peered":"2024-03-29T09:08:57.409612+0000","last_unstale":"2024-03-29T09:09:01.527406+0000","last_undegraded":"2024-03-29T09:09:01.527406+0000","last_fullsized":"2024-03-29T09:09:01.527406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:48:53.977674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533989+0000","last_change":"2024-03-29T09:09:01.436295+0000","last_active":"2024-03-29T09:09:01.533989+0000","last_peered":"2024-03-29T09:09:01.533989+0000","last_clean":"2024-03-29T09:09:01.533989+0000","last_became_active":"2024-03-29T09:09:01.435972+0000","last_became_peered":"2024-03-29T09:09:01.435972+0000","last_unstale":"2024-03-29T09:09:01.533989+0000","last_undegraded":"2024-03-29T09:09:01.533989+0000","last_fullsized":"2024-03-29T09:09:01.533989+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:23.390004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429481+0000","last_change":"2024-03-29T09:08:56.907160+0000","last_active":"2024-03-29T09:09:01.429481+0000","last_peered":"2024-03-29T09:09:01.429481+0000","last_clean":"2024-03-29T09:09:01.429481+0000","last_became_active":"2024-03-29T09:08:56.906801+0000","last_became_peered":"2024-03-29T09:08:56.906801+0000","last_unstale":"2024-03-29T09:09:01.429481+0000","last_undegraded":"2024-03-29T09:09:01.429481+0000","last_fullsized":"2024-03-29T09:09:01.429481+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:12.904579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527323+0000","last_change":"2024-03-29T09:08:57.425389+0000","last_active":"2024-03-29T09:09:01.527323+0000","last_peered":"2024-03-29T09:09:01.527323+0000","last_clean":"2024-03-29T09:09:01.527323+0000","last_became_active":"2024-03-29T09:08:57.425127+0000","last_became_peered":"2024-03-29T09:08:57.425127+0000","last_unstale":"2024-03-29T09:09:01.527323+0000","last_undegraded":"2024-03-29T09:09:01.527323+0000","last_fullsized":"2024-03-29T09:09:01.527323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:19.730633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527793+0000","last_change":"2024-03-29T09:09:01.432980+0000","last_active":"2024-03-29T09:09:01.527793+0000","last_peered":"2024-03-29T09:09:01.527793+0000","last_clean":"2024-03-29T09:09:01.527793+0000","last_became_active":"2024-03-29T09:09:01.432597+0000","last_became_peered":"2024-03-29T09:09:01.432597+0000","last_unstale":"2024-03-29T09:09:01.527793+0000","last_undegraded":"2024-03-29T09:09:01.527793+0000","last_fullsized":"2024-03-29T09:09:01.527793+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:52.143266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.414584+0000","last_change":"2024-03-29T09:08:56.901765+0000","last_active":"2024-03-29T09:09:01.414584+0000","last_peered":"2024-03-29T09:09:01.414584+0000","last_clean":"2024-03-29T09:09:01.414584+0000","last_became_active":"2024-03-29T09:08:56.901513+0000","last_became_peered":"2024-03-29T09:08:56.901513+0000","last_unstale":"2024-03-29T09:09:01.414584+0000","last_undegraded":"2024-03-29T09:09:01.414584+0000","last_fullsized":"2024-03-29T09:09:01.414584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:55.597965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527664+0000","last_change":"2024-03-29T09:08:57.406785+0000","last_active":"2024-03-29T09:09:01.527664+0000","last_peered":"2024-03-29T09:09:01.527664+0000","last_clean":"2024-03-29T09:09:01.527664+0000","last_became_active":"2024-03-29T09:08:57.406261+0000","last_became_peered":"2024-03-29T09:08:57.406261+0000","last_unstale":"2024-03-29T09:09:01.527664+0000","last_undegraded":"2024-03-29T09:09:01.527664+0000","last_fullsized":"2024-03-29T09:09:01.527664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:09:43.625475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530845+0000","last_change":"2024-03-29T09:09:01.445228+0000","last_active":"2024-03-29T09:09:01.530845+0000","last_peered":"2024-03-29T09:09:01.530845+0000","last_clean":"2024-03-29T09:09:01.530845+0000","last_became_active":"2024-03-29T09:09:01.444917+0000","last_became_peered":"2024-03-29T09:09:01.444917+0000","last_unstale":"2024-03-29T09:09:01.530845+0000","last_undegraded":"2024-03-29T09:09:01.530845+0000","last_fullsized":"2024-03-29T09:09:01.530845+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:00.179094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.431916+0000","last_change":"2024-03-29T09:08:56.914015+0000","last_active":"2024-03-29T09:09:01.431916+0000","last_peered":"2024-03-29T09:09:01.431916+0000","last_clean":"2024-03-29T09:09:01.431916+0000","last_became_active":"2024-03-29T09:08:56.913673+0000","last_became_peered":"2024-03-29T09:08:56.913673+0000","last_unstale":"2024-03-29T09:09:01.431916+0000","last_undegraded":"2024-03-29T09:09:01.431916+0000","last_fullsized":"2024-03-29T09:09:01.431916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:00.945239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527811+0000","last_change":"2024-03-29T09:08:57.410999+0000","last_active":"2024-03-29T09:09:01.527811+0000","last_peered":"2024-03-29T09:09:01.527811+0000","last_clean":"2024-03-29T09:09:01.527811+0000","last_became_active":"2024-03-29T09:08:57.410742+0000","last_became_peered":"2024-03-29T09:08:57.410742+0000","last_unstale":"2024-03-29T09:09:01.527811+0000","last_undegraded":"2024-03-29T09:09:01.527811+0000","last_fullsized":"2024-03-29T09:09:01.527811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:24.187192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530767+0000","last_change":"2024-03-29T09:09:01.434584+0000","last_active":"2024-03-29T09:09:01.530767+0000","last_peered":"2024-03-29T09:09:01.530767+0000","last_clean":"2024-03-29T09:09:01.530767+0000","last_became_active":"2024-03-29T09:09:01.434233+0000","last_became_peered":"2024-03-29T09:09:01.434233+0000","last_unstale":"2024-03-29T09:09:01.530767+0000","last_undegraded":"2024-03-29T09:09:01.530767+0000","last_fullsized":"2024-03-29T09:09:01.530767+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:18:14.395090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.420352+0000","last_change":"2024-03-29T09:08:56.908753+0000","last_active":"2024-03-29T09:09:01.420352+0000","last_peered":"2024-03-29T09:09:01.420352+0000","last_clean":"2024-03-29T09:09:01.420352+0000","last_became_active":"2024-03-29T09:08:56.908505+0000","last_became_peered":"2024-03-29T09:08:56.908505+0000","last_unstale":"2024-03-29T09:09:01.420352+0000","last_undegraded":"2024-03-29T09:09:01.420352+0000","last_fullsized":"2024-03-29T09:09:01.420352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:27.280372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.533999+0000","last_change":"2024-03-29T09:08:57.418006+0000","last_active":"2024-03-29T09:09:01.533999+0000","last_peered":"2024-03-29T09:09:01.533999+0000","last_clean":"2024-03-29T09:09:01.533999+0000","last_became_active":"2024-03-29T09:08:57.417644+0000","last_became_peered":"2024-03-29T09:08:57.417644+0000","last_unstale":"2024-03-29T09:09:01.533999+0000","last_undegraded":"2024-03-29T09:09:01.533999+0000","last_fullsized":"2024-03-29T09:09:01.533999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:16.357271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527566+0000","last_change":"2024-03-29T09:09:01.431602+0000","last_active":"2024-03-29T09:09:01.527566+0000","last_peered":"2024-03-29T09:09:01.527566+0000","last_clean":"2024-03-29T09:09:01.527566+0000","last_became_active":"2024-03-29T09:09:01.430422+0000","last_became_peered":"2024-03-29T09:09:01.430422+0000","last_unstale":"2024-03-29T09:09:01.527566+0000","last_undegraded":"2024-03-29T09:09:01.527566+0000","last_fullsized":"2024-03-29T09:09:01.527566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:09:00.398719+0000","last_clean_scrub_stamp":"2024-03-29T09:09:00.398719+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:21:33.289630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.429605+0000","last_change":"2024-03-29T09:08:56.907602+0000","last_active":"2024-03-29T09:09:01.429605+0000","last_peered":"2024-03-29T09:09:01.429605+0000","last_clean":"2024-03-29T09:09:01.429605+0000","last_became_active":"2024-03-29T09:08:56.907321+0000","last_became_peered":"2024-03-29T09:08:56.907321+0000","last_unstale":"2024-03-29T09:09:01.429605+0000","last_undegraded":"2024-03-29T09:09:01.429605+0000","last_fullsized":"2024-03-29T09:09:01.429605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:55.868083+0000","last_clean_scrub_stamp":"2024-03-29T09:08:55.868083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:15.169017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534106+0000","last_change":"2024-03-29T09:08:57.419001+0000","last_active":"2024-03-29T09:09:01.534106+0000","last_peered":"2024-03-29T09:09:01.534106+0000","last_clean":"2024-03-29T09:09:01.534106+0000","last_became_active":"2024-03-29T09:08:57.418769+0000","last_became_peered":"2024-03-29T09:08:57.418769+0000","last_unstale":"2024-03-29T09:09:01.534106+0000","last_undegraded":"2024-03-29T09:09:01.534106+0000","last_fullsized":"2024-03-29T09:09:01.534106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:53.889081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530742+0000","last_change":"2024-03-29T09:08:57.408417+0000","last_active":"2024-03-29T09:09:01.530742+0000","last_peered":"2024-03-29T09:09:01.530742+0000","last_clean":"2024-03-29T09:09:01.530742+0000","last_became_active":"2024-03-29T09:08:57.408131+0000","last_became_peered":"2024-03-29T09:08:57.408131+0000","last_unstale":"2024-03-29T09:09:01.530742+0000","last_undegraded":"2024-03-29T09:09:01.530742+0000","last_fullsized":"2024-03-29T09:09:01.530742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:52.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527306+0000","last_change":"2024-03-29T09:08:57.418220+0000","last_active":"2024-03-29T09:09:01.527306+0000","last_peered":"2024-03-29T09:09:01.527306+0000","last_clean":"2024-03-29T09:09:01.527306+0000","last_became_active":"2024-03-29T09:08:57.417900+0000","last_became_peered":"2024-03-29T09:08:57.417900+0000","last_unstale":"2024-03-29T09:09:01.527306+0000","last_undegraded":"2024-03-29T09:09:01.527306+0000","last_fullsized":"2024-03-29T09:09:01.527306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:16.177048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527708+0000","last_change":"2024-03-29T09:08:57.424647+0000","last_active":"2024-03-29T09:09:01.527708+0000","last_peered":"2024-03-29T09:09:01.527708+0000","last_clean":"2024-03-29T09:09:01.527708+0000","last_became_active":"2024-03-29T09:08:57.424404+0000","last_became_peered":"2024-03-29T09:08:57.424404+0000","last_unstale":"2024-03-29T09:09:01.527708+0000","last_undegraded":"2024-03-29T09:09:01.527708+0000","last_fullsized":"2024-03-29T09:09:01.527708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:06:29.651626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530787+0000","last_change":"2024-03-29T09:08:57.419276+0000","last_active":"2024-03-29T09:09:01.530787+0000","last_peered":"2024-03-29T09:09:01.530787+0000","last_clean":"2024-03-29T09:09:01.530787+0000","last_became_active":"2024-03-29T09:08:57.418974+0000","last_became_peered":"2024-03-29T09:08:57.418974+0000","last_unstale":"2024-03-29T09:09:01.530787+0000","last_undegraded":"2024-03-29T09:09:01.530787+0000","last_fullsized":"2024-03-29T09:09:01.530787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:23.817066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525521+0000","last_change":"2024-03-29T09:08:57.429156+0000","last_active":"2024-03-29T09:09:01.525521+0000","last_peered":"2024-03-29T09:09:01.525521+0000","last_clean":"2024-03-29T09:09:01.525521+0000","last_became_active":"2024-03-29T09:08:57.428873+0000","last_became_peered":"2024-03-29T09:08:57.428873+0000","last_unstale":"2024-03-29T09:09:01.525521+0000","last_undegraded":"2024-03-29T09:09:01.525521+0000","last_fullsized":"2024-03-29T09:09:01.525521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:10:37.361769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.525556+0000","last_change":"2024-03-29T09:08:57.423650+0000","last_active":"2024-03-29T09:09:01.525556+0000","last_peered":"2024-03-29T09:09:01.525556+0000","last_clean":"2024-03-29T09:09:01.525556+0000","last_became_active":"2024-03-29T09:08:57.423385+0000","last_became_peered":"2024-03-29T09:08:57.423385+0000","last_unstale":"2024-03-29T09:09:01.525556+0000","last_undegraded":"2024-03-29T09:09:01.525556+0000","last_fullsized":"2024-03-29T09:09:01.525556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:07:25.078600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.531076+0000","last_change":"2024-03-29T09:08:57.413866+0000","last_active":"2024-03-29T09:09:01.531076+0000","last_peered":"2024-03-29T09:09:01.531076+0000","last_clean":"2024-03-29T09:09:01.531076+0000","last_became_active":"2024-03-29T09:08:57.413483+0000","last_became_peered":"2024-03-29T09:08:57.413483+0000","last_unstale":"2024-03-29T09:09:01.531076+0000","last_undegraded":"2024-03-29T09:09:01.531076+0000","last_fullsized":"2024-03-29T09:09:01.531076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530897+0000","last_change":"2024-03-29T09:08:57.422855+0000","last_active":"2024-03-29T09:09:01.530897+0000","last_peered":"2024-03-29T09:09:01.530897+0000","last_clean":"2024-03-29T09:09:01.530897+0000","last_became_active":"2024-03-29T09:08:57.422350+0000","last_became_peered":"2024-03-29T09:08:57.422350+0000","last_unstale":"2024-03-29T09:09:01.530897+0000","last_undegraded":"2024-03-29T09:09:01.530897+0000","last_fullsized":"2024-03-29T09:09:01.530897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:02.669666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527637+0000","last_change":"2024-03-29T09:08:57.427109+0000","last_active":"2024-03-29T09:09:01.527637+0000","last_peered":"2024-03-29T09:09:01.527637+0000","last_clean":"2024-03-29T09:09:01.527637+0000","last_became_active":"2024-03-29T09:08:57.424525+0000","last_became_peered":"2024-03-29T09:08:57.424525+0000","last_unstale":"2024-03-29T09:09:01.527637+0000","last_undegraded":"2024-03-29T09:09:01.527637+0000","last_fullsized":"2024-03-29T09:09:01.527637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:52.774904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.530755+0000","last_change":"2024-03-29T09:08:57.416922+0000","last_active":"2024-03-29T09:09:01.530755+0000","last_peered":"2024-03-29T09:09:01.530755+0000","last_clean":"2024-03-29T09:09:01.530755+0000","last_became_active":"2024-03-29T09:08:57.415900+0000","last_became_peered":"2024-03-29T09:08:57.415900+0000","last_unstale":"2024-03-29T09:09:01.530755+0000","last_undegraded":"2024-03-29T09:09:01.530755+0000","last_fullsized":"2024-03-29T09:09:01.530755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.601067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.528724+0000","last_change":"2024-03-29T09:08:57.416177+0000","last_active":"2024-03-29T09:09:01.528724+0000","last_peered":"2024-03-29T09:09:01.528724+0000","last_clean":"2024-03-29T09:09:01.528724+0000","last_became_active":"2024-03-29T09:08:57.415844+0000","last_became_peered":"2024-03-29T09:08:57.415844+0000","last_unstale":"2024-03-29T09:09:01.528724+0000","last_undegraded":"2024-03-29T09:09:01.528724+0000","last_fullsized":"2024-03-29T09:09:01.528724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:28.813187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.527479+0000","last_change":"2024-03-29T09:08:57.411579+0000","last_active":"2024-03-29T09:09:01.527479+0000","last_peered":"2024-03-29T09:09:01.527479+0000","last_clean":"2024-03-29T09:09:01.527479+0000","last_became_active":"2024-03-29T09:08:57.411297+0000","last_became_peered":"2024-03-29T09:08:57.411297+0000","last_unstale":"2024-03-29T09:09:01.527479+0000","last_undegraded":"2024-03-29T09:09:01.527479+0000","last_fullsized":"2024-03-29T09:09:01.527479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:56:13.990570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-29T09:09:01.534013+0000","last_change":"2024-03-29T09:08:57.417300+0000","last_active":"2024-03-29T09:09:01.534013+0000","last_peered":"2024-03-29T09:09:01.534013+0000","last_clean":"2024-03-29T09:09:01.534013+0000","last_became_active":"2024-03-29T09:08:57.416733+0000","last_became_peered":"2024-03-29T09:08:57.416733+0000","last_unstale":"2024-03-29T09:09:01.534013+0000","last_undegraded":"2024-03-29T09:09:01.534013+0000","last_fullsized":"2024-03-29T09:09:01.534013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:56.876114+0000","last_clean_scrub_stamp":"2024-03-29T09:08:56.876114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:51.863125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4594,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":99788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":690226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":99844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":100370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":104256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":690248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:09:32.457 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T09:09:32.457 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T09:09:32.457 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-29T09:09:32.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T09:09:32.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.680+0000 7f920e200700 1 Processor -- start 2024-03-29T09:09:32.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.680+0000 7f920e200700 1 -- start start 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f920e200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92080d1760 0x7f92080d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f920e200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92080afdb0 0x7f92080b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f920e200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92080d2080 con 0x7f92080d0d50 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f920e200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92080d15f0 con 0x7f92080d1760 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f920e200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92080d0760 con 0x7f92080afdb0 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f9207400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92080d1760 0x7f92080d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f9206a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f92080d0d50 0x7f92080d3ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49734/0 (socket says 172.21.15.99:49734) 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f9206a00700 1 -- 172.21.15.99:0/1404115696 learned_addr learned my addr 172.21.15.99:0/1404115696 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:32.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.682+0000 7f9207e00700 1 --2- 172.21.15.99:0/1404115696 >> v2:172.21.15.133:3300/0 conn(0x7f92080afdb0 0x7f92080b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9207400700 1 -- 172.21.15.99:0/1404115696 >> v2:172.21.15.133:3300/0 conn(0x7f92080afdb0 msgr2=0x7f92080b0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9207400700 1 --2- 172.21.15.99:0/1404115696 >> v2:172.21.15.133:3300/0 conn(0x7f92080afdb0 0x7f92080b0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9207400700 1 -- 172.21.15.99:0/1404115696 >> v1:172.21.15.99:6789/0 conn(0x7f92080d0d50 legacy=0x7f92080d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9207400700 1 -- 172.21.15.99:0/1404115696 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92080d1310 con 0x7f92080d1760 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9206000700 1 -- 172.21.15.99:0/1404115696 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414164683 0 0) 0x7f92080d2080 con 0x7f92080d0d50 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9207400700 1 --2- 172.21.15.99:0/1404115696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080d1760 0x7f92080d1b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92080b4c10 tx=0x7f91f8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56599dded1b21ea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9206000700 1 -- 172.21.15.99:0/1404115696 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8011830 con 0x7f92080d1760 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9206000700 1 -- 172.21.15.99:0/1404115696 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f8011990 con 0x7f92080d1760 2024-03-29T09:09:32.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.683+0000 7f9206000700 1 -- 172.21.15.99:0/1404115696 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8011d20 con 0x7f92080d1760 2024-03-29T09:09:32.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 -- 172.21.15.99:0/1404115696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080d1760 msgr2=0x7f92080d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 --2- 172.21.15.99:0/1404115696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080d1760 0x7f92080d1b40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f92080b4c10 tx=0x7f91f8009b20 comp rx=0 tx=0).stop 2024-03-29T09:09:32.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 -- 172.21.15.99:0/1404115696 shutdown_connections 2024-03-29T09:09:32.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 --2- 172.21.15.99:0/1404115696 >> v2:172.21.15.133:3300/0 conn(0x7f92080afdb0 0x7f92080b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 --2- 172.21.15.99:0/1404115696 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080d1760 0x7f92080d1b40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 -- 172.21.15.99:0/1404115696 >> 172.21.15.99:0/1404115696 conn(0x7f920809c4c0 msgr2=0x7f92080d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:32.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 -- 172.21.15.99:0/1404115696 shutdown_connections 2024-03-29T09:09:32.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.684+0000 7f920e200700 1 -- 172.21.15.99:0/1404115696 wait complete. 2024-03-29T09:09:32.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.686+0000 7f920e200700 1 Processor -- start 2024-03-29T09:09:32.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.686+0000 7f920e200700 1 -- start start 2024-03-29T09:09:32.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f920e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080afdb0 0x7f9208155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f920e200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92080d0d50 0x7f92080b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f920e200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92080d1760 0x7f92080b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f920e200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9208155c60 con 0x7f92080afdb0 2024-03-29T09:09:32.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f920e200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92080b8e20 con 0x7f92080d1760 2024-03-29T09:09:32.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f920e200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92080b7840 con 0x7f92080d0d50 2024-03-29T09:09:32.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080afdb0 0x7f9208155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080afdb0 0x7f9208155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53522/0 (socket says 172.21.15.99:53522) 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 -- 172.21.15.99:0/733476043 learned_addr learned my addr 172.21.15.99:0/733476043 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207e00700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92080d1760 0x7f92080b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9206a00700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92080d0d50 0x7f92080b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 -- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92080d0d50 msgr2=0x7f92080b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92080d0d50 0x7f92080b88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 -- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92080d1760 msgr2=0x7f92080b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92080d1760 0x7f92080b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207400700 1 -- 172.21.15.99:0/733476043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f80094a0 con 0x7f92080afdb0 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9207e00700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92080d1760 0x7f92080b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.687+0000 7f9206a00700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92080d0d50 0x7f92080b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.688+0000 7f9207400700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080afdb0 0x7f9208155720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f920808ca20 tx=0x7f91f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.688+0000 7f91ff400700 1 -- 172.21.15.99:0/733476043 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80038a0 con 0x7f92080afdb0 2024-03-29T09:09:32.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.688+0000 7f91ff400700 1 -- 172.21.15.99:0/733476043 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f801f070 con 0x7f92080afdb0 2024-03-29T09:09:32.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.688+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92080b7b20 con 0x7f92080afdb0 2024-03-29T09:09:32.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.688+0000 7f91ff400700 1 -- 172.21.15.99:0/733476043 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8003c40 con 0x7f92080afdb0 2024-03-29T09:09:32.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.688+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92080b80e0 con 0x7f92080afdb0 2024-03-29T09:09:32.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.689+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92080678b0 con 0x7f92080afdb0 2024-03-29T09:09:32.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.690+0000 7f91ff400700 1 -- 172.21.15.99:0/733476043 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f91f8026480 con 0x7f92080afdb0 2024-03-29T09:09:32.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.691+0000 7f91ff400700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f91e806ac90 0x7f91e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:32.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.691+0000 7f91ff400700 1 -- 172.21.15.99:0/733476043 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f91f8021070 con 0x7f92080afdb0 2024-03-29T09:09:32.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.691+0000 7f9206a00700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f91e806ac90 0x7f91e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:32.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.691+0000 7f9206a00700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f91e806ac90 0x7f91e806d150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91f4000f60 tx=0x7f91f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:32.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.695+0000 7f91ff400700 1 -- 172.21.15.99:0/733476043 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f91f8055ed0 con 0x7f92080afdb0 2024-03-29T09:09:32.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.945+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f92080b2650 con 0x7f92080afdb0 2024-03-29T09:09:32.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.946+0000 7f91ff400700 1 -- 172.21.15.99:0/733476043 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f91f802b080 con 0x7f92080afdb0 2024-03-29T09:09:32.945 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:32.945 INFO:teuthology.orchestra.run.smithi099.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T09:09:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f91e806ac90 msgr2=0x7f91e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f91e806ac90 0x7f91e806d150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91f4000f60 tx=0x7f91f4009040 comp rx=0 tx=0).stop 2024-03-29T09:09:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080afdb0 msgr2=0x7f9208155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:32.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080afdb0 0x7f9208155720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f920808ca20 tx=0x7f91f8004750 comp rx=0 tx=0).stop 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 shutdown_connections 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f91e806ac90 0x7f91e806d150 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92080d1760 0x7f92080b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92080afdb0 0x7f9208155720 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 --2- 172.21.15.99:0/733476043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92080d0d50 0x7f92080b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 >> 172.21.15.99:0/733476043 conn(0x7f920809c4c0 msgr2=0x7f92080b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 shutdown_connections 2024-03-29T09:09:32.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:32.948+0000 7f920e200700 1 -- 172.21.15.99:0/733476043 wait complete. 2024-03-29T09:09:32.965 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-29T09:09:32.966 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:33.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.156+0000 7f1a0ba00700 1 Processor -- start 2024-03-29T09:09:33.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.156+0000 7f1a0ba00700 1 -- start start 2024-03-29T09:09:33.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.157+0000 7f1a0ba00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1a041608e0 0x7f1a040d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.157+0000 7f1a0ba00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a04160cc0 0x7f1a040ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.157+0000 7f1a0ba00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a040d3a70 0x7f1a040d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.157+0000 7f1a0ba00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a040cddc0 con 0x7f1a04160cc0 2024-03-29T09:09:33.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a0ba00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a040ce0a0 con 0x7f1a041608e0 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a0ba00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040ce380 con 0x7f1a040d3a70 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a04160cc0 0x7f1a040ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a04160cc0 0x7f1a040ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53534/0 (socket says 172.21.15.99:53534) 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 -- 172.21.15.99:0/386135667 learned_addr learned my addr 172.21.15.99:0/386135667 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a09000700 1 --2- 172.21.15.99:0/386135667 >> v2:172.21.15.133:3300/0 conn(0x7f1a041608e0 0x7f1a040d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a09a00700 1 --2- 172.21.15.99:0/386135667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a040d3a70 0x7f1a040d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 -- 172.21.15.99:0/386135667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a040d3a70 msgr2=0x7f1a040d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 --2- 172.21.15.99:0/386135667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a040d3a70 0x7f1a040d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 -- 172.21.15.99:0/386135667 >> v2:172.21.15.133:3300/0 conn(0x7f1a041608e0 msgr2=0x7f1a040d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:33.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 --2- 172.21.15.99:0/386135667 >> v2:172.21.15.133:3300/0 conn(0x7f1a041608e0 0x7f1a040d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.158+0000 7f1a03e00700 1 -- 172.21.15.99:0/386135667 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040ce660 con 0x7f1a04160cc0 2024-03-29T09:09:33.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a03e00700 1 --2- 172.21.15.99:0/386135667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a04160cc0 0x7f1a040ceb80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a0408d790 tx=0x7f19f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1322c2b751b084d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a03400700 1 -- 172.21.15.99:0/386135667 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0011830 con 0x7f1a04160cc0 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a03400700 1 -- 172.21.15.99:0/386135667 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f0011990 con 0x7f1a04160cc0 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a03400700 1 -- 172.21.15.99:0/386135667 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0011d20 con 0x7f1a04160cc0 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 -- 172.21.15.99:0/386135667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a04160cc0 msgr2=0x7f1a040ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 --2- 172.21.15.99:0/386135667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a04160cc0 0x7f1a040ceb80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1a0408d790 tx=0x7f19f0009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 -- 172.21.15.99:0/386135667 shutdown_connections 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 --2- 172.21.15.99:0/386135667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a040d3a70 0x7f1a040d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 --2- 172.21.15.99:0/386135667 >> v2:172.21.15.133:3300/0 conn(0x7f1a041608e0 0x7f1a040d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 --2- 172.21.15.99:0/386135667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a04160cc0 0x7f1a040ceb80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 -- 172.21.15.99:0/386135667 >> 172.21.15.99:0/386135667 conn(0x7f1a0409c490 msgr2=0x7f1a040d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.159+0000 7f1a0ba00700 1 -- 172.21.15.99:0/386135667 shutdown_connections 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.160+0000 7f1a0ba00700 1 -- 172.21.15.99:0/386135667 wait complete. 2024-03-29T09:09:33.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.161+0000 7f1a0ba00700 1 Processor -- start 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.161+0000 7f1a0ba00700 1 -- start start 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a0ba00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a040d3a70 0x7f1a040c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a0ba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a041608e0 0x7f1a041f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a0ba00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a04160cc0 0x7f1a041f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a0ba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04158f80 con 0x7f1a041608e0 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a0ba00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04158300 con 0x7f1a040d3a70 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a0ba00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a041585e0 con 0x7f1a04160cc0 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a03e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a041608e0 0x7f1a041f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a03e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a041608e0 0x7f1a041f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53548/0 (socket says 172.21.15.99:53548) 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.162+0000 7f1a03e00700 1 -- 172.21.15.99:0/1161567739 learned_addr learned my addr 172.21.15.99:0/1161567739 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a09000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a040d3a70 0x7f1a040c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a09a00700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a04160cc0 0x7f1a041f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a03e00700 1 -- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a04160cc0 msgr2=0x7f1a041f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a03e00700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a04160cc0 0x7f1a041f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a03e00700 1 -- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a040d3a70 msgr2=0x7f1a040c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a03e00700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a040d3a70 0x7f1a040c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a03e00700 1 -- 172.21.15.99:0/1161567739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f0009550 con 0x7f1a041608e0 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a09a00700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a04160cc0 0x7f1a041f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a03e00700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a041608e0 0x7f1a041f6150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009c10 tx=0x7f19f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a00c00700 1 -- 172.21.15.99:0/1161567739 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f0004510 con 0x7f1a041608e0 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a00c00700 1 -- 172.21.15.99:0/1161567739 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f001e070 con 0x7f1a041608e0 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a00c00700 1 -- 172.21.15.99:0/1161567739 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f00038e0 con 0x7f1a041608e0 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.163+0000 7f1a0ba00700 1 -- 172.21.15.99:0/1161567739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a041588c0 con 0x7f1a041608e0 2024-03-29T09:09:33.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.164+0000 7f1a0ba00700 1 -- 172.21.15.99:0/1161567739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a040c9930 con 0x7f1a041608e0 2024-03-29T09:09:33.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.165+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d80052f0 con 0x7f1a041608e0 2024-03-29T09:09:33.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.165+0000 7f1a00c00700 1 -- 172.21.15.99:0/1161567739 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f19f0003a40 con 0x7f1a041608e0 2024-03-29T09:09:33.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.166+0000 7f1a00c00700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19e406ac90 0x7f19e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.166+0000 7f1a00c00700 1 -- 172.21.15.99:0/1161567739 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f19f008a500 con 0x7f1a041608e0 2024-03-29T09:09:33.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.166+0000 7f1a09000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19e406ac90 0x7f19e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.167+0000 7f1a09000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19e406ac90 0x7f19e406d150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009af0 tx=0x7f19f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:33.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.171+0000 7f1a00c00700 1 -- 172.21.15.99:0/1161567739 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f19f0055350 con 0x7f1a041608e0 2024-03-29T09:09:33.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.376+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19d80055a0 con 0x7f1a041608e0 2024-03-29T09:09:33.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.377+0000 7f1a00c00700 1 -- 172.21.15.99:0/1161567739 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f19f0058970 con 0x7f1a041608e0 2024-03-29T09:09:33.376 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:33.376 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:33.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19e406ac90 msgr2=0x7f19e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19e406ac90 0x7f19e406d150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009af0 tx=0x7f19f8008040 comp rx=0 tx=0).stop 2024-03-29T09:09:33.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a041608e0 msgr2=0x7f1a041f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a041608e0 0x7f1a041f6150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009c10 tx=0x7f19f0004750 comp rx=0 tx=0).stop 2024-03-29T09:09:33.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 shutdown_connections 2024-03-29T09:09:33.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19e406ac90 0x7f19e406d150 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a04160cc0 0x7f1a041f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a040d3a70 0x7f1a040c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 --2- 172.21.15.99:0/1161567739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a041608e0 0x7f1a041f6150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 >> 172.21.15.99:0/1161567739 conn(0x7f1a0409c490 msgr2=0x7f1a040b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:33.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 shutdown_connections 2024-03-29T09:09:33.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.380+0000 7f19f6000700 1 -- 172.21.15.99:0/1161567739 wait complete. 2024-03-29T09:09:33.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:09:33.398 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:09:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.593+0000 7f676a200700 1 Processor -- start 2024-03-29T09:09:33.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.593+0000 7f676a200700 1 -- start start 2024-03-29T09:09:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.594+0000 7f676a200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6764160770 0x7f67640b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.594+0000 7f676a200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67640d3550 0x7f67640ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.594+0000 7f676a200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67640d1520 con 0x7f67640d0d60 2024-03-29T09:09:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.594+0000 7f676a200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67640d1800 con 0x7f6764160770 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.594+0000 7f676a200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67640d1ae0 con 0x7f67640d3550 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6764160770 0x7f67640b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6764160770 0x7f67640b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53558/0 (socket says 172.21.15.99:53558) 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 -- 172.21.15.99:0/866049382 learned_addr learned my addr 172.21.15.99:0/866049382 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6762a00700 1 --1- 172.21.15.99:0/866049382 >> v1:172.21.15.99:6789/0 conn(0x7f67640d0d60 0x7f67640d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49744/0 (socket says 172.21.15.99:49744) 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763e00700 1 --2- 172.21.15.99:0/866049382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67640d3550 0x7f67640ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 -- 172.21.15.99:0/866049382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67640d3550 msgr2=0x7f67640ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 --2- 172.21.15.99:0/866049382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67640d3550 0x7f67640ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 -- 172.21.15.99:0/866049382 >> v1:172.21.15.99:6789/0 conn(0x7f67640d0d60 legacy=0x7f67640d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 -- 172.21.15.99:0/866049382 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67640d1dc0 con 0x7f6764160770 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.595+0000 7f6763400700 1 --2- 172.21.15.99:0/866049382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6764160770 0x7f67640b3cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f67540098f0 tx=0x7f6754009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba33f306e863a136 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f6762000700 1 -- 172.21.15.99:0/866049382 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400b8a0 con 0x7f6764160770 2024-03-29T09:09:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f6762000700 1 -- 172.21.15.99:0/866049382 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675400ba00 con 0x7f6764160770 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f6762000700 1 -- 172.21.15.99:0/866049382 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675400bd90 con 0x7f6764160770 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 -- 172.21.15.99:0/866049382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6764160770 msgr2=0x7f67640b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 --2- 172.21.15.99:0/866049382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6764160770 0x7f67640b3cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f67540098f0 tx=0x7f6754009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 -- 172.21.15.99:0/866049382 shutdown_connections 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 --2- 172.21.15.99:0/866049382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67640d3550 0x7f67640ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 --2- 172.21.15.99:0/866049382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6764160770 0x7f67640b3cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 -- 172.21.15.99:0/866049382 >> 172.21.15.99:0/866049382 conn(0x7f676408afd0 msgr2=0x7f676409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 -- 172.21.15.99:0/866049382 shutdown_connections 2024-03-29T09:09:33.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.596+0000 7f676a200700 1 -- 172.21.15.99:0/866049382 wait complete. 2024-03-29T09:09:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.598+0000 7f676a200700 1 Processor -- start 2024-03-29T09:09:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.598+0000 7f676a200700 1 -- start start 2024-03-29T09:09:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f676a200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67640d0d60 0x7f6764152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f676a200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67640d3550 0x7f67641f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f676a200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6764160770 0x7f67641f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f676a200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67640cd100 con 0x7f67640d0d60 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f676a200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641f28f0 con 0x7f67640d3550 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f676a200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67641f6ea0 con 0x7f6764160770 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f6763400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67640d0d60 0x7f6764152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f6763400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67640d0d60 0x7f6764152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53568/0 (socket says 172.21.15.99:53568) 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f6763400700 1 -- 172.21.15.99:0/1598683252 learned_addr learned my addr 172.21.15.99:0/1598683252 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f6762a00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67640d3550 0x7f67641f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.599+0000 7f6763e00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6764160770 0x7f67641f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f6763400700 1 -- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6764160770 msgr2=0x7f67641f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f6763400700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6764160770 0x7f67641f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f6763400700 1 -- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67640d3550 msgr2=0x7f67641f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f6763400700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67640d3550 0x7f67641f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f6763400700 1 -- 172.21.15.99:0/1598683252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754009550 con 0x7f67640d0d60 2024-03-29T09:09:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f6763e00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6764160770 0x7f67641f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f6763400700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67640d0d60 0x7f6764152710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6754009c40 tx=0x7f6754004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f675b400700 1 -- 172.21.15.99:0/1598683252 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754004590 con 0x7f67640d0d60 2024-03-29T09:09:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f675b400700 1 -- 172.21.15.99:0/1598683252 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675401e070 con 0x7f67640d0d60 2024-03-29T09:09:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f676a200700 1 -- 172.21.15.99:0/1598683252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67640ccb40 con 0x7f67640d0d60 2024-03-29T09:09:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f675b400700 1 -- 172.21.15.99:0/1598683252 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540038e0 con 0x7f67640d0d60 2024-03-29T09:09:33.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.600+0000 7f676a200700 1 -- 172.21.15.99:0/1598683252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67641f6160 con 0x7f67640d0d60 2024-03-29T09:09:33.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.601+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67380052f0 con 0x7f67640d0d60 2024-03-29T09:09:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.602+0000 7f675b400700 1 -- 172.21.15.99:0/1598683252 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6754003a40 con 0x7f67640d0d60 2024-03-29T09:09:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.603+0000 7f675b400700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f674406ac90 0x7f674406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.603+0000 7f675b400700 1 -- 172.21.15.99:0/1598683252 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6754020070 con 0x7f67640d0d60 2024-03-29T09:09:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.603+0000 7f6762a00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f674406ac90 0x7f674406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.604+0000 7f6762a00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f674406ac90 0x7f674406d150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6750000f60 tx=0x7f6750007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:33.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.607+0000 7f675b400700 1 -- 172.21.15.99:0/1598683252 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6754055980 con 0x7f67640d0d60 2024-03-29T09:09:33.806 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.807+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6738005b20 con 0x7f67640d0d60 2024-03-29T09:09:33.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.808+0000 7f675b400700 1 -- 172.21.15.99:0/1598683252 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67540150a0 con 0x7f67640d0d60 2024-03-29T09:09:33.807 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:09:33.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f674406ac90 msgr2=0x7f674406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f674406ac90 0x7f674406d150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6750000f60 tx=0x7f6750007ed0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67640d0d60 msgr2=0x7f6764152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:33.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67640d0d60 0x7f6764152710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6754009c40 tx=0x7f6754004750 comp rx=0 tx=0).stop 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 shutdown_connections 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f674406ac90 0x7f674406d150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6764160770 0x7f67641f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67640d0d60 0x7f6764152710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 --2- 172.21.15.99:0/1598683252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67640d3550 0x7f67641f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 >> 172.21.15.99:0/1598683252 conn(0x7f676408afd0 msgr2=0x7f6764073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 shutdown_connections 2024-03-29T09:09:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:33.810+0000 7f6758c00700 1 -- 172.21.15.99:0/1598683252 wait complete. 2024-03-29T09:09:33.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:09:34.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.021+0000 7f966be00700 1 Processor -- start 2024-03-29T09:09:34.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.021+0000 7f966be00700 1 -- start start 2024-03-29T09:09:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f966be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96640b3cc0 0x7f96640d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f966be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96640d3380 0x7f96640ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f966be00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96640d3810 con 0x7f9664160770 2024-03-29T09:09:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f966be00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96640b4140 con 0x7f96640d3380 2024-03-29T09:09:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f966be00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640d1520 con 0x7f96640b3cc0 2024-03-29T09:09:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f9669e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f96640d3380 0x7f96640ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f9669400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9664160770 0x7f96640d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49752/0 (socket says 172.21.15.99:49752) 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.022+0000 7f9669400700 1 -- 172.21.15.99:0/1834152312 learned_addr learned my addr 172.21.15.99:0/1834152312 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9668a00700 1 --2- 172.21.15.99:0/1834152312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96640b3cc0 0x7f96640d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9669e00700 1 -- 172.21.15.99:0/1834152312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96640b3cc0 msgr2=0x7f96640d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9669e00700 1 --2- 172.21.15.99:0/1834152312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96640b3cc0 0x7f96640d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9669e00700 1 -- 172.21.15.99:0/1834152312 >> v1:172.21.15.99:6789/0 conn(0x7f9664160770 legacy=0x7f96640d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9669e00700 1 -- 172.21.15.99:0/1834152312 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640d1800 con 0x7f96640d3380 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9663e00700 1 -- 172.21.15.99:0/1834152312 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424540459 0 0) 0x7f96640d3810 con 0x7f9664160770 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9669e00700 1 --2- 172.21.15.99:0/1834152312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640d3380 0x7f96640ce080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f965c009d90 tx=0x7f965c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e981b34ade66a8b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9663e00700 1 -- 172.21.15.99:0/1834152312 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c00e070 con 0x7f96640d3380 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9663e00700 1 -- 172.21.15.99:0/1834152312 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965c00bbd0 con 0x7f96640d3380 2024-03-29T09:09:34.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.023+0000 7f9663e00700 1 -- 172.21.15.99:0/1834152312 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c012430 con 0x7f96640d3380 2024-03-29T09:09:34.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 -- 172.21.15.99:0/1834152312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640d3380 msgr2=0x7f96640ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 --2- 172.21.15.99:0/1834152312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640d3380 0x7f96640ce080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f965c009d90 tx=0x7f965c00b330 comp rx=0 tx=0).stop 2024-03-29T09:09:34.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 -- 172.21.15.99:0/1834152312 shutdown_connections 2024-03-29T09:09:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 --2- 172.21.15.99:0/1834152312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640d3380 0x7f96640ce080 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 --2- 172.21.15.99:0/1834152312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96640b3cc0 0x7f96640d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 -- 172.21.15.99:0/1834152312 >> 172.21.15.99:0/1834152312 conn(0x7f966408afd0 msgr2=0x7f966409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 -- 172.21.15.99:0/1834152312 shutdown_connections 2024-03-29T09:09:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.024+0000 7f966be00700 1 -- 172.21.15.99:0/1834152312 wait complete. 2024-03-29T09:09:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.026+0000 7f966be00700 1 Processor -- start 2024-03-29T09:09:34.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.026+0000 7f966be00700 1 -- start start 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f966be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640b3cc0 0x7f96640b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f966be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96640d3380 0x7f96640b8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f966be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9664160770 0x7f966415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f966be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640b06b0 con 0x7f96640b3cc0 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f966be00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96640afdc0 con 0x7f96640d3380 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f966be00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640b8f10 con 0x7f9664160770 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640b3cc0 0x7f96640b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640b3cc0 0x7f96640b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53592/0 (socket says 172.21.15.99:53592) 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 -- 172.21.15.99:0/327092831 learned_addr learned my addr 172.21.15.99:0/327092831 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9668a00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96640d3380 0x7f96640b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669e00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9664160770 0x7f966415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 -- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9664160770 msgr2=0x7f966415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9664160770 0x7f966415d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 -- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96640d3380 msgr2=0x7f96640b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96640d3380 0x7f96640b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.027+0000 7f9669400700 1 -- 172.21.15.99:0/327092831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c009a70 con 0x7f96640b3cc0 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.028+0000 7f9669400700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640b3cc0 0x7f96640b0b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f965800c550 tx=0x7f965800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.028+0000 7f9661600700 1 -- 172.21.15.99:0/327092831 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658004c80 con 0x7f96640b3cc0 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.028+0000 7f9661600700 1 -- 172.21.15.99:0/327092831 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9658004e00 con 0x7f96640b3cc0 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.028+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96640b0060 con 0x7f96640b3cc0 2024-03-29T09:09:34.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.028+0000 7f9661600700 1 -- 172.21.15.99:0/327092831 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658007310 con 0x7f96640b3cc0 2024-03-29T09:09:34.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.028+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966415de20 con 0x7f96640b3cc0 2024-03-29T09:09:34.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.029+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96640678b0 con 0x7f96640b3cc0 2024-03-29T09:09:34.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.030+0000 7f9661600700 1 -- 172.21.15.99:0/327092831 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9658007470 con 0x7f96640b3cc0 2024-03-29T09:09:34.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.030+0000 7f9661600700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f964406ac90 0x7f964406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.031+0000 7f9661600700 1 -- 172.21.15.99:0/327092831 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9658089320 con 0x7f96640b3cc0 2024-03-29T09:09:34.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.031+0000 7f9668a00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f964406ac90 0x7f964406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.030 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.031+0000 7f9668a00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f964406ac90 0x7f964406d150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96640b18a0 tx=0x7f9650008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.035+0000 7f9661600700 1 -- 172.21.15.99:0/327092831 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9658054210 con 0x7f96640b3cc0 2024-03-29T09:09:34.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.240+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96640d1240 con 0x7f96640b3cc0 2024-03-29T09:09:34.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.240+0000 7f9661600700 1 -- 172.21.15.99:0/327092831 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9658057830 con 0x7f96640b3cc0 2024-03-29T09:09:34.239 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.242+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f964406ac90 msgr2=0x7f964406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.242+0000 7f966be00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f964406ac90 0x7f964406d150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96640b18a0 tx=0x7f9650008040 comp rx=0 tx=0).stop 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.242+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640b3cc0 msgr2=0x7f96640b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.242+0000 7f966be00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640b3cc0 0x7f96640b0b40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f965800c550 tx=0x7f965800c910 comp rx=0 tx=0).stop 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 shutdown_connections 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f964406ac90 0x7f964406d150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9664160770 0x7f966415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f96640b3cc0 0x7f96640b0b40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 --2- 172.21.15.99:0/327092831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96640d3380 0x7f96640b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 >> 172.21.15.99:0/327092831 conn(0x7f966408afd0 msgr2=0x7f96640d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:34.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 shutdown_connections 2024-03-29T09:09:34.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.243+0000 7f966be00700 1 -- 172.21.15.99:0/327092831 wait complete. 2024-03-29T09:09:34.260 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:09:34.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.453+0000 7fd9ba200700 1 Processor -- start 2024-03-29T09:09:34.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.454+0000 7fd9ba200700 1 -- start start 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9ba200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd9b4160770 0x7fd9b40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9ba200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd9b40d3550 0x7fd9b40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9ba200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40d1520 con 0x7fd9b40d0d60 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9ba200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40d1800 con 0x7fd9b40d3550 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9ba200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40d1ae0 con 0x7fd9b4160770 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9b3e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd9b40d3550 0x7fd9b40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9abe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd9b40d0d60 0x7fd9b40d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49762/0 (socket says 172.21.15.99:49762) 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9b3e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd9b40d3550 0x7fd9b40ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53598/0 (socket says 172.21.15.99:53598) 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9abe00700 1 -- 172.21.15.99:0/214965375 learned_addr learned my addr 172.21.15.99:0/214965375 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9b3400700 1 --2- 172.21.15.99:0/214965375 >> v2:172.21.15.133:3300/0 conn(0x7fd9b4160770 0x7fd9b40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9b3e00700 1 -- 172.21.15.99:0/214965375 >> v2:172.21.15.133:3300/0 conn(0x7fd9b4160770 msgr2=0x7fd9b40b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:34.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.455+0000 7fd9b3e00700 1 --2- 172.21.15.99:0/214965375 >> v2:172.21.15.133:3300/0 conn(0x7fd9b4160770 0x7fd9b40b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9b3e00700 1 -- 172.21.15.99:0/214965375 >> v1:172.21.15.99:6789/0 conn(0x7fd9b40d0d60 legacy=0x7fd9b40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9b3e00700 1 -- 172.21.15.99:0/214965375 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40d1dc0 con 0x7fd9b40d3550 2024-03-29T09:09:34.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9b3e00700 1 --2- 172.21.15.99:0/214965375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b40d3550 0x7fd9b40ce080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40b5d20 tx=0x7fd9ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996283ebbffe4f19 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9b2a00700 1 -- 172.21.15.99:0/214965375 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00e070 con 0x7fd9b40d3550 2024-03-29T09:09:34.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9b2a00700 1 -- 172.21.15.99:0/214965375 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac00bbd0 con 0x7fd9b40d3550 2024-03-29T09:09:34.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9b2a00700 1 -- 172.21.15.99:0/214965375 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac012430 con 0x7fd9b40d3550 2024-03-29T09:09:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9ba200700 1 -- 172.21.15.99:0/214965375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b40d3550 msgr2=0x7fd9b40ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.456+0000 7fd9ba200700 1 --2- 172.21.15.99:0/214965375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b40d3550 0x7fd9b40ce080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40b5d20 tx=0x7fd9ac00b330 comp rx=0 tx=0).stop 2024-03-29T09:09:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.457+0000 7fd9ba200700 1 -- 172.21.15.99:0/214965375 shutdown_connections 2024-03-29T09:09:34.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.457+0000 7fd9ba200700 1 --2- 172.21.15.99:0/214965375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b40d3550 0x7fd9b40ce080 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.457+0000 7fd9ba200700 1 --2- 172.21.15.99:0/214965375 >> v2:172.21.15.133:3300/0 conn(0x7fd9b4160770 0x7fd9b40b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.457+0000 7fd9ba200700 1 -- 172.21.15.99:0/214965375 >> 172.21.15.99:0/214965375 conn(0x7fd9b408afd0 msgr2=0x7fd9b409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.457+0000 7fd9ba200700 1 -- 172.21.15.99:0/214965375 shutdown_connections 2024-03-29T09:09:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.457+0000 7fd9ba200700 1 -- 172.21.15.99:0/214965375 wait complete. 2024-03-29T09:09:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.459+0000 7fd9ba200700 1 Processor -- start 2024-03-29T09:09:34.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.459+0000 7fd9ba200700 1 -- start start 2024-03-29T09:09:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.460+0000 7fd9ba200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9b40d0d60 0x7fd9b415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.460+0000 7fd9ba200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9b40d3550 0x7fd9b409d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.460+0000 7fd9ba200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4160770 0x7fd9b409dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.460+0000 7fd9ba200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40a03b0 con 0x7fd9b4160770 2024-03-29T09:09:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.460+0000 7fd9ba200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40a0690 con 0x7fd9b40d0d60 2024-03-29T09:09:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.460+0000 7fd9ba200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40a0970 con 0x7fd9b40d3550 2024-03-29T09:09:34.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4160770 0x7fd9b409dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4160770 0x7fd9b409dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53604/0 (socket says 172.21.15.99:53604) 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 -- 172.21.15.99:0/4040700777 learned_addr learned my addr 172.21.15.99:0/4040700777 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9abe00700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9b40d3550 0x7fd9b409d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3400700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9b40d0d60 0x7fd9b415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 -- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9b40d3550 msgr2=0x7fd9b409d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9b40d3550 0x7fd9b409d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 -- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9b40d0d60 msgr2=0x7fd9b415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9b40d0d60 0x7fd9b415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 -- 172.21.15.99:0/4040700777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac009a70 con 0x7fd9b4160770 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3400700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9b40d0d60 0x7fd9b415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9b3e00700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4160770 0x7fd9b409dc70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac014040 tx=0x7fd9ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9ab400700 1 -- 172.21.15.99:0/4040700777 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac012430 con 0x7fd9b4160770 2024-03-29T09:09:34.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.461+0000 7fd9ab400700 1 -- 172.21.15.99:0/4040700777 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac020070 con 0x7fd9b4160770 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.462+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b40a0c50 con 0x7fd9b4160770 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.462+0000 7fd9ab400700 1 -- 172.21.15.99:0/4040700777 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac004280 con 0x7fd9b4160770 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.462+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b409f7d0 con 0x7fd9b4160770 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.463+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b40678b0 con 0x7fd9b4160770 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.463+0000 7fd9ab400700 1 -- 172.21.15.99:0/4040700777 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd9ac0043e0 con 0x7fd9b4160770 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.464+0000 7fd9ab400700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd99006ac90 0x7fd99006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.464+0000 7fd9ab400700 1 -- 172.21.15.99:0/4040700777 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd9ac022070 con 0x7fd9b4160770 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.464+0000 7fd9b3400700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd99006ac90 0x7fd99006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.465+0000 7fd9b3400700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd99006ac90 0x7fd99006d150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd9b41f5bd0 tx=0x7fd9a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.468+0000 7fd9ab400700 1 -- 172.21.15.99:0/4040700777 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd9ac056360 con 0x7fd9b4160770 2024-03-29T09:09:34.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.668+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd9b4072c80 con 0x7fd9b4160770 2024-03-29T09:09:34.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.668+0000 7fd9ab400700 1 -- 172.21.15.99:0/4040700777 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd9ac059980 con 0x7fd9b4160770 2024-03-29T09:09:34.667 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:09:34.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd99006ac90 msgr2=0x7fd99006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd99006ac90 0x7fd99006d150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd9b41f5bd0 tx=0x7fd9a4008040 comp rx=0 tx=0).stop 2024-03-29T09:09:34.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4160770 msgr2=0x7fd9b409dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4160770 0x7fd9b409dc70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac014040 tx=0x7fd9ac004ad0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 shutdown_connections 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd99006ac90 0x7fd99006d150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd9b4160770 0x7fd9b409dc70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9b40d0d60 0x7fd9b415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 --2- 172.21.15.99:0/4040700777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9b40d3550 0x7fd9b409d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 >> 172.21.15.99:0/4040700777 conn(0x7fd9b408afd0 msgr2=0x7fd9b40cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 shutdown_connections 2024-03-29T09:09:34.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.671+0000 7fd9ba200700 1 -- 172.21.15.99:0/4040700777 wait complete. 2024-03-29T09:09:34.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:09:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.881+0000 7f750b400700 1 Processor -- start 2024-03-29T09:09:34.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.882+0000 7f750b400700 1 -- start start 2024-03-29T09:09:34.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f750b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75040b3cc0 0x7f75040d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f750b400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75040d3810 con 0x7f7504160770 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f750b400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75040d1520 con 0x7f75040d3380 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f750b400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75040b4140 con 0x7f75040b3cc0 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f7508a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7504160770 0x7f75040d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49768/0 (socket says 172.21.15.99:49768) 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f7508a00700 1 -- 172.21.15.99:0/1164607404 learned_addr learned my addr 172.21.15.99:0/1164607404 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f7503e00700 1 --2- 172.21.15.99:0/1164607404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75040b3cc0 0x7f75040d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.883+0000 7f7503400700 1 -- 172.21.15.99:0/1164607404 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765685800 0 0) 0x7f75040d3810 con 0x7f7504160770 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503400700 1 -- 172.21.15.99:0/1164607404 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74ec0035b0 con 0x7f7504160770 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503400700 1 -- 172.21.15.99:0/1164607404 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534689546 0 0) 0x7f75040d1520 con 0x7f75040d3380 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503400700 1 -- 172.21.15.99:0/1164607404 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75040d3810 con 0x7f75040d3380 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503e00700 1 -- 172.21.15.99:0/1164607404 >> v1:172.21.15.133:6789/0 conn(0x7f75040d3380 legacy=0x7f75040ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503e00700 1 -- 172.21.15.99:0/1164607404 >> v1:172.21.15.99:6789/0 conn(0x7f7504160770 legacy=0x7f75040d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503e00700 1 -- 172.21.15.99:0/1164607404 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040d1800 con 0x7f75040b3cc0 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503e00700 1 --2- 172.21.15.99:0/1164607404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75040b3cc0 0x7f75040d2e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009870 tx=0x7f74f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d7508a32741f2b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503400700 1 -- 172.21.15.99:0/1164607404 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0011830 con 0x7f75040b3cc0 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503400700 1 -- 172.21.15.99:0/1164607404 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f0011990 con 0x7f75040b3cc0 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.884+0000 7f7503400700 1 -- 172.21.15.99:0/1164607404 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f0011d20 con 0x7f75040b3cc0 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.885+0000 7f750b400700 1 -- 172.21.15.99:0/1164607404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75040b3cc0 msgr2=0x7f75040d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.885+0000 7f750b400700 1 --2- 172.21.15.99:0/1164607404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75040b3cc0 0x7f75040d2e40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009870 tx=0x7f74f0009b80 comp rx=0 tx=0).stop 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.885+0000 7f750b400700 1 -- 172.21.15.99:0/1164607404 shutdown_connections 2024-03-29T09:09:34.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.885+0000 7f750b400700 1 --2- 172.21.15.99:0/1164607404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75040b3cc0 0x7f75040d2e40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.885+0000 7f750b400700 1 -- 172.21.15.99:0/1164607404 >> 172.21.15.99:0/1164607404 conn(0x7f750408afd0 msgr2=0x7f750409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.885+0000 7f750b400700 1 -- 172.21.15.99:0/1164607404 shutdown_connections 2024-03-29T09:09:34.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.885+0000 7f750b400700 1 -- 172.21.15.99:0/1164607404 wait complete. 2024-03-29T09:09:34.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.887+0000 7f750b400700 1 Processor -- start 2024-03-29T09:09:34.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.887+0000 7f750b400700 1 -- start start 2024-03-29T09:09:34.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f750b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75040b3cc0 0x7f7504153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f750b400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75040d3380 0x7f750409f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f750b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7504160770 0x7f750409fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f750b400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7504154500 con 0x7f75040b3cc0 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f750b400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75040a0f00 con 0x7f75040d3380 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f750b400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75040a0380 con 0x7f7504160770 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75040b3cc0 0x7f7504153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75040b3cc0 0x7f7504153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53608/0 (socket says 172.21.15.99:53608) 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 -- 172.21.15.99:0/4095835519 learned_addr learned my addr 172.21.15.99:0/4095835519 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7503e00700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75040d3380 0x7f750409f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7509400700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7504160770 0x7f750409fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 -- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7504160770 msgr2=0x7f750409fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7504160770 0x7f750409fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 -- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75040d3380 msgr2=0x7f750409f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75040d3380 0x7f750409f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.888+0000 7f7508a00700 1 -- 172.21.15.99:0/4095835519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f00094a0 con 0x7f75040b3cc0 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.889+0000 7f7508a00700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75040b3cc0 0x7f7504153fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74f800c700 tx=0x7f74f800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.889+0000 7f7500c00700 1 -- 172.21.15.99:0/4095835519 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8005d70 con 0x7f75040b3cc0 2024-03-29T09:09:34.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.889+0000 7f7500c00700 1 -- 172.21.15.99:0/4095835519 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f80070d0 con 0x7f75040b3cc0 2024-03-29T09:09:34.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.889+0000 7f7500c00700 1 -- 172.21.15.99:0/4095835519 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8007460 con 0x7f75040b3cc0 2024-03-29T09:09:34.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.889+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75040a0660 con 0x7f75040b3cc0 2024-03-29T09:09:34.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.889+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75040a0c20 con 0x7f75040b3cc0 2024-03-29T09:09:34.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.890+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75040b5de0 con 0x7f75040b3cc0 2024-03-29T09:09:34.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.891+0000 7f7500c00700 1 -- 172.21.15.99:0/4095835519 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f74f80075c0 con 0x7f75040b3cc0 2024-03-29T09:09:34.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.892+0000 7f7500c00700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74e406ac90 0x7f74e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:34.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.893+0000 7f7500c00700 1 -- 172.21.15.99:0/4095835519 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74f8088ad0 con 0x7f75040b3cc0 2024-03-29T09:09:34.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.893+0000 7f7503e00700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74e406ac90 0x7f74e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:34.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.893+0000 7f7503e00700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74e406ac90 0x7f74e406d150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f74f00097c0 tx=0x7f74f001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:34.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:34.897+0000 7f7500c00700 1 -- 172.21.15.99:0/4095835519 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f75040b5de0 con 0x7f75040b3cc0 2024-03-29T09:09:35.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.102+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f75040678b0 con 0x7f75040b3cc0 2024-03-29T09:09:35.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.103+0000 7f7500c00700 1 -- 172.21.15.99:0/4095835519 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f74f80170a0 con 0x7f75040b3cc0 2024-03-29T09:09:35.102 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74e406ac90 msgr2=0x7f74e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74e406ac90 0x7f74e406d150 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f74f00097c0 tx=0x7f74f001c040 comp rx=0 tx=0).stop 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75040b3cc0 msgr2=0x7f7504153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75040b3cc0 0x7f7504153fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74f800c700 tx=0x7f74f800ca10 comp rx=0 tx=0).stop 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 shutdown_connections 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74e406ac90 0x7f74e406d150 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7504160770 0x7f750409fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f75040b3cc0 0x7f7504153fc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 --2- 172.21.15.99:0/4095835519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75040d3380 0x7f750409f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.105+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 >> 172.21.15.99:0/4095835519 conn(0x7f750408afd0 msgr2=0x7f75040d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.106+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 shutdown_connections 2024-03-29T09:09:35.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.106+0000 7f750b400700 1 -- 172.21.15.99:0/4095835519 wait complete. 2024-03-29T09:09:35.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:09:35.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.312+0000 7fd4b7200700 1 Processor -- start 2024-03-29T09:09:35.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.312+0000 7fd4b7200700 1 -- start start 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b7200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b00d1b20 0x7fd4b00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b7200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd4b0160cc0 0x7fd4b00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b7200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b00cddc0 con 0x7fd4b01608e0 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b7200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b00ce0a0 con 0x7fd4b0160cc0 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b7200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b00ce380 con 0x7fd4b00d1b20 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b5200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd4b0160cc0 0x7fd4b00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b5200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd4b0160cc0 0x7fd4b00ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53612/0 (socket says 172.21.15.99:53612) 2024-03-29T09:09:35.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4b5200700 1 -- 172.21.15.99:0/1908269440 learned_addr learned my addr 172.21.15.99:0/1908269440 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.313+0000 7fd4afe00700 1 --1- 172.21.15.99:0/1908269440 >> v1:172.21.15.99:6789/0 conn(0x7fd4b01608e0 0x7fd4b00d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49782/0 (socket says 172.21.15.99:49782) 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4af400700 1 --2- 172.21.15.99:0/1908269440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b00d1b20 0x7fd4b00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4b5200700 1 -- 172.21.15.99:0/1908269440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b00d1b20 msgr2=0x7fd4b00d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4b5200700 1 --2- 172.21.15.99:0/1908269440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b00d1b20 0x7fd4b00d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4b5200700 1 -- 172.21.15.99:0/1908269440 >> v1:172.21.15.99:6789/0 conn(0x7fd4b01608e0 legacy=0x7fd4b00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4b5200700 1 -- 172.21.15.99:0/1908269440 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b00ce660 con 0x7fd4b0160cc0 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4aea00700 1 -- 172.21.15.99:0/1908269440 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418655149 0 0) 0x7fd4b00cddc0 con 0x7fd4b01608e0 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4b5200700 1 --2- 172.21.15.99:0/1908269440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b0160cc0 0x7fd4b00ceb80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776f4e3efb389df7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4aea00700 1 -- 172.21.15.99:0/1908269440 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800e070 con 0x7fd4b0160cc0 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4aea00700 1 -- 172.21.15.99:0/1908269440 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a800b9c0 con 0x7fd4b0160cc0 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.314+0000 7fd4aea00700 1 -- 172.21.15.99:0/1908269440 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a800bd70 con 0x7fd4b0160cc0 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 -- 172.21.15.99:0/1908269440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b0160cc0 msgr2=0x7fd4b00ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 --2- 172.21.15.99:0/1908269440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b0160cc0 0x7fd4b00ceb80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009e50 tx=0x7fd4a800b330 comp rx=0 tx=0).stop 2024-03-29T09:09:35.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 -- 172.21.15.99:0/1908269440 shutdown_connections 2024-03-29T09:09:35.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 --2- 172.21.15.99:0/1908269440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b0160cc0 0x7fd4b00ceb80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 --2- 172.21.15.99:0/1908269440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b00d1b20 0x7fd4b00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 -- 172.21.15.99:0/1908269440 >> 172.21.15.99:0/1908269440 conn(0x7fd4b009c490 msgr2=0x7fd4b00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:35.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 -- 172.21.15.99:0/1908269440 shutdown_connections 2024-03-29T09:09:35.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.315+0000 7fd4b7200700 1 -- 172.21.15.99:0/1908269440 wait complete. 2024-03-29T09:09:35.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.317+0000 7fd4b7200700 1 Processor -- start 2024-03-29T09:09:35.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.317+0000 7fd4b7200700 1 -- start start 2024-03-29T09:09:35.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4b7200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b00d1b20 0x7fd4b01531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4b7200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01608e0 0x7fd4b00ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4b7200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b0160cc0 0x7fd4b015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4b7200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0155740 con 0x7fd4b00d1b20 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4b7200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0155a20 con 0x7fd4b01608e0 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4b7200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0155d00 con 0x7fd4b0160cc0 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4afe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b00d1b20 0x7fd4b01531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4afe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b00d1b20 0x7fd4b01531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53624/0 (socket says 172.21.15.99:53624) 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4afe00700 1 -- 172.21.15.99:0/3199927613 learned_addr learned my addr 172.21.15.99:0/3199927613 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4af400700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01608e0 0x7fd4b00ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.318+0000 7fd4b5200700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b0160cc0 0x7fd4b015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4afe00700 1 -- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b0160cc0 msgr2=0x7fd4b015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4afe00700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b0160cc0 0x7fd4b015a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4afe00700 1 -- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01608e0 msgr2=0x7fd4b00ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4afe00700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01608e0 0x7fd4b00ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4afe00700 1 -- 172.21.15.99:0/3199927613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a8009b30 con 0x7fd4b00d1b20 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4afe00700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b00d1b20 0x7fd4b01531a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00011e0 tx=0x7fd4a000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4a7e00700 1 -- 172.21.15.99:0/3199927613 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0004c80 con 0x7fd4b00d1b20 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4a7e00700 1 -- 172.21.15.99:0/3199927613 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a0004e00 con 0x7fd4b00d1b20 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4a7e00700 1 -- 172.21.15.99:0/3199927613 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0007310 con 0x7fd4b00d1b20 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b0155fe0 con 0x7fd4b00d1b20 2024-03-29T09:09:35.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.319+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b01536e0 con 0x7fd4b00d1b20 2024-03-29T09:09:35.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.320+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b00677a0 con 0x7fd4b00d1b20 2024-03-29T09:09:35.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.321+0000 7fd4a7e00700 1 -- 172.21.15.99:0/3199927613 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd4a0007470 con 0x7fd4b00d1b20 2024-03-29T09:09:35.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.322+0000 7fd4a7e00700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd49006ac90 0x7fd49006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.322+0000 7fd4a7e00700 1 -- 172.21.15.99:0/3199927613 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd4a0089220 con 0x7fd4b00d1b20 2024-03-29T09:09:35.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.322+0000 7fd4af400700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd49006ac90 0x7fd49006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.323+0000 7fd4af400700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd49006ac90 0x7fd49006d150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4b01f6020 tx=0x7fd49c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:35.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.326+0000 7fd4a7e00700 1 -- 172.21.15.99:0/3199927613 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd4a0054100 con 0x7fd4b00d1b20 2024-03-29T09:09:35.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.571+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4b00d0700 con 0x7fd4b00d1b20 2024-03-29T09:09:35.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.571+0000 7fd4a7e00700 1 -- 172.21.15.99:0/3199927613 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7fd4a0057720 con 0x7fd4b00d1b20 2024-03-29T09:09:35.571 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:35.571 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:09:35.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd49006ac90 msgr2=0x7fd49006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd49006ac90 0x7fd49006d150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4b01f6020 tx=0x7fd49c008040 comp rx=0 tx=0).stop 2024-03-29T09:09:35.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b00d1b20 msgr2=0x7fd4b01531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b00d1b20 0x7fd4b01531a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00011e0 tx=0x7fd4a000ca30 comp rx=0 tx=0).stop 2024-03-29T09:09:35.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 shutdown_connections 2024-03-29T09:09:35.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd49006ac90 0x7fd49006d150 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4b0160cc0 0x7fd4b015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4b00d1b20 0x7fd4b01531a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 --2- 172.21.15.99:0/3199927613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01608e0 0x7fd4b00ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 >> 172.21.15.99:0/3199927613 conn(0x7fd4b009c490 msgr2=0x7fd4b015df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:35.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 shutdown_connections 2024-03-29T09:09:35.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.574+0000 7fd4b7200700 1 -- 172.21.15.99:0/3199927613 wait complete. 2024-03-29T09:09:35.574 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:09:35.591 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:35.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.781+0000 7f11e5200700 1 Processor -- start 2024-03-29T09:09:35.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.782+0000 7f11e5200700 1 -- start start 2024-03-29T09:09:35.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11e5200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11e01608e0 0x7f11e00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11e5200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e00cddc0 con 0x7f11e0160cc0 2024-03-29T09:09:35.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11e5200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e00ce0a0 con 0x7f11e00d1b20 2024-03-29T09:09:35.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11e5200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e00ce380 con 0x7f11e01608e0 2024-03-29T09:09:35.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11d7e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f11e0160cc0 0x7f11e00d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49798/0 (socket says 172.21.15.99:49798) 2024-03-29T09:09:35.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11d7e00700 1 -- 172.21.15.99:0/3248046561 learned_addr learned my addr 172.21.15.99:0/3248046561 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:35.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11dde00700 1 --2- 172.21.15.99:0/3248046561 >> v2:172.21.15.133:3300/0 conn(0x7f11e01608e0 0x7f11e00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11dd400700 1 -- 172.21.15.99:0/3248046561 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685994593 0 0) 0x7f11e00cddc0 con 0x7f11e0160cc0 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11dd400700 1 -- 172.21.15.99:0/3248046561 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c40035b0 con 0x7f11e0160cc0 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11dde00700 1 -- 172.21.15.99:0/3248046561 >> v1:172.21.15.133:6789/0 conn(0x7f11e00d1b20 legacy=0x7f11e00afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11dde00700 1 -- 172.21.15.99:0/3248046561 >> v1:172.21.15.99:6789/0 conn(0x7f11e0160cc0 legacy=0x7f11e00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.783+0000 7f11dde00700 1 -- 172.21.15.99:0/3248046561 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00ce660 con 0x7f11e01608e0 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.784+0000 7f11dde00700 1 --2- 172.21.15.99:0/3248046561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 0x7f11e00d0150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009840 tx=0x7f11d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbaaa224104f1eb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.784+0000 7f11dd400700 1 -- 172.21.15.99:0/3248046561 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0011830 con 0x7f11e01608e0 2024-03-29T09:09:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.784+0000 7f11dd400700 1 -- 172.21.15.99:0/3248046561 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d0011990 con 0x7f11e01608e0 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.784+0000 7f11dd400700 1 -- 172.21.15.99:0/3248046561 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0011d20 con 0x7f11e01608e0 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.785+0000 7f11e5200700 1 -- 172.21.15.99:0/3248046561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 msgr2=0x7f11e00d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.785+0000 7f11e5200700 1 --2- 172.21.15.99:0/3248046561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 0x7f11e00d0150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009840 tx=0x7f11d0009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.785+0000 7f11e5200700 1 -- 172.21.15.99:0/3248046561 shutdown_connections 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.785+0000 7f11e5200700 1 --2- 172.21.15.99:0/3248046561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 0x7f11e00d0150 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.785+0000 7f11e5200700 1 -- 172.21.15.99:0/3248046561 >> 172.21.15.99:0/3248046561 conn(0x7f11e009c490 msgr2=0x7f11e00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.785+0000 7f11e5200700 1 -- 172.21.15.99:0/3248046561 shutdown_connections 2024-03-29T09:09:35.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.785+0000 7f11e5200700 1 -- 172.21.15.99:0/3248046561 wait complete. 2024-03-29T09:09:35.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.786+0000 7f11e5200700 1 Processor -- start 2024-03-29T09:09:35.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.787+0000 7f11e5200700 1 -- start start 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.787+0000 7f11e5200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11e00d1b20 0x7f11e00cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.787+0000 7f11e5200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 0x7f11e0159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11e5200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0160cc0 0x7f11e015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11e5200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00a0870 con 0x7f11e0160cc0 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11e5200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00cc760 con 0x7f11e01608e0 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11e5200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00cca40 con 0x7f11e00d1b20 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0160cc0 0x7f11e015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0160cc0 0x7f11e015c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53636/0 (socket says 172.21.15.99:53636) 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 -- 172.21.15.99:0/443355207 learned_addr learned my addr 172.21.15.99:0/443355207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11dde00700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11e00d1b20 0x7f11e00cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11d7e00700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 0x7f11e0159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 -- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11e00d1b20 msgr2=0x7f11e00cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11e00d1b20 0x7f11e00cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 -- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 msgr2=0x7f11e0159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 0x7f11e0159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 -- 172.21.15.99:0/443355207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00094a0 con 0x7f11e0160cc0 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.788+0000 7f11de800700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0160cc0 0x7f11e015c3e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11d800cc50 tx=0x7f11d800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.789+0000 7f11d6000700 1 -- 172.21.15.99:0/443355207 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8003c00 con 0x7f11e0160cc0 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.789+0000 7f11d6000700 1 -- 172.21.15.99:0/443355207 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d8003d60 con 0x7f11e0160cc0 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.789+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e00a0220 con 0x7f11e0160cc0 2024-03-29T09:09:35.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.789+0000 7f11d6000700 1 -- 172.21.15.99:0/443355207 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800aab0 con 0x7f11e0160cc0 2024-03-29T09:09:35.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.789+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e00cc470 con 0x7f11e0160cc0 2024-03-29T09:09:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.790+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e00a0080 con 0x7f11e0160cc0 2024-03-29T09:09:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.791+0000 7f11d6000700 1 -- 172.21.15.99:0/443355207 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11d800ac10 con 0x7f11e0160cc0 2024-03-29T09:09:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.791+0000 7f11d6000700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11bc06ac90 0x7f11bc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.792+0000 7f11d6000700 1 -- 172.21.15.99:0/443355207 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f11d8089660 con 0x7f11e0160cc0 2024-03-29T09:09:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.792+0000 7f11dde00700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11bc06ac90 0x7f11bc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:35.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.792+0000 7f11dde00700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11bc06ac90 0x7f11bc06d150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11d00090b0 tx=0x7f11d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:35.795 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:35.796+0000 7f11d6000700 1 -- 172.21.15.99:0/443355207 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11e00a0080 con 0x7f11e0160cc0 2024-03-29T09:09:36.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.002+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11e00a0080 con 0x7f11e0160cc0 2024-03-29T09:09:36.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.003+0000 7f11d6000700 1 -- 172.21.15.99:0/443355207 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f11e00a0080 con 0x7f11e0160cc0 2024-03-29T09:09:36.002 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:36.003 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.005+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11bc06ac90 msgr2=0x7f11bc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.005+0000 7f11e5200700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11bc06ac90 0x7f11bc06d150 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11d00090b0 tx=0x7f11d0008030 comp rx=0 tx=0).stop 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.005+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0160cc0 msgr2=0x7f11e015c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.005+0000 7f11e5200700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0160cc0 0x7f11e015c3e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11d800cc50 tx=0x7f11d800cf60 comp rx=0 tx=0).stop 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 shutdown_connections 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11bc06ac90 0x7f11bc06d150 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11e0160cc0 0x7f11e015c3e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11e00d1b20 0x7f11e00cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 --2- 172.21.15.99:0/443355207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e01608e0 0x7f11e0159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 >> 172.21.15.99:0/443355207 conn(0x7f11e009c490 msgr2=0x7f11e00d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 shutdown_connections 2024-03-29T09:09:36.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.006+0000 7f11e5200700 1 -- 172.21.15.99:0/443355207 wait complete. 2024-03-29T09:09:36.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:09:36.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:09:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.216+0000 7ff553a00700 1 Processor -- start 2024-03-29T09:09:36.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.217+0000 7ff553a00700 1 -- start start 2024-03-29T09:09:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff553a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff54c0b3cc0 0x7ff54c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff553a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff54c0d3380 0x7ff54c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff553a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c0d3810 con 0x7ff54c160770 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff553a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c0b4140 con 0x7ff54c0d3380 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff553a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c0d1520 con 0x7ff54c0b3cc0 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff551a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff54c0d3380 0x7ff54c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff551000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ff54c160770 0x7ff54c0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49804/0 (socket says 172.21.15.99:49804) 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff551000700 1 -- 172.21.15.99:0/3263440437 learned_addr learned my addr 172.21.15.99:0/3263440437 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff54be00700 1 --2- 172.21.15.99:0/3263440437 >> v2:172.21.15.133:3300/0 conn(0x7ff54c0b3cc0 0x7ff54c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.218+0000 7ff54b400700 1 -- 172.21.15.99:0/3263440437 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819536895 0 0) 0x7ff54c0d3810 con 0x7ff54c160770 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff54b400700 1 -- 172.21.15.99:0/3263440437 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5340035b0 con 0x7ff54c160770 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff551a00700 1 -- 172.21.15.99:0/3263440437 >> v2:172.21.15.133:3300/0 conn(0x7ff54c0b3cc0 msgr2=0x7ff54c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff551a00700 1 --2- 172.21.15.99:0/3263440437 >> v2:172.21.15.133:3300/0 conn(0x7ff54c0b3cc0 0x7ff54c0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff551a00700 1 -- 172.21.15.99:0/3263440437 >> v1:172.21.15.99:6789/0 conn(0x7ff54c160770 legacy=0x7ff54c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff551a00700 1 -- 172.21.15.99:0/3263440437 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0d1800 con 0x7ff54c0d3380 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff551a00700 1 --2- 172.21.15.99:0/3263440437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c0ce080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0b5cf0 tx=0x7ff54400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d276c6fe1e2ce7cb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff54b400700 1 -- 172.21.15.99:0/3263440437 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400e070 con 0x7ff54c0d3380 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff54b400700 1 -- 172.21.15.99:0/3263440437 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54400bb20 con 0x7ff54c0d3380 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.219+0000 7ff54b400700 1 -- 172.21.15.99:0/3263440437 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544012430 con 0x7ff54c0d3380 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 -- 172.21.15.99:0/3263440437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 msgr2=0x7ff54c0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 --2- 172.21.15.99:0/3263440437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c0ce080 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0b5cf0 tx=0x7ff54400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 -- 172.21.15.99:0/3263440437 shutdown_connections 2024-03-29T09:09:36.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 --2- 172.21.15.99:0/3263440437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c0ce080 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 --2- 172.21.15.99:0/3263440437 >> v2:172.21.15.133:3300/0 conn(0x7ff54c0b3cc0 0x7ff54c0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 -- 172.21.15.99:0/3263440437 >> 172.21.15.99:0/3263440437 conn(0x7ff54c08afd0 msgr2=0x7ff54c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 -- 172.21.15.99:0/3263440437 shutdown_connections 2024-03-29T09:09:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.220+0000 7ff553a00700 1 -- 172.21.15.99:0/3263440437 wait complete. 2024-03-29T09:09:36.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.221+0000 7ff553a00700 1 Processor -- start 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.222+0000 7ff553a00700 1 -- start start 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.222+0000 7ff553a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff54c0b3cc0 0x7ff54c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.222+0000 7ff553a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c15bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff553a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff54c160770 0x7ff54c0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff553a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0cb2b0 con 0x7ff54c0d3380 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff553a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0cc110 con 0x7ff54c0b3cc0 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff553a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1562b0 con 0x7ff54c160770 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c15bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c15bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53658/0 (socket says 172.21.15.99:53658) 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 -- 172.21.15.99:0/2241334112 learned_addr learned my addr 172.21.15.99:0/2241334112 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff551000700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff54c0b3cc0 0x7ff54c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff551a00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff54c160770 0x7ff54c0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 -- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff54c160770 msgr2=0x7ff54c0b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff54c160770 0x7ff54c0b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 -- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff54c0b3cc0 msgr2=0x7ff54c155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff54c0b3cc0 0x7ff54c155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 -- 172.21.15.99:0/2241334112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009a70 con 0x7ff54c0d3380 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.223+0000 7ff54be00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c15bb90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff53800e9b0 tx=0x7ff53800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.224+0000 7ff548c00700 1 -- 172.21.15.99:0/2241334112 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538009710 con 0x7ff54c0d3380 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.224+0000 7ff548c00700 1 -- 172.21.15.99:0/2241334112 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53800c850 con 0x7ff54c0d3380 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.224+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c0cb500 con 0x7ff54c0d3380 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.224+0000 7ff548c00700 1 -- 172.21.15.99:0/2241334112 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800cbc0 con 0x7ff54c0d3380 2024-03-29T09:09:36.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.224+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c0cbac0 con 0x7ff54c0d3380 2024-03-29T09:09:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.225+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c0678b0 con 0x7ff54c0d3380 2024-03-29T09:09:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.226+0000 7ff548c00700 1 -- 172.21.15.99:0/2241334112 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff53800cd20 con 0x7ff54c0d3380 2024-03-29T09:09:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.226+0000 7ff548c00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff52c06ac90 0x7ff52c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.226+0000 7ff548c00700 1 -- 172.21.15.99:0/2241334112 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff538089b00 con 0x7ff54c0d3380 2024-03-29T09:09:36.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.227+0000 7ff551000700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff52c06ac90 0x7ff52c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.227+0000 7ff551000700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff52c06ac90 0x7ff52c06d150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5400011e0 tx=0x7ff540009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:36.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.231+0000 7ff548c00700 1 -- 172.21.15.99:0/2241334112 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff5380549f0 con 0x7ff54c0d3380 2024-03-29T09:09:36.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.430+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff54c0b3890 con 0x7ff54c0d3380 2024-03-29T09:09:36.430 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.431+0000 7ff548c00700 1 -- 172.21.15.99:0/2241334112 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5380170a0 con 0x7ff54c0d3380 2024-03-29T09:09:36.430 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:09:36.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff52c06ac90 msgr2=0x7ff52c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff52c06ac90 0x7ff52c06d150 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5400011e0 tx=0x7ff540009040 comp rx=0 tx=0).stop 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 msgr2=0x7ff54c15bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c15bb90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff53800e9b0 tx=0x7ff53800ed70 comp rx=0 tx=0).stop 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 shutdown_connections 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff52c06ac90 0x7ff52c06d150 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff54c160770 0x7ff54c0b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff54c0b3cc0 0x7ff54c155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 --2- 172.21.15.99:0/2241334112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff54c0d3380 0x7ff54c15bb90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 >> 172.21.15.99:0/2241334112 conn(0x7ff54c08afd0 msgr2=0x7ff54c073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 shutdown_connections 2024-03-29T09:09:36.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.433+0000 7ff553a00700 1 -- 172.21.15.99:0/2241334112 wait complete. 2024-03-29T09:09:36.451 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:09:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.644+0000 7fb9fda00700 1 Processor -- start 2024-03-29T09:09:36.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.645+0000 7fb9fda00700 1 -- start start 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9fda00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb9f8160770 0x7fb9f80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9fda00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb9f80d0d60 0x7fb9f80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9fda00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f80d3810 con 0x7fb9f80d3380 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9fda00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f80d1520 con 0x7fb9f8160770 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9fda00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f80d1800 con 0x7fb9f80d0d60 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9f6a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb9f8160770 0x7fb9f80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9f7400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb9f80d3380 0x7fb9f80cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49806/0 (socket says 172.21.15.99:49806) 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9f7400700 1 -- 172.21.15.99:0/3489261831 learned_addr learned my addr 172.21.15.99:0/3489261831 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9f6a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb9f8160770 0x7fb9f80b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53668/0 (socket says 172.21.15.99:53668) 2024-03-29T09:09:36.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.646+0000 7fb9efe00700 1 --2- 172.21.15.99:0/3489261831 >> v2:172.21.15.133:3300/0 conn(0x7fb9f80d0d60 0x7fb9f80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6a00700 1 -- 172.21.15.99:0/3489261831 >> v2:172.21.15.133:3300/0 conn(0x7fb9f80d0d60 msgr2=0x7fb9f80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6a00700 1 --2- 172.21.15.99:0/3489261831 >> v2:172.21.15.133:3300/0 conn(0x7fb9f80d0d60 0x7fb9f80d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6a00700 1 -- 172.21.15.99:0/3489261831 >> v1:172.21.15.99:6789/0 conn(0x7fb9f80d3380 legacy=0x7fb9f80cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6a00700 1 -- 172.21.15.99:0/3489261831 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80d1ae0 con 0x7fb9f8160770 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6000700 1 -- 172.21.15.99:0/3489261831 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918806494 0 0) 0x7fb9f80d3810 con 0x7fb9f80d3380 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9efe00700 1 --2- 172.21.15.99:0/3489261831 >> v2:172.21.15.133:3300/0 conn(0x7fb9f80d0d60 0x7fb9f80d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6a00700 1 --2- 172.21.15.99:0/3489261831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f8160770 0x7fb9f80b3cc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80b5cf0 tx=0x7fb9e8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1babff69039f7f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6000700 1 -- 172.21.15.99:0/3489261831 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8011830 con 0x7fb9f8160770 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6000700 1 -- 172.21.15.99:0/3489261831 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e8011990 con 0x7fb9f8160770 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.647+0000 7fb9f6000700 1 -- 172.21.15.99:0/3489261831 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8011d20 con 0x7fb9f8160770 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 -- 172.21.15.99:0/3489261831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f8160770 msgr2=0x7fb9f80b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3489261831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f8160770 0x7fb9f80b3cc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80b5cf0 tx=0x7fb9e8009b20 comp rx=0 tx=0).stop 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 -- 172.21.15.99:0/3489261831 shutdown_connections 2024-03-29T09:09:36.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3489261831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f8160770 0x7fb9f80b3cc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3489261831 >> v2:172.21.15.133:3300/0 conn(0x7fb9f80d0d60 0x7fb9f80d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 -- 172.21.15.99:0/3489261831 >> 172.21.15.99:0/3489261831 conn(0x7fb9f808afd0 msgr2=0x7fb9f809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:36.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 -- 172.21.15.99:0/3489261831 shutdown_connections 2024-03-29T09:09:36.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.648+0000 7fb9fda00700 1 -- 172.21.15.99:0/3489261831 wait complete. 2024-03-29T09:09:36.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.649+0000 7fb9fda00700 1 Processor -- start 2024-03-29T09:09:36.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.650+0000 7fb9fda00700 1 -- start start 2024-03-29T09:09:36.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.650+0000 7fb9fda00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9f80d0d60 0x7fb9f815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.650+0000 7fb9fda00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f80d3380 0x7fb9f809dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.650+0000 7fb9fda00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9f8160770 0x7fb9f80b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.650+0000 7fb9fda00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80b0d60 con 0x7fb9f80d3380 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.650+0000 7fb9fda00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f815cd30 con 0x7fb9f80d0d60 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.650+0000 7fb9fda00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80b0a80 con 0x7fb9f8160770 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f80d3380 0x7fb9f809dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f80d3380 0x7fb9f809dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53674/0 (socket says 172.21.15.99:53674) 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 -- 172.21.15.99:0/3653957915 learned_addr learned my addr 172.21.15.99:0/3653957915 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9f6a00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9f80d0d60 0x7fb9f815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9f7400700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9f8160770 0x7fb9f80b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 -- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9f8160770 msgr2=0x7fb9f80b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9f8160770 0x7fb9f80b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 -- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9f80d0d60 msgr2=0x7fb9f815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9f80d0d60 0x7fb9f815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 -- 172.21.15.99:0/3653957915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80094a0 con 0x7fb9f80d3380 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9f6a00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9f80d0d60 0x7fb9f815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9efe00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f80d3380 0x7fb9f809dea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000e810 tx=0x7fb9e000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.651+0000 7fb9eea00700 1 -- 172.21.15.99:0/3653957915 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0013610 con 0x7fb9f80d3380 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.652+0000 7fb9eea00700 1 -- 172.21.15.99:0/3653957915 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e0013770 con 0x7fb9f80d3380 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.652+0000 7fb9eea00700 1 -- 172.21.15.99:0/3653957915 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0013b20 con 0x7fb9f80d3380 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.652+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f80b5e50 con 0x7fb9f80d3380 2024-03-29T09:09:36.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.652+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f80b6410 con 0x7fb9f80d3380 2024-03-29T09:09:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.653+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f80678b0 con 0x7fb9f80d3380 2024-03-29T09:09:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.654+0000 7fb9eea00700 1 -- 172.21.15.99:0/3653957915 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb9e0013c80 con 0x7fb9f80d3380 2024-03-29T09:09:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.654+0000 7fb9eea00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb9d406ac90 0x7fb9d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.654+0000 7fb9eea00700 1 -- 172.21.15.99:0/3653957915 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb9e0089240 con 0x7fb9f80d3380 2024-03-29T09:09:36.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.655+0000 7fb9f6a00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb9d406ac90 0x7fb9d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:36.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.655+0000 7fb9f6a00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb9d406ac90 0x7fb9d406d150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80090b0 tx=0x7fb9e801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:36.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.658+0000 7fb9eea00700 1 -- 172.21.15.99:0/3653957915 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb9e00541c0 con 0x7fb9f80d3380 2024-03-29T09:09:36.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.859+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9f806cc90 con 0x7fb9f80d3380 2024-03-29T09:09:36.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.859+0000 7fb9eea00700 1 -- 172.21.15.99:0/3653957915 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9e00577e0 con 0x7fb9f80d3380 2024-03-29T09:09:36.859 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:36.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb9d406ac90 msgr2=0x7fb9d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb9d406ac90 0x7fb9d406d150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80090b0 tx=0x7fb9e801c040 comp rx=0 tx=0).stop 2024-03-29T09:09:36.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f80d3380 msgr2=0x7fb9f809dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:36.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f80d3380 0x7fb9f809dea0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000e810 tx=0x7fb9e000ebd0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 shutdown_connections 2024-03-29T09:09:36.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb9d406ac90 0x7fb9d406d150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9f8160770 0x7fb9f80b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9f80d0d60 0x7fb9f815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 --2- 172.21.15.99:0/3653957915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb9f80d3380 0x7fb9f809dea0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:36.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 >> 172.21.15.99:0/3653957915 conn(0x7fb9f808afd0 msgr2=0x7fb9f815e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:36.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 shutdown_connections 2024-03-29T09:09:36.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:36.862+0000 7fb9fda00700 1 -- 172.21.15.99:0/3653957915 wait complete. 2024-03-29T09:09:36.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:09:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.075+0000 7f596ec00700 1 Processor -- start 2024-03-29T09:09:37.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.075+0000 7f596ec00700 1 -- start start 2024-03-29T09:09:37.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.076+0000 7f596ec00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5968160770 0x7f59680b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.076+0000 7f596ec00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 0x7f59680d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.076+0000 7f596ec00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59680d3810 con 0x7f59680d3380 2024-03-29T09:09:37.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.076+0000 7f596ec00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59680d1520 con 0x7f5968160770 2024-03-29T09:09:37.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.076+0000 7f596ec00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59680d1800 con 0x7f59680d0d60 2024-03-29T09:09:37.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.076+0000 7f596cc00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f59680d3380 0x7f59680cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49810/0 (socket says 172.21.15.99:49810) 2024-03-29T09:09:37.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f596cc00700 1 -- 172.21.15.99:0/2789392041 learned_addr learned my addr 172.21.15.99:0/2789392041 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:37.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5967e00700 1 --2- 172.21.15.99:0/2789392041 >> v2:172.21.15.133:3300/0 conn(0x7f5968160770 0x7f59680b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5967400700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 0x7f59680d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663226474 0 0) 0x7f59680d3810 con 0x7f59680d3380 2024-03-29T09:09:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f594c0035b0 con 0x7f59680d3380 2024-03-29T09:09:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639929274 0 0) 0x7f594c0035b0 con 0x7f59680d3380 2024-03-29T09:09:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 msgr2=0x7f59680d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 0x7f59680d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 >> v2:172.21.15.133:3300/0 conn(0x7f5968160770 msgr2=0x7f59680b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 --2- 172.21.15.99:0/2789392041 >> v2:172.21.15.133:3300/0 conn(0x7f5968160770 0x7f59680b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59680d1ae0 con 0x7f59680d3380 2024-03-29T09:09:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f59600019f0 con 0x7f59680d3380 2024-03-29T09:09:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 >> v1:172.21.15.99:6789/0 conn(0x7f59680d3380 legacy=0x7f59680cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f594c003a00 0x7f594c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 0x7f594c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 0x7f594c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59680d1520 con 0x7f594c003a00 2024-03-29T09:09:37.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.077+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59680d1800 con 0x7f594c004850 2024-03-29T09:09:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c00ba10 con 0x7f59680d0d60 2024-03-29T09:09:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967e00700 1 --2- 172.21.15.99:0/2789392041 >> v2:172.21.15.133:3300/0 conn(0x7f5968160770 0x7f59680b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967400700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 0x7f594c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967e00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f594c003a00 0x7f594c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f596cc00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 0x7f594c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967400700 1 -- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 msgr2=0x7f594c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967400700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59680d0d60 0x7f594c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967400700 1 -- 172.21.15.99:0/2789392041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f594c003a00 msgr2=0x7f594c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967400700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f594c003a00 0x7f594c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967400700 1 -- 172.21.15.99:0/2789392041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c0035b0 con 0x7f594c004850 2024-03-29T09:09:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967e00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f594c003a00 0x7f594c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.078+0000 7f5967400700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 0x7f594c004cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5954009b40 tx=0x7f594c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c86c49bf5916a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595400e6c0 con 0x7f594c004850 2024-03-29T09:09:37.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595400e820 con 0x7f594c004850 2024-03-29T09:09:37.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f5966a00700 1 -- 172.21.15.99:0/2789392041 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595400ebd0 con 0x7f594c004850 2024-03-29T09:09:37.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f596ec00700 1 -- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 msgr2=0x7f594c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f596ec00700 1 --2- 172.21.15.99:0/2789392041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 0x7f594c004cd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5954009b40 tx=0x7f594c005880 comp rx=0 tx=0).stop 2024-03-29T09:09:37.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f596cc00700 1 -- 172.21.15.99:0/2789392041 reap_dead start 2024-03-29T09:09:37.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f596ec00700 1 -- 172.21.15.99:0/2789392041 shutdown_connections 2024-03-29T09:09:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f596ec00700 1 -- 172.21.15.99:0/2789392041 >> 172.21.15.99:0/2789392041 conn(0x7f596808afd0 msgr2=0x7f596809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f596ec00700 1 -- 172.21.15.99:0/2789392041 shutdown_connections 2024-03-29T09:09:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.079+0000 7f596ec00700 1 -- 172.21.15.99:0/2789392041 wait complete. 2024-03-29T09:09:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.081+0000 7f596ec00700 1 Processor -- start 2024-03-29T09:09:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.081+0000 7f596ec00700 1 -- start start 2024-03-29T09:09:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f596ec00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 0x7f59680b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f596ec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59680b6840 0x7f59680b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f596ec00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f596806c590 0x7f596806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f596ec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596809e0c0 con 0x7f59680b6840 2024-03-29T09:09:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f596ec00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596806cf50 con 0x7f594c004850 2024-03-29T09:09:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f596ec00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596809d8b0 con 0x7f596806c590 2024-03-29T09:09:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f5967400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59680b6840 0x7f59680b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f5967400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59680b6840 0x7f59680b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53692/0 (socket says 172.21.15.99:53692) 2024-03-29T09:09:37.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.082+0000 7f5967400700 1 -- 172.21.15.99:0/4042901359 learned_addr learned my addr 172.21.15.99:0/4042901359 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f5967400700 1 -- 172.21.15.99:0/4042901359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f596806c590 msgr2=0x7f596806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f5967400700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f596806c590 0x7f596806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f5967400700 1 -- 172.21.15.99:0/4042901359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 msgr2=0x7f59680b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f5967400700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 0x7f59680b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f5967400700 1 -- 172.21.15.99:0/4042901359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596809fb60 con 0x7f59680b6840 2024-03-29T09:09:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f5967400700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59680b6840 0x7f59680b5620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f595400a010 tx=0x7f59540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f595fe00700 1 -- 172.21.15.99:0/4042901359 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595400e6c0 con 0x7f59680b6840 2024-03-29T09:09:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f595fe00700 1 -- 172.21.15.99:0/4042901359 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954013dd0 con 0x7f59680b6840 2024-03-29T09:09:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596809f550 con 0x7f59680b6840 2024-03-29T09:09:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f595fe00700 1 -- 172.21.15.99:0/4042901359 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954006e60 con 0x7f59680b6840 2024-03-29T09:09:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.083+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596809dde0 con 0x7f59680b6840 2024-03-29T09:09:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.084+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596809e000 con 0x7f59680b6840 2024-03-29T09:09:37.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.085+0000 7f595fe00700 1 -- 172.21.15.99:0/4042901359 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5954004270 con 0x7f59680b6840 2024-03-29T09:09:37.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.086+0000 7f595fe00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f594806ac90 0x7f594806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.086+0000 7f595fe00700 1 -- 172.21.15.99:0/4042901359 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f595408ae60 con 0x7f59680b6840 2024-03-29T09:09:37.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.086+0000 7f5967e00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f594806ac90 0x7f594806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.087+0000 7f5967e00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f594806ac90 0x7f594806d150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5958006f10 tx=0x7f5958006ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.091+0000 7f595fe00700 1 -- 172.21.15.99:0/4042901359 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5954055e60 con 0x7f59680b6840 2024-03-29T09:09:37.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.293+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59680678b0 con 0x7f59680b6840 2024-03-29T09:09:37.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.293+0000 7f595fe00700 1 -- 172.21.15.99:0/4042901359 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5954017020 con 0x7f59680b6840 2024-03-29T09:09:37.293 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f594806ac90 msgr2=0x7f594806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f594806ac90 0x7f594806d150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5958006f10 tx=0x7f5958006ea0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59680b6840 msgr2=0x7f59680b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59680b6840 0x7f59680b5620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f595400a010 tx=0x7f59540047c0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 shutdown_connections 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f594806ac90 0x7f594806d150 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f596806c590 0x7f596806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f594c004850 0x7f59680b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 --2- 172.21.15.99:0/4042901359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59680b6840 0x7f59680b5620 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 >> 172.21.15.99:0/4042901359 conn(0x7f596808afd0 msgr2=0x7f59680b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 shutdown_connections 2024-03-29T09:09:37.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.296+0000 7f596ec00700 1 -- 172.21.15.99:0/4042901359 wait complete. 2024-03-29T09:09:37.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:09:37.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.511+0000 7f567ec00700 1 Processor -- start 2024-03-29T09:09:37.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.511+0000 7f567ec00700 1 -- start start 2024-03-29T09:09:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.512+0000 7f567ec00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5678160770 0x7f56780b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.512+0000 7f567ec00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56780d1520 con 0x7f56780d3550 2024-03-29T09:09:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.512+0000 7f567ec00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56780d1800 con 0x7f56780d0d60 2024-03-29T09:09:37.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.512+0000 7f567ec00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56780d1ae0 con 0x7f5678160770 2024-03-29T09:09:37.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f567cc00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f56780d3550 0x7f56780ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:49818/0 (socket says 172.21.15.99:49818) 2024-03-29T09:09:37.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f567cc00700 1 -- 172.21.15.99:0/2461294819 learned_addr learned my addr 172.21.15.99:0/2461294819 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:37.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5677e00700 1 --2- 172.21.15.99:0/2461294819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5678160770 0x7f56780b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5676a00700 1 -- 172.21.15.99:0/2461294819 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220628860 0 0) 0x7f56780d1520 con 0x7f56780d3550 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5676a00700 1 -- 172.21.15.99:0/2461294819 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f565c0035b0 con 0x7f56780d3550 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5676a00700 1 -- 172.21.15.99:0/2461294819 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499773575 0 0) 0x7f56780d1800 con 0x7f56780d0d60 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5676a00700 1 -- 172.21.15.99:0/2461294819 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56780d1520 con 0x7f56780d0d60 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5677e00700 1 -- 172.21.15.99:0/2461294819 >> v1:172.21.15.133:6789/0 conn(0x7f56780d0d60 legacy=0x7f56780d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5677e00700 1 -- 172.21.15.99:0/2461294819 >> v1:172.21.15.99:6789/0 conn(0x7f56780d3550 legacy=0x7f56780ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.513+0000 7f5677e00700 1 -- 172.21.15.99:0/2461294819 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56780d1dc0 con 0x7f5678160770 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f5677e00700 1 --2- 172.21.15.99:0/2461294819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5678160770 0x7f56780b3cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5668009870 tx=0x7f5668009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2cf9a49cbf9f088 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f5676a00700 1 -- 172.21.15.99:0/2461294819 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668011830 con 0x7f5678160770 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f5676a00700 1 -- 172.21.15.99:0/2461294819 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5668011990 con 0x7f5678160770 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f5676a00700 1 -- 172.21.15.99:0/2461294819 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668011d20 con 0x7f5678160770 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f567ec00700 1 -- 172.21.15.99:0/2461294819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5678160770 msgr2=0x7f56780b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f567ec00700 1 --2- 172.21.15.99:0/2461294819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5678160770 0x7f56780b3cc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5668009870 tx=0x7f5668009b80 comp rx=0 tx=0).stop 2024-03-29T09:09:37.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f567ec00700 1 -- 172.21.15.99:0/2461294819 shutdown_connections 2024-03-29T09:09:37.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f567ec00700 1 --2- 172.21.15.99:0/2461294819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5678160770 0x7f56780b3cc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f567ec00700 1 -- 172.21.15.99:0/2461294819 >> 172.21.15.99:0/2461294819 conn(0x7f567808afd0 msgr2=0x7f567809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:37.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.514+0000 7f567ec00700 1 -- 172.21.15.99:0/2461294819 shutdown_connections 2024-03-29T09:09:37.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.515+0000 7f567ec00700 1 -- 172.21.15.99:0/2461294819 wait complete. 2024-03-29T09:09:37.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.516+0000 7f567ec00700 1 Processor -- start 2024-03-29T09:09:37.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.516+0000 7f567ec00700 1 -- start start 2024-03-29T09:09:37.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.517+0000 7f567ec00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56780d0d60 0x7f56780c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.517+0000 7f567ec00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56780d3550 0x7f5678149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.517+0000 7f567ec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678160770 0x7f56780cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.517+0000 7f567ec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567815b9f0 con 0x7f5678160770 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.517+0000 7f567ec00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56780cd1f0 con 0x7f56780d3550 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.517+0000 7f567ec00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5678149d60 con 0x7f56780d0d60 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678160770 0x7f56780cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678160770 0x7f56780cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53698/0 (socket says 172.21.15.99:53698) 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 -- 172.21.15.99:0/844779606 learned_addr learned my addr 172.21.15.99:0/844779606 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f5677e00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56780d0d60 0x7f56780c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f5677400700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56780d3550 0x7f5678149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 -- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56780d0d60 msgr2=0x7f56780c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56780d0d60 0x7f56780c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 -- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56780d3550 msgr2=0x7f5678149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56780d3550 0x7f5678149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 -- 172.21.15.99:0/844779606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680094a0 con 0x7f5678160770 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f5677400700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56780d3550 0x7f5678149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f5677e00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56780d0d60 0x7f56780c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f567cc00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678160770 0x7f56780cf7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f567000cb70 tx=0x7f567000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.518+0000 7f566fe00700 1 -- 172.21.15.99:0/844779606 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000aaf0 con 0x7f5678160770 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.519+0000 7f566fe00700 1 -- 172.21.15.99:0/844779606 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567000ac50 con 0x7f5678160770 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.519+0000 7f566fe00700 1 -- 172.21.15.99:0/844779606 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5670012400 con 0x7f5678160770 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.519+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567815b370 con 0x7f5678160770 2024-03-29T09:09:37.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.519+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56780ccc30 con 0x7f5678160770 2024-03-29T09:09:37.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.520+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567815bb30 con 0x7f5678160770 2024-03-29T09:09:37.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.521+0000 7f566fe00700 1 -- 172.21.15.99:0/844779606 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f567001b3b0 con 0x7f5678160770 2024-03-29T09:09:37.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.522+0000 7f566fe00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f565806ac90 0x7f565806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.522+0000 7f566fe00700 1 -- 172.21.15.99:0/844779606 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f56700894d0 con 0x7f5678160770 2024-03-29T09:09:37.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.522+0000 7f5677e00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f565806ac90 0x7f565806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.523+0000 7f5677e00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f565806ac90 0x7f565806d150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56680097c0 tx=0x7f566801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.526+0000 7f566fe00700 1 -- 172.21.15.99:0/844779606 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f567815bb30 con 0x7f5678160770 2024-03-29T09:09:37.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.728+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f567815bb30 con 0x7f5678160770 2024-03-29T09:09:37.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.729+0000 7f566fe00700 1 -- 172.21.15.99:0/844779606 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f567815bb30 con 0x7f5678160770 2024-03-29T09:09:37.728 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f565806ac90 msgr2=0x7f565806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f565806ac90 0x7f565806d150 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56680097c0 tx=0x7f566801c040 comp rx=0 tx=0).stop 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678160770 msgr2=0x7f56780cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678160770 0x7f56780cf7f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f567000cb70 tx=0x7f567000cf30 comp rx=0 tx=0).stop 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 shutdown_connections 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f565806ac90 0x7f565806d150 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5678160770 0x7f56780cf7f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56780d0d60 0x7f56780c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 --2- 172.21.15.99:0/844779606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56780d3550 0x7f5678149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 >> 172.21.15.99:0/844779606 conn(0x7f567808afd0 msgr2=0x7f56780b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 shutdown_connections 2024-03-29T09:09:37.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.731+0000 7f567ec00700 1 -- 172.21.15.99:0/844779606 wait complete. 2024-03-29T09:09:37.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:09:37.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.941+0000 7ff050800700 1 Processor -- start 2024-03-29T09:09:37.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.941+0000 7ff050800700 1 -- start start 2024-03-29T09:09:37.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff050800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0481608e0 0x7ff0480d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff050800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0480d1b20 0x7ff0480ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff050800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0480cddc0 con 0x7ff048160cc0 2024-03-29T09:09:37.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff050800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0480ce0a0 con 0x7ff0480d1b20 2024-03-29T09:09:37.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff050800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480ce380 con 0x7ff0481608e0 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff04e800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0480d1b20 0x7ff0480ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff04e800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff0480d1b20 0x7ff0480ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51794/0 (socket says 172.21.15.99:51794) 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.942+0000 7ff04e800700 1 -- 172.21.15.99:0/416327987 learned_addr learned my addr 172.21.15.99:0/416327987 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04de00700 1 --2- 172.21.15.99:0/416327987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0481608e0 0x7ff0480d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04e800700 1 -- 172.21.15.99:0/416327987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0481608e0 msgr2=0x7ff0480d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04e800700 1 --2- 172.21.15.99:0/416327987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0481608e0 0x7ff0480d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04e800700 1 -- 172.21.15.99:0/416327987 >> v1:172.21.15.133:6789/0 conn(0x7ff048160cc0 legacy=0x7ff0480d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04e800700 1 -- 172.21.15.99:0/416327987 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0480ce660 con 0x7ff0480d1b20 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04e800700 1 --2- 172.21.15.99:0/416327987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480ceb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff044009e50 tx=0x7ff04400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c7f90820408016 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04ca00700 1 -- 172.21.15.99:0/416327987 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400e070 con 0x7ff0480d1b20 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04ca00700 1 -- 172.21.15.99:0/416327987 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04400b9c0 con 0x7ff0480d1b20 2024-03-29T09:09:37.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.943+0000 7ff04ca00700 1 -- 172.21.15.99:0/416327987 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04400bd70 con 0x7ff0480d1b20 2024-03-29T09:09:37.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 -- 172.21.15.99:0/416327987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 msgr2=0x7ff0480ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 --2- 172.21.15.99:0/416327987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480ceb80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff044009e50 tx=0x7ff04400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:37.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 -- 172.21.15.99:0/416327987 shutdown_connections 2024-03-29T09:09:37.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 --2- 172.21.15.99:0/416327987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480ceb80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 --2- 172.21.15.99:0/416327987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff0481608e0 0x7ff0480d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 -- 172.21.15.99:0/416327987 >> 172.21.15.99:0/416327987 conn(0x7ff04809c490 msgr2=0x7ff0480d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 -- 172.21.15.99:0/416327987 shutdown_connections 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.944+0000 7ff050800700 1 -- 172.21.15.99:0/416327987 wait complete. 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.946+0000 7ff050800700 1 Processor -- start 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.946+0000 7ff050800700 1 -- start start 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff050800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff050800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0481608e0 0x7ff0480ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff050800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff048160cc0 0x7ff0480ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff050800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480cce00 con 0x7ff0480d1b20 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff050800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480cc460 con 0x7ff0481608e0 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff050800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0480cc740 con 0x7ff048160cc0 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51802/0 (socket says 172.21.15.99:51802) 2024-03-29T09:09:37.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 -- 172.21.15.99:0/4151247848 learned_addr learned my addr 172.21.15.99:0/4151247848 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04d400700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0481608e0 0x7ff0480ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04e800700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff048160cc0 0x7ff0480ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 -- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff048160cc0 msgr2=0x7ff0480ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff048160cc0 0x7ff0480ca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 -- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0481608e0 msgr2=0x7ff0480ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0481608e0 0x7ff0480ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.947+0000 7ff04de00700 1 -- 172.21.15.99:0/4151247848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044009b30 con 0x7ff0480d1b20 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.948+0000 7ff04de00700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480cd220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e810 tx=0x7ff03c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.948+0000 7ff042000700 1 -- 172.21.15.99:0/4151247848 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00c070 con 0x7ff0480d1b20 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.948+0000 7ff042000700 1 -- 172.21.15.99:0/4151247848 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03c013610 con 0x7ff0480d1b20 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.948+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0480cca20 con 0x7ff0480d1b20 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.948+0000 7ff042000700 1 -- 172.21.15.99:0/4151247848 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0139c0 con 0x7ff0480d1b20 2024-03-29T09:09:37.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.948+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0481eff10 con 0x7ff0480d1b20 2024-03-29T09:09:37.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.949+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0480cd070 con 0x7ff0480d1b20 2024-03-29T09:09:37.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.950+0000 7ff042000700 1 -- 172.21.15.99:0/4151247848 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff03c013b20 con 0x7ff0480d1b20 2024-03-29T09:09:37.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.951+0000 7ff042000700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff02c06ac90 0x7ff02c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:37.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.951+0000 7ff042000700 1 -- 172.21.15.99:0/4151247848 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff03c0894a0 con 0x7ff0480d1b20 2024-03-29T09:09:37.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.951+0000 7ff04d400700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff02c06ac90 0x7ff02c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:37.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.951+0000 7ff04d400700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff02c06ac90 0x7ff02c06d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff048145f50 tx=0x7ff038009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:37.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:37.955+0000 7ff042000700 1 -- 172.21.15.99:0/4151247848 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff03c0542f0 con 0x7ff0480d1b20 2024-03-29T09:09:38.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.204+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0480677a0 con 0x7ff0480d1b20 2024-03-29T09:09:38.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.204+0000 7ff042000700 1 -- 172.21.15.99:0/4151247848 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7ff03c057910 con 0x7ff0480d1b20 2024-03-29T09:09:38.204 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:38.204 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:09:38.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff02c06ac90 msgr2=0x7ff02c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff02c06ac90 0x7ff02c06d150 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff048145f50 tx=0x7ff038009040 comp rx=0 tx=0).stop 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 msgr2=0x7ff0480cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480cd220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e810 tx=0x7ff03c00eb20 comp rx=0 tx=0).stop 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 shutdown_connections 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff02c06ac90 0x7ff02c06d150 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff048160cc0 0x7ff0480ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0480d1b20 0x7ff0480cd220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 --2- 172.21.15.99:0/4151247848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff0481608e0 0x7ff0480ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 >> 172.21.15.99:0/4151247848 conn(0x7ff04809c490 msgr2=0x7ff0480b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 shutdown_connections 2024-03-29T09:09:38.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.207+0000 7ff050800700 1 -- 172.21.15.99:0/4151247848 wait complete. 2024-03-29T09:09:38.208 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:09:38.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:38.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.412+0000 7f15dca00700 1 Processor -- start 2024-03-29T09:09:38.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.413+0000 7f15dca00700 1 -- start start 2024-03-29T09:09:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15dca00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f15d80d1b20 0x7f15d80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15dca00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d8160cc0 0x7f15d80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15dca00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d80cddc0 con 0x7f15d81608e0 2024-03-29T09:09:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15dca00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d80ce0a0 con 0x7f15d80d1b20 2024-03-29T09:09:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15dca00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d80ce380 con 0x7f15d8160cc0 2024-03-29T09:09:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15d5400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f15d80d1b20 0x7f15d80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15d5e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f15d81608e0 0x7f15d80d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37554/0 (socket says 172.21.15.99:37554) 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15d5e00700 1 -- 172.21.15.99:0/2140945761 learned_addr learned my addr 172.21.15.99:0/2140945761 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.414+0000 7f15d6800700 1 --2- 172.21.15.99:0/2140945761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d8160cc0 0x7f15d80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d5400700 1 -- 172.21.15.99:0/2140945761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d8160cc0 msgr2=0x7f15d80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d5400700 1 --2- 172.21.15.99:0/2140945761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d8160cc0 0x7f15d80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d5400700 1 -- 172.21.15.99:0/2140945761 >> v1:172.21.15.99:6789/0 conn(0x7f15d81608e0 legacy=0x7f15d80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d4a00700 1 -- 172.21.15.99:0/2140945761 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195074604 0 0) 0x7f15d80cddc0 con 0x7f15d81608e0 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d5400700 1 -- 172.21.15.99:0/2140945761 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d80ce660 con 0x7f15d80d1b20 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d5400700 1 --2- 172.21.15.99:0/2140945761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d80d0150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009840 tx=0x7f15c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b46f1eb504ee536 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d4a00700 1 -- 172.21.15.99:0/2140945761 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4011830 con 0x7f15d80d1b20 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d4a00700 1 -- 172.21.15.99:0/2140945761 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c4011990 con 0x7f15d80d1b20 2024-03-29T09:09:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.415+0000 7f15d4a00700 1 -- 172.21.15.99:0/2140945761 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4011d20 con 0x7f15d80d1b20 2024-03-29T09:09:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 -- 172.21.15.99:0/2140945761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 msgr2=0x7f15d80d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 --2- 172.21.15.99:0/2140945761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d80d0150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009840 tx=0x7f15c4009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 -- 172.21.15.99:0/2140945761 shutdown_connections 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 --2- 172.21.15.99:0/2140945761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d8160cc0 0x7f15d80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 --2- 172.21.15.99:0/2140945761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d80d0150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 -- 172.21.15.99:0/2140945761 >> 172.21.15.99:0/2140945761 conn(0x7f15d809c490 msgr2=0x7f15d80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 -- 172.21.15.99:0/2140945761 shutdown_connections 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.416+0000 7f15dca00700 1 -- 172.21.15.99:0/2140945761 wait complete. 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.417+0000 7f15dca00700 1 Processor -- start 2024-03-29T09:09:38.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.418+0000 7f15dca00700 1 -- start start 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.418+0000 7f15dca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d81534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.418+0000 7f15dca00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d81608e0 0x7f15d8155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.418+0000 7f15dca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d8160cc0 0x7f15d80b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.418+0000 7f15dca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81539e0 con 0x7f15d80d1b20 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.418+0000 7f15dca00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d8155f40 con 0x7f15d8160cc0 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.418+0000 7f15dca00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d8156220 con 0x7f15d81608e0 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d81534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d81534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51810/0 (socket says 172.21.15.99:51810) 2024-03-29T09:09:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 -- 172.21.15.99:0/2214667599 learned_addr learned my addr 172.21.15.99:0/2214667599 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5400700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d81608e0 0x7f15d8155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d6800700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d8160cc0 0x7f15d80b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 -- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d81608e0 msgr2=0x7f15d8155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d81608e0 0x7f15d8155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 -- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d8160cc0 msgr2=0x7f15d80b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d8160cc0 0x7f15d80b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 -- 172.21.15.99:0/2214667599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c40094a0 con 0x7f15d80d1b20 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d6800700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d8160cc0 0x7f15d80b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15d5e00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d81534a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f15c80011e0 tx=0x7f15c800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.419+0000 7f15ce000700 1 -- 172.21.15.99:0/2214667599 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8004c80 con 0x7f15d80d1b20 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.420+0000 7f15ce000700 1 -- 172.21.15.99:0/2214667599 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8004e00 con 0x7f15d80d1b20 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.420+0000 7f15ce000700 1 -- 172.21.15.99:0/2214667599 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8007310 con 0x7f15d80d1b20 2024-03-29T09:09:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.420+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d8156500 con 0x7f15d80d1b20 2024-03-29T09:09:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.420+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d8156ac0 con 0x7f15d80d1b20 2024-03-29T09:09:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.421+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d80677a0 con 0x7f15d80d1b20 2024-03-29T09:09:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.422+0000 7f15ce000700 1 -- 172.21.15.99:0/2214667599 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f15c8007470 con 0x7f15d80d1b20 2024-03-29T09:09:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.423+0000 7f15ce000700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f15b806ac90 0x7f15b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.423+0000 7f15ce000700 1 -- 172.21.15.99:0/2214667599 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f15c80893a0 con 0x7f15d80d1b20 2024-03-29T09:09:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.423+0000 7f15d5400700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f15b806ac90 0x7f15b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.424+0000 7f15d5400700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f15b806ac90 0x7f15b806d150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f15c40090b0 tx=0x7f15c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:38.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.428+0000 7f15ce000700 1 -- 172.21.15.99:0/2214667599 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f15c8054290 con 0x7f15d80d1b20 2024-03-29T09:09:38.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.634+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15d80c9f90 con 0x7f15d80d1b20 2024-03-29T09:09:38.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.635+0000 7f15ce000700 1 -- 172.21.15.99:0/2214667599 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+17882 (secure 0 0 0) 0x7f15c80578b0 con 0x7f15d80d1b20 2024-03-29T09:09:38.634 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:38.634 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":24,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:01.914219+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f15b806ac90 msgr2=0x7f15b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f15b806ac90 0x7f15b806d150 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f15c40090b0 tx=0x7f15c4008030 comp rx=0 tx=0).stop 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 msgr2=0x7f15d81534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d81534a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f15c80011e0 tx=0x7f15c800cae0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 shutdown_connections 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f15b806ac90 0x7f15b806d150 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15d8160cc0 0x7f15d80b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15d80d1b20 0x7f15d81534a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 --2- 172.21.15.99:0/2214667599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15d81608e0 0x7f15d8155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.637+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 >> 172.21.15.99:0/2214667599 conn(0x7f15d809c490 msgr2=0x7f15d806c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:38.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.638+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 shutdown_connections 2024-03-29T09:09:38.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.638+0000 7f15dca00700 1 -- 172.21.15.99:0/2214667599 wait complete. 2024-03-29T09:09:38.655 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:09:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.845+0000 7f8d44400700 1 Processor -- start 2024-03-29T09:09:38.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.846+0000 7f8d44400700 1 -- start start 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d44400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d3c1608e0 0x7f8d3c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d44400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8d3c0d1b20 0x7f8d3c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d44400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3c0cddc0 con 0x7f8d3c160cc0 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d44400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c0ce0a0 con 0x7f8d3c1608e0 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d44400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d3c0ce380 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d41a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d3c1608e0 0x7f8d3c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d41a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d3c1608e0 0x7f8d3c0d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51824/0 (socket says 172.21.15.99:51824) 2024-03-29T09:09:38.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d41a00700 1 -- 172.21.15.99:0/3107385759 learned_addr learned my addr 172.21.15.99:0/3107385759 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d42400700 1 --2- 172.21.15.99:0/3107385759 >> v2:172.21.15.133:3300/0 conn(0x7f8d3c0d1b20 0x7f8d3c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.847+0000 7f8d41a00700 1 -- 172.21.15.99:0/3107385759 >> v2:172.21.15.133:3300/0 conn(0x7f8d3c0d1b20 msgr2=0x7f8d3c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.848+0000 7f8d41a00700 1 --2- 172.21.15.99:0/3107385759 >> v2:172.21.15.133:3300/0 conn(0x7f8d3c0d1b20 0x7f8d3c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.848+0000 7f8d41a00700 1 -- 172.21.15.99:0/3107385759 >> v1:172.21.15.133:6789/0 conn(0x7f8d3c160cc0 legacy=0x7f8d3c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.848+0000 7f8d41a00700 1 -- 172.21.15.99:0/3107385759 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3c0ce660 con 0x7f8d3c1608e0 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.848+0000 7f8d41a00700 1 --2- 172.21.15.99:0/3107385759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c1608e0 0x7f8d3c0d0150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c09cb70 tx=0x7f8d30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b6d0e4a7be4da5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.848+0000 7f8d3be00700 1 -- 172.21.15.99:0/3107385759 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000b870 con 0x7f8d3c1608e0 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.848+0000 7f8d3be00700 1 -- 172.21.15.99:0/3107385759 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3000b9d0 con 0x7f8d3c1608e0 2024-03-29T09:09:38.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.848+0000 7f8d3be00700 1 -- 172.21.15.99:0/3107385759 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000bd60 con 0x7f8d3c1608e0 2024-03-29T09:09:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 -- 172.21.15.99:0/3107385759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c1608e0 msgr2=0x7f8d3c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 --2- 172.21.15.99:0/3107385759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c1608e0 0x7f8d3c0d0150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c09cb70 tx=0x7f8d30009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 -- 172.21.15.99:0/3107385759 shutdown_connections 2024-03-29T09:09:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 --2- 172.21.15.99:0/3107385759 >> v2:172.21.15.133:3300/0 conn(0x7f8d3c0d1b20 0x7f8d3c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 --2- 172.21.15.99:0/3107385759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c1608e0 0x7f8d3c0d0150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 -- 172.21.15.99:0/3107385759 >> 172.21.15.99:0/3107385759 conn(0x7f8d3c09c490 msgr2=0x7f8d3c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 -- 172.21.15.99:0/3107385759 shutdown_connections 2024-03-29T09:09:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.849+0000 7f8d44400700 1 -- 172.21.15.99:0/3107385759 wait complete. 2024-03-29T09:09:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.850+0000 7f8d44400700 1 Processor -- start 2024-03-29T09:09:38.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.851+0000 7f8d44400700 1 -- start start 2024-03-29T09:09:38.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.851+0000 7f8d44400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c0d1b20 0x7f8d3c154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.851+0000 7f8d44400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d3c1608e0 0x7f8d3c0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.851+0000 7f8d44400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3c160cc0 0x7f8d3c0b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d44400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c15b180 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d44400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c15ba20 con 0x7f8d3c160cc0 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d44400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3c154e60 con 0x7f8d3c1608e0 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c0d1b20 0x7f8d3c154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c0d1b20 0x7f8d3c154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51840/0 (socket says 172.21.15.99:51840) 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 -- 172.21.15.99:0/2857897385 learned_addr learned my addr 172.21.15.99:0/2857897385 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d42400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3c160cc0 0x7f8d3c0b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41000700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d3c1608e0 0x7f8d3c0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 -- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d3c1608e0 msgr2=0x7f8d3c0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d3c1608e0 0x7f8d3c0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 -- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3c160cc0 msgr2=0x7f8d3c0b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3c160cc0 0x7f8d3c0b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 -- 172.21.15.99:0/2857897385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30009550 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d42400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3c160cc0 0x7f8d3c0b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41000700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d3c1608e0 0x7f8d3c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.852+0000 7f8d41a00700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c0d1b20 0x7f8d3c154920 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000c00 tx=0x7f8d30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.853+0000 7f8d39600700 1 -- 172.21.15.99:0/2857897385 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004540 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.853+0000 7f8d39600700 1 -- 172.21.15.99:0/2857897385 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d300036a0 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.853+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3c0cb660 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.853+0000 7f8d39600700 1 -- 172.21.15.99:0/2857897385 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30003a30 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.853+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3c15b6b0 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.854+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3c0677a0 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.855+0000 7f8d39600700 1 -- 172.21.15.99:0/2857897385 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d30021070 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.855+0000 7f8d39600700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d1c06ac90 0x7f8d1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.856+0000 7f8d39600700 1 -- 172.21.15.99:0/2857897385 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8d3008adc0 con 0x7f8d3c0d1b20 2024-03-29T09:09:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.856+0000 7f8d41000700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d1c06ac90 0x7f8d1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:38.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.856+0000 7f8d41000700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d1c06ac90 0x7f8d1c06d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c155d70 tx=0x7f8d28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:38.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:38.860+0000 7f8d39600700 1 -- 172.21.15.99:0/2857897385 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d30055dc0 con 0x7f8d3c0d1b20 2024-03-29T09:09:39.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.106+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d3c0d0700 con 0x7f8d3c0d1b20 2024-03-29T09:09:39.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.106+0000 7f8d39600700 1 -- 172.21.15.99:0/2857897385 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f8d30015020 con 0x7f8d3c0d1b20 2024-03-29T09:09:39.105 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:39.106 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.108+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d1c06ac90 msgr2=0x7f8d1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.108+0000 7f8d44400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d1c06ac90 0x7f8d1c06d150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c155d70 tx=0x7f8d28009040 comp rx=0 tx=0).stop 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.108+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c0d1b20 msgr2=0x7f8d3c154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.108+0000 7f8d44400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c0d1b20 0x7f8d3c154920 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8d30000c00 tx=0x7f8d30004750 comp rx=0 tx=0).stop 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 shutdown_connections 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d1c06ac90 0x7f8d1c06d150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3c160cc0 0x7f8d3c0b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d3c0d1b20 0x7f8d3c154920 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 --2- 172.21.15.99:0/2857897385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d3c1608e0 0x7f8d3c0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 >> 172.21.15.99:0/2857897385 conn(0x7f8d3c09c490 msgr2=0x7f8d3c15fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:39.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 shutdown_connections 2024-03-29T09:09:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.109+0000 7f8d44400700 1 -- 172.21.15.99:0/2857897385 wait complete. 2024-03-29T09:09:39.109 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:09:39.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T09:08:57.386923+0000', 'modified': '2024-03-29T09:09:24.917304+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6835/440339259', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 440339259}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 440339259}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T09:09:39.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T09:09:39.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-29T09:09:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.723+0000 7f79aea00700 1 Processor -- start 2024-03-29T09:09:39.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.723+0000 7f79aea00700 1 -- start start 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79aea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79a81608e0 0x7f79a80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79aea00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79a8160cc0 0x7f79a80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79aea00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79aea00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a80cddc0 con 0x7f79a81608e0 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79aea00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a80ce0a0 con 0x7f79a8160cc0 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79aea00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80ce380 con 0x7f79a80d3a70 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79a7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79a81608e0 0x7f79a80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79a7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79a81608e0 0x7f79a80d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51856/0 (socket says 172.21.15.99:51856) 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79a7e00700 1 -- 172.21.15.99:0/3501220131 learned_addr learned my addr 172.21.15.99:0/3501220131 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79a7400700 1 --2- 172.21.15.99:0/3501220131 >> v2:172.21.15.133:3300/0 conn(0x7f79a8160cc0 0x7f79a80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79a7e00700 1 -- 172.21.15.99:0/3501220131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 msgr2=0x7f79a80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.724+0000 7f79aca00700 1 --2- 172.21.15.99:0/3501220131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a7e00700 1 --2- 172.21.15.99:0/3501220131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a7e00700 1 -- 172.21.15.99:0/3501220131 >> v2:172.21.15.133:3300/0 conn(0x7f79a8160cc0 msgr2=0x7f79a80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a7e00700 1 --2- 172.21.15.99:0/3501220131 >> v2:172.21.15.133:3300/0 conn(0x7f79a8160cc0 0x7f79a80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a7e00700 1 -- 172.21.15.99:0/3501220131 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a80ce660 con 0x7f79a81608e0 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a7400700 1 --2- 172.21.15.99:0/3501220131 >> v2:172.21.15.133:3300/0 conn(0x7f79a8160cc0 0x7f79a80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79aca00700 1 --2- 172.21.15.99:0/3501220131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a7e00700 1 --2- 172.21.15.99:0/3501220131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80d0150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79a806ac50 tx=0x7f7998009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793f98299740fd1c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a6a00700 1 -- 172.21.15.99:0/3501220131 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800b870 con 0x7f79a81608e0 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a6a00700 1 -- 172.21.15.99:0/3501220131 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799800b9d0 con 0x7f79a81608e0 2024-03-29T09:09:39.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79a6a00700 1 -- 172.21.15.99:0/3501220131 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800bd60 con 0x7f79a81608e0 2024-03-29T09:09:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79aea00700 1 -- 172.21.15.99:0/3501220131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 msgr2=0x7f79a80d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.725+0000 7f79aea00700 1 --2- 172.21.15.99:0/3501220131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80d0150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79a806ac50 tx=0x7f7998009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.726+0000 7f79aea00700 1 -- 172.21.15.99:0/3501220131 shutdown_connections 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.726+0000 7f79aea00700 1 --2- 172.21.15.99:0/3501220131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.726+0000 7f79aea00700 1 --2- 172.21.15.99:0/3501220131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80d0150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.726+0000 7f79aea00700 1 --2- 172.21.15.99:0/3501220131 >> v2:172.21.15.133:3300/0 conn(0x7f79a8160cc0 0x7f79a80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.726+0000 7f79aea00700 1 -- 172.21.15.99:0/3501220131 >> 172.21.15.99:0/3501220131 conn(0x7f79a809c490 msgr2=0x7f79a80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.726+0000 7f79aea00700 1 -- 172.21.15.99:0/3501220131 shutdown_connections 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.726+0000 7f79aea00700 1 -- 172.21.15.99:0/3501220131 wait complete. 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.727+0000 7f79aea00700 1 Processor -- start 2024-03-29T09:09:39.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.728+0000 7f79aea00700 1 -- start start 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.728+0000 7f79aea00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.728+0000 7f79aea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79aea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a8160cc0 0x7f79a815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79aea00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80ccd60 con 0x7f79a81608e0 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79aea00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a815ce00 con 0x7f79a8160cc0 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79aea00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80cc0e0 con 0x7f79a80d3a70 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51870/0 (socket says 172.21.15.99:51870) 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 -- 172.21.15.99:0/1821781030 learned_addr learned my addr 172.21.15.99:0/1821781030 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7e00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79aca00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a8160cc0 0x7f79a815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 -- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 msgr2=0x7f79a80cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 -- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a8160cc0 msgr2=0x7f79a815c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a8160cc0 0x7f79a815c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79a7400700 1 -- 172.21.15.99:0/1821781030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998009550 con 0x7f79a81608e0 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.729+0000 7f79aca00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a8160cc0 0x7f79a815c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.730+0000 7f79a7400700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80cd180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f799400c7f0 tx=0x7f799400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.730+0000 7f799fe00700 1 -- 172.21.15.99:0/1821781030 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994011800 con 0x7f79a81608e0 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.730+0000 7f799fe00700 1 -- 172.21.15.99:0/1821781030 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7994011960 con 0x7f79a81608e0 2024-03-29T09:09:39.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.730+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a80cc3c0 con 0x7f79a81608e0 2024-03-29T09:09:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.730+0000 7f799fe00700 1 -- 172.21.15.99:0/1821781030 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994011cf0 con 0x7f79a81608e0 2024-03-29T09:09:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.730+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a80cc980 con 0x7f79a81608e0 2024-03-29T09:09:39.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.732+0000 7f799fe00700 1 -- 172.21.15.99:0/1821781030 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7994018360 con 0x7f79a81608e0 2024-03-29T09:09:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.733+0000 7f799fe00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798806ac90 0x7f798806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.733+0000 7f799fe00700 1 -- 172.21.15.99:0/1821781030 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7994089450 con 0x7f79a81608e0 2024-03-29T09:09:39.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.734+0000 7f79a7e00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798806ac90 0x7f798806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.734+0000 7f79a7e00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798806ac90 0x7f798806d150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7998000c00 tx=0x7f7998008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.735+0000 7f79c2485b80 1 Processor -- start 2024-03-29T09:09:39.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.736+0000 7f79c2485b80 1 -- start start 2024-03-29T09:09:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.736+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93382c20 0x558c93386a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.736+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93387fc0 0x558c9335e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.736+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.736+0000 7f79c2485b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c932b7940 con 0x558c93382c20 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.736+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c932b5300 con 0x558c9335d980 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.736+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c932b0bf0 con 0x558c93387fc0 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93382c20 0x558c93386a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93382c20 0x558c93386a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51872/0 (socket says 172.21.15.99:51872) 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 -- 172.21.15.99:0/645471201 learned_addr learned my addr 172.21.15.99:0/645471201 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79aca00700 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7400700 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93387fc0 0x558c9335e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 -- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93387fc0 msgr2=0x558c9335e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93387fc0 0x558c9335e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 -- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 msgr2=0x558c9335eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 -- 172.21.15.99:0/645471201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c930deb10 con 0x558c93382c20 2024-03-29T09:09:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79aca00700 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7400700 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93387fc0 0x558c9335e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f79a7e00700 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93382c20 0x558c93386a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7998009fd0 tx=0x7f7998010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b5fd405d33ef83 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f7993e00700 1 -- 172.21.15.99:0/645471201 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998020070 con 0x558c93382c20 2024-03-29T09:09:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f7993e00700 1 -- 172.21.15.99:0/645471201 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79980036a0 con 0x558c93382c20 2024-03-29T09:09:39.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.737+0000 7f7993e00700 1 -- 172.21.15.99:0/645471201 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998003a10 con 0x558c93382c20 2024-03-29T09:09:39.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 -- 172.21.15.99:0/645471201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93382c20 msgr2=0x558c93386a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93382c20 0x558c93386a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7998009fd0 tx=0x7f7998010000 comp rx=0 tx=0).stop 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 -- 172.21.15.99:0/645471201 shutdown_connections 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93382c20 0x558c93386a60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 --2- 172.21.15.99:0/645471201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93387fc0 0x558c9335e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 -- 172.21.15.99:0/645471201 >> 172.21.15.99:0/645471201 conn(0x558c932be650 msgr2=0x558c933a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 -- 172.21.15.99:0/645471201 shutdown_connections 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.738+0000 7f79c2485b80 1 -- 172.21.15.99:0/645471201 wait complete. 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.739+0000 7f79c2485b80 1 -- start start 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.739+0000 7f79c2485b80 1 Processor -- start 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c9335d980 0x558c933863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93382c20 0x558c93383530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79c2485b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9324cd30 con 0x558c93387fc0 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9324c9d0 con 0x558c93382c20 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c93134650 con 0x558c9335d980 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79c2485b80 10 client.0 _ll_register_callbacks cb 0x558c93256420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79aca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79aca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51878/0 (socket says 172.21.15.99:51878) 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79aca00700 1 -- 172.21.15.99:0/2983459476 learned_addr learned my addr 172.21.15.99:0/2983459476 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79a7e00700 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c9335d980 0x558c933863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.740+0000 7f79a7400700 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93382c20 0x558c93383530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f79aca00700 1 -- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c9335d980 msgr2=0x558c933863c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f79aca00700 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c9335d980 0x558c933863c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f79aca00700 1 -- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93382c20 msgr2=0x558c93383530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f79aca00700 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93382c20 0x558c93383530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f79aca00700 1 -- 172.21.15.99:0/2983459476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a80ce660 con 0x558c93387fc0 2024-03-29T09:09:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f79aca00700 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b5330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f79a000ffb0 tx=0x7f79a0007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4effdda1c49bc65 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f7990c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f7990c00700 1 -- 172.21.15.99:0/2983459476 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0007e80 con 0x558c93387fc0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f7990c00700 1 -- 172.21.15.99:0/2983459476 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f79a000aaa0 con 0x558c93387fc0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.741+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c93271330 con 0x558c93387fc0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f7990c00700 1 -- 172.21.15.99:0/2983459476 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a000ac60 con 0x558c93387fc0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 10 client.4815 fetch_fsmap learned FSMap version 8 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x558c93276e50 con 0x558c93387fc0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f7990c00700 1 -- 172.21.15.99:0/2983459476 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558c93271330 con 0x558c93387fc0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f7990c00700 1 -- 172.21.15.99:0/2983459476 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f79a0017410 con 0x558c93387fc0 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 10 client.4815 fetch_fsmap finished waiting for FSMap version 8 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 10 client.4815 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 20 client.4815 populate_metadata read hostname 'smithi099' 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c9338d960 0x558c9338dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 4 client.4815 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T09:09:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 --> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c932a96b0 con 0x558c9338d960 2024-03-29T09:09:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.742+0000 7f79a7e00700 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c9338d960 0x558c9338dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.743+0000 7f7990c00700 10 client.4815 ms_handle_connect on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.743+0000 7f79a7e00700 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c9338d960 0x558c9338dd40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ceb44301fe79b5f server_cookie=62f8a24a56ec7527 in_seq=0 out_seq=0 2024-03-29T09:09:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f7990c00700 1 -- 172.21.15.99:0/2983459476 <== mds.0 v2:172.21.15.133:6834/440339259 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558c932a96b0 con 0x558c9338d960 2024-03-29T09:09:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f7990c00700 10 client.4815 handle_command_reply: tid=0 2024-03-29T09:09:39.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 client.4815 shutdown 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 msgr2=0x558c933b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b5330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f79a000ffb0 tx=0x7f79a0007d50 comp rx=0 tx=0).stop 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 shutdown_connections 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c9338d960 0x558c9338dd40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b5330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c9335d980 0x558c933863c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2983459476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93382c20 0x558c93383530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f7990c00700 0 client.4815 ms_handle_reset on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:39.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 >> 172.21.15.99:0/2983459476 conn(0x558c932be650 msgr2=0x558c9335f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:39.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 shutdown_connections 2024-03-29T09:09:39.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 1 -- 172.21.15.99:0/2983459476 wait complete. 2024-03-29T09:09:39.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.744+0000 7f79c2485b80 20 client.4815 trim_cache size 0 max 16384 2024-03-29T09:09:39.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.762+0000 7f79c2485b80 1 Processor -- start 2024-03-29T09:09:39.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.762+0000 7f79c2485b80 1 -- start start 2024-03-29T09:09:39.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.763+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.763+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c933b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.763+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.763+0000 7f79c2485b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c933522c0 con 0x558c93387fc0 2024-03-29T09:09:39.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.763+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9338a960 con 0x558c9335d980 2024-03-29T09:09:39.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.763+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c9335d500 con 0x558c93382c20 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51894/0 (socket says 172.21.15.99:51894) 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 -- 172.21.15.99:0/2234092731 learned_addr learned my addr 172.21.15.99:0/2234092731 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79a7e00700 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79a7400700 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c933b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 -- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 msgr2=0x558c933b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c933b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 -- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 msgr2=0x558c9335fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 -- 172.21.15.99:0/2234092731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c93136c80 con 0x558c93387fc0 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f79aca00700 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b1e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f79a0002230 tx=0x7f79a0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48d4705bc320440 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:39.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f799d200700 1 -- 172.21.15.99:0/2234092731 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0007620 con 0x558c93387fc0 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f799d200700 1 -- 172.21.15.99:0/2234092731 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0007bd0 con 0x558c93387fc0 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.764+0000 7f799d200700 1 -- 172.21.15.99:0/2234092731 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00178d0 con 0x558c93387fc0 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 -- 172.21.15.99:0/2234092731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 msgr2=0x558c933b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b1e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f79a0002230 tx=0x7f79a0012000 comp rx=0 tx=0).stop 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 -- 172.21.15.99:0/2234092731 shutdown_connections 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c93387fc0 0x558c933b1e00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c9335d980 0x558c9335fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 --2- 172.21.15.99:0/2234092731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c933b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 -- 172.21.15.99:0/2234092731 >> 172.21.15.99:0/2234092731 conn(0x558c932be650 msgr2=0x558c9338dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 -- 172.21.15.99:0/2234092731 shutdown_connections 2024-03-29T09:09:39.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.765+0000 7f79c2485b80 1 -- 172.21.15.99:0/2234092731 wait complete. 2024-03-29T09:09:39.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.766+0000 7f79c2485b80 1 -- start start 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.766+0000 7f79c2485b80 1 Processor -- start 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9335d980 0x558c93410bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c93383000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79c2485b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93387fc0 0x558c93383540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79c2485b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c933a3bb0 con 0x558c9335d980 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9335f030 con 0x558c93387fc0 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79c2485b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c9335ed80 con 0x558c93382c20 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79c2485b80 10 client.0 _ll_register_callbacks cb 0x558c93256420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79a7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9335d980 0x558c93410bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79a7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9335d980 0x558c93410bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51898/0 (socket says 172.21.15.99:51898) 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79a7e00700 1 -- 172.21.15.99:0/4206356085 learned_addr learned my addr 172.21.15.99:0/4206356085 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:39.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.767+0000 7f79aca00700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93387fc0 0x558c93383540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7400700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c93383000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7e00700 1 -- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 msgr2=0x558c93383000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7e00700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c93383000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7e00700 1 -- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93387fc0 msgr2=0x558c93383540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7e00700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93387fc0 0x558c93383540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7e00700 1 -- 172.21.15.99:0/4206356085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c93276e50 con 0x558c9335d980 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79aca00700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93387fc0 0x558c93383540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7400700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c93383000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79a7e00700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9335d980 0x558c93410bb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7998009130 tx=0x7f799800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c520503e87f696 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f7990c00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f7990c00700 1 -- 172.21.15.99:0/4206356085 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998003c10 con 0x558c9335d980 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f7990c00700 1 -- 172.21.15.99:0/4206356085 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7998039040 con 0x558c9335d980 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79980036a0 con 0x558c9335d980 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f7990c00700 1 -- 172.21.15.99:0/4206356085 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7998003d70 con 0x558c9335d980 2024-03-29T09:09:39.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79c2485b80 10 client.4821 fetch_fsmap learned FSMap version 8 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.768+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x558c930dcfe0 con 0x558c9335d980 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f7990c00700 1 -- 172.21.15.99:0/4206356085 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79980036a0 con 0x558c9335d980 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f7990c00700 1 -- 172.21.15.99:0/4206356085 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 8) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f799803e960 con 0x558c9335d980 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f79c2485b80 10 client.4821 fetch_fsmap finished waiting for FSMap version 8 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f79c2485b80 10 client.4821 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f79c2485b80 20 client.4821 populate_metadata read hostname 'smithi099' 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f79c2485b80 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c933a2fb0 0x558c933a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f79c2485b80 4 client.4821 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 --> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558c930ffca0 con 0x558c933a2fb0 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.769+0000 7f79a7400700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c933a2fb0 0x558c933a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f7990c00700 10 client.4821 ms_handle_connect on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79a7400700 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c933a2fb0 0x558c933a3390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcfd3bd076d26ef0 server_cookie=3144f7af08aedb6c in_seq=0 out_seq=0 2024-03-29T09:09:39.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f7990c00700 1 -- 172.21.15.99:0/4206356085 <== mds.0 v2:172.21.15.133:6834/440339259 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x558c930ffca0 con 0x558c933a2fb0 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f7990c00700 10 client.4821 handle_command_reply: tid=0 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 client.4821 shutdown 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9335d980 msgr2=0x558c93410bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9335d980 0x558c93410bb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7998009130 tx=0x7f799800e000 comp rx=0 tx=0).stop 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 shutdown_connections 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:6834/440339259,v1:172.21.15.133:6835/440339259] conn(0x558c933a2fb0 0x558c933a3390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558c93387fc0 0x558c93383540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558c9335d980 0x558c93410bb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f7990c00700 0 client.4821 ms_handle_reset on v2:172.21.15.133:6834/440339259 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 --2- 172.21.15.99:0/4206356085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558c93382c20 0x558c93383000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 >> 172.21.15.99:0/4206356085 conn(0x558c932be650 msgr2=0x558c933a5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 shutdown_connections 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.770+0000 7f79c2485b80 1 -- 172.21.15.99:0/4206356085 wait complete. 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79c2485b80 20 client.4821 trim_cache size 0 max 16384 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "83e3f490-14b9-4a33-8e2d-bbf792b299cc", 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4285, 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-29T09:09:39.775 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-29T09:09:39.776 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 42.379418156, 2024-03-29T09:09:39.776 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 8, 2024-03-29T09:09:39.776 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 24, 2024-03-29T09:09:39.776 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 24, 2024-03-29T09:09:39.776 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 46.893831401 2024-03-29T09:09:39.776 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798806ac90 msgr2=0x7f798806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798806ac90 0x7f798806d150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7998000c00 tx=0x7f7998008030 comp rx=0 tx=0).stop 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 msgr2=0x7f79a80cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80cd180 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f799400c7f0 tx=0x7f799400cb00 comp rx=0 tx=0).stop 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 shutdown_connections 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798806ac90 0x7f798806d150 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a8160cc0 0x7f79a815c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a80d3a70 0x7f79a80cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 --2- 172.21.15.99:0/1821781030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a81608e0 0x7f79a80cd180 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 >> 172.21.15.99:0/1821781030 conn(0x7f79a809c490 msgr2=0x7f79a80cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 shutdown_connections 2024-03-29T09:09:39.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:39.771+0000 7f79aea00700 1 -- 172.21.15.99:0/1821781030 wait complete. 2024-03-29T09:09:39.789 INFO:teuthology.run_tasks:Running task sequential... 2024-03-29T09:09:39.798 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-29T09:09:39.798 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-29T09:09:39.799 INFO:teuthology.task.print:Enabling mgr modules 2024-03-29T09:09:39.799 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-29T09:09:39.819 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-29T09:09:39.819 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-29T09:09:39.819 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-29T09:09:39.819 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T09:09:39.819 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:09:39.819 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-29T09:09:39.819 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T09:09:39.819 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T09:09:39.819 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-29T09:09:39.819 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T09:09:39.820 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T09:09:39.820 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-29T09:09:39.820 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T09:09:39.820 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T09:09:39.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:39.820 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:09:39.861 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-29T09:09:39.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:39.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-29T09:09:39.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:39.969 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-03-29T09:09:40.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:40.080 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-29T09:09:40.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:40.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-03-29T09:09:40.200 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-03-29T09:09:40.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:09:40.206 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-29T09:09:40.206 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:09:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.399+0000 7f6fa3600700 1 Processor -- start 2024-03-29T09:09:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.399+0000 7f6fa3600700 1 -- start start 2024-03-29T09:09:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa3600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f9c0d1a40 0x7f6f9c0cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa3600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa3600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c0d0460 con 0x7f6f9c15e7a0 2024-03-29T09:09:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa3600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0cdf80 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa3600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0d1eb0 con 0x7f6f9c0cfad0 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa1600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6f9be00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f9c0d1a40 0x7f6f9c0cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa1600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c0cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:41750/0 (socket says 172.21.15.99:41750) 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.400+0000 7f6fa1600700 1 -- 172.21.15.99:0/2105466623 learned_addr learned my addr 172.21.15.99:0/2105466623 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6fa1600700 1 -- 172.21.15.99:0/2105466623 >> v2:172.21.15.133:3300/0 conn(0x7f6f9c0d1a40 msgr2=0x7f6f9c0cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6fa1600700 1 --2- 172.21.15.99:0/2105466623 >> v2:172.21.15.133:3300/0 conn(0x7f6f9c0d1a40 0x7f6f9c0cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6fa1600700 1 -- 172.21.15.99:0/2105466623 >> v1:172.21.15.133:6789/0 conn(0x7f6f9c15e7a0 legacy=0x7f6f9c0d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6fa1600700 1 -- 172.21.15.99:0/2105466623 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c15eb80 con 0x7f6f9c0cfad0 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6fa1600700 1 --2- 172.21.15.99:0/2105466623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c0cff20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009e50 tx=0x7f6f9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80616a8475aca562 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6f9b400700 1 -- 172.21.15.99:0/2105466623 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400e070 con 0x7f6f9c0cfad0 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6f9b400700 1 -- 172.21.15.99:0/2105466623 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9400b910 con 0x7f6f9c0cfad0 2024-03-29T09:09:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.401+0000 7f6f9b400700 1 -- 172.21.15.99:0/2105466623 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400bcc0 con 0x7f6f9c0cfad0 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 -- 172.21.15.99:0/2105466623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 msgr2=0x7f6f9c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 --2- 172.21.15.99:0/2105466623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c0cff20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009e50 tx=0x7f6f9400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 -- 172.21.15.99:0/2105466623 shutdown_connections 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 --2- 172.21.15.99:0/2105466623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c0cff20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 --2- 172.21.15.99:0/2105466623 >> v2:172.21.15.133:3300/0 conn(0x7f6f9c0d1a40 0x7f6f9c0cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 -- 172.21.15.99:0/2105466623 >> 172.21.15.99:0/2105466623 conn(0x7f6f9c09c480 msgr2=0x7f6f9c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 -- 172.21.15.99:0/2105466623 shutdown_connections 2024-03-29T09:09:40.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.402+0000 7f6fa3600700 1 -- 172.21.15.99:0/2105466623 wait complete. 2024-03-29T09:09:40.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.403+0000 7f6fa3600700 1 Processor -- start 2024-03-29T09:09:40.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.404+0000 7f6fa3600700 1 -- start start 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.404+0000 7f6fa3600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.404+0000 7f6fa3600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f9c0d1a40 0x7f6f9c0c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.404+0000 7f6fa3600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15e7a0 0x7f6f9c0b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.404+0000 7f6fa3600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0c77e0 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.404+0000 7f6fa3600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c155420 con 0x7f6f9c15e7a0 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.404+0000 7f6fa3600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c155700 con 0x7f6f9c0cfad0 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f9c0d1a40 0x7f6f9c0c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f9c0d1a40 0x7f6f9c0c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51910/0 (socket says 172.21.15.99:51910) 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 -- 172.21.15.99:0/1521972611 learned_addr learned my addr 172.21.15.99:0/1521972611 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6fa1600700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15e7a0 0x7f6f9c0b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6fa0c00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 -- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 msgr2=0x7f6f9c1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c1533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 -- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15e7a0 msgr2=0x7f6f9c0b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15e7a0 0x7f6f9c0b7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6f9be00700 1 -- 172.21.15.99:0/1521972611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94009b30 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.405+0000 7f6fa0c00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.406+0000 7f6f9be00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f9c0d1a40 0x7f6f9c0c72a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e9b0 tx=0x7f6f8800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.406+0000 7f6f98c00700 1 -- 172.21.15.99:0/1521972611 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88009710 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.406+0000 7f6f98c00700 1 -- 172.21.15.99:0/1521972611 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8800ee50 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.406+0000 7f6f98c00700 1 -- 172.21.15.99:0/1521972611 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800ca50 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.406+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c1559e0 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.406+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c155fa0 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.407+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c0c6700 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.408+0000 7f6f98c00700 1 -- 172.21.15.99:0/1521972611 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6f8800cbb0 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.409+0000 7f6f98c00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6f7c06ac90 0x7f6f7c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:40.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.409+0000 7f6f98c00700 1 -- 172.21.15.99:0/1521972611 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6f88089670 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.409+0000 7f6fa0c00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6f7c06ac90 0x7f6f7c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:40.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.410+0000 7f6fa0c00700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6f7c06ac90 0x7f6f7c06d150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6f90007fd0 tx=0x7f6f90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:40.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.413+0000 7f6f98c00700 1 -- 172.21.15.99:0/1521972611 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6f88054550 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.664+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6f9c0677a0 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.664+0000 7f6f98c00700 1 -- 172.21.15.99:0/1521972611 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6f8800ce60 con 0x7f6f9c0d1a40 2024-03-29T09:09:40.663 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T09:09:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6f7c06ac90 msgr2=0x7f6f7c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6f7c06ac90 0x7f6f7c06d150 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6f90007fd0 tx=0x7f6f90009040 comp rx=0 tx=0).stop 2024-03-29T09:09:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f9c0d1a40 msgr2=0x7f6f9c0c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f9c0d1a40 0x7f6f9c0c72a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e9b0 tx=0x7f6f8800ecc0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 shutdown_connections 2024-03-29T09:09:40.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6f7c06ac90 0x7f6f7c06d150 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15e7a0 0x7f6f9c0b7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6f9c0cfad0 0x7f6f9c1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 --2- 172.21.15.99:0/1521972611 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f9c0d1a40 0x7f6f9c0c72a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:40.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 >> 172.21.15.99:0/1521972611 conn(0x7f6f9c09c480 msgr2=0x7f6f9c0cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:40.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 shutdown_connections 2024-03-29T09:09:40.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:40.667+0000 7f6fa3600700 1 -- 172.21.15.99:0/1521972611 wait complete. 2024-03-29T09:09:40.684 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:09:40.684 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:09:40.685 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:09:40.685 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T09:09:40.685 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:09:40.685 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:09:40.685 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:09:40.685 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:09:40.685 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:09:40.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:40.685 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42207sec preferred_lft 42207sec 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:09:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:40.729 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T09:09:40.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:40.729 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:40.729 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:40.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add name ceph-brx type bridge 2024-03-29T09:09:40.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr flush dev ceph-brx 2024-03-29T09:09:40.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set ceph-brx up 2024-03-29T09:09:40.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T09:09:40.730 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:40.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:40.958 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:09:41.027 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:09:41.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:41.030 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:09:41.074 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:09:41.074 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:09:41.074 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:09:41.074 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:09:41.074 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:09:41.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:41.076 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:41.076 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:41.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:09:41.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:09:41.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:09:41.076 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:41.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:41.279 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:09:41.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:41.321 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-03-29T09:09:41.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:41.404 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:41.404 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:41.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:09:41.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T09:09:41.404 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:41.579 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T09:09:41.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T09:09:41.579 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:41.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:41.813 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:41.814 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:41.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:09:41.814 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T09:09:41.814 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:41.978 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T09:09:41.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:41.978 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T09:09:42.022 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:09:42.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:42.023 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:09:42.103 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:09:42.209 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:42.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:42.272 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:42.275 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:42.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:42.276 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:09:42.317 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:09:42.317 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T09:09:42.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:09:42.426 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:42.426 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:42.495 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:42.499 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:42.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:42.499 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:09:42.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-29T09:09:42.530+0000 7fbc7ff8e5c0 -1 init, newargv = 0x56047c38f1d0 newargc=15 2024-03-29T09:09:42.531 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[53340]: starting ceph client 2024-03-29T09:09:42.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[53340]: starting fuse 2024-03-29T09:09:42.542 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:09:42.542 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:09:42.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:09:42.542 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:09:42.542 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:09:42.543 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-29T09:09:42.544 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:09:42.721 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.53340.asok 2024-03-29T09:09:42.900 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53340.asok 2024-03-29T09:09:42.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:42.900 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53340.asok status 2024-03-29T09:09:43.119 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "53340", 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:43.120 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4830, 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4830 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:09:43.121 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1093669271 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1093669271 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4830 192.168.0.1:0/1093669271", 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/1093669271", 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:09:43.122 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 8, 2024-03-29T09:09:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 26, 2024-03-29T09:09:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:09:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:09:43.123 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:09:43.123 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:09:43.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:09:43.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.323+0000 7f9f75200700 1 Processor -- start 2024-03-29T09:09:43.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.324+0000 7f9f75200700 1 -- start start 2024-03-29T09:09:43.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f75200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f7015da70 0x7f9f700cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:43.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f75200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f700afde0 con 0x7f9f700d1740 2024-03-29T09:09:43.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f75200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f700d01d0 con 0x7f9f7015d690 2024-03-29T09:09:43.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f75200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f700d04b0 con 0x7f9f7015da70 2024-03-29T09:09:43.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f6e800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9f700d1740 0x7f9f700b0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:37562/0 (socket says 172.21.15.99:37562) 2024-03-29T09:09:43.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f6e800700 1 -- 172.21.15.99:0/468297039 learned_addr learned my addr 172.21.15.99:0/468297039 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:43.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f6d400700 1 --2- 172.21.15.99:0/468297039 >> v2:172.21.15.133:3300/0 conn(0x7f9f7015da70 0x7f9f700cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f6ca00700 1 -- 172.21.15.99:0/468297039 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698337679 0 0) 0x7f9f700afde0 con 0x7f9f700d1740 2024-03-29T09:09:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.325+0000 7f9f6ca00700 1 -- 172.21.15.99:0/468297039 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f540035b0 con 0x7f9f700d1740 2024-03-29T09:09:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6ca00700 1 -- 172.21.15.99:0/468297039 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403176179 0 0) 0x7f9f700d01d0 con 0x7f9f7015d690 2024-03-29T09:09:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6ca00700 1 -- 172.21.15.99:0/468297039 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f700afde0 con 0x7f9f7015d690 2024-03-29T09:09:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6d400700 1 -- 172.21.15.99:0/468297039 >> v1:172.21.15.133:6789/0 conn(0x7f9f7015d690 legacy=0x7f9f700cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6d400700 1 -- 172.21.15.99:0/468297039 >> v1:172.21.15.99:6789/0 conn(0x7f9f700d1740 legacy=0x7f9f700b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:43.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6d400700 1 -- 172.21.15.99:0/468297039 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700b0030 con 0x7f9f7015da70 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6d400700 1 --2- 172.21.15.99:0/468297039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 0x7f9f700cfb60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009840 tx=0x7f9f5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e91c459dd9cf7946 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6ca00700 1 -- 172.21.15.99:0/468297039 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c011830 con 0x7f9f7015da70 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6ca00700 1 -- 172.21.15.99:0/468297039 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c011990 con 0x7f9f7015da70 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.326+0000 7f9f6ca00700 1 -- 172.21.15.99:0/468297039 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c011d20 con 0x7f9f7015da70 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.327+0000 7f9f75200700 1 -- 172.21.15.99:0/468297039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 msgr2=0x7f9f700cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.327+0000 7f9f75200700 1 --2- 172.21.15.99:0/468297039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 0x7f9f700cfb60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009840 tx=0x7f9f5c009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.327+0000 7f9f75200700 1 -- 172.21.15.99:0/468297039 shutdown_connections 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.327+0000 7f9f75200700 1 --2- 172.21.15.99:0/468297039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 0x7f9f700cfb60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.327+0000 7f9f75200700 1 -- 172.21.15.99:0/468297039 >> 172.21.15.99:0/468297039 conn(0x7f9f7009c480 msgr2=0x7f9f700d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.327+0000 7f9f75200700 1 -- 172.21.15.99:0/468297039 shutdown_connections 2024-03-29T09:09:43.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.327+0000 7f9f75200700 1 -- 172.21.15.99:0/468297039 wait complete. 2024-03-29T09:09:43.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.329+0000 7f9f75200700 1 Processor -- start 2024-03-29T09:09:43.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.329+0000 7f9f75200700 1 -- start start 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f75200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f700d1740 0x7f9f700b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f75200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f7015d690 0x7f9f700c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f75200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 0x7f9f700b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f75200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f700b6dc0 con 0x7f9f7015d690 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f75200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f700b6520 con 0x7f9f7015da70 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f75200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f700b6800 con 0x7f9f700d1740 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f7015d690 0x7f9f700c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f7015d690 0x7f9f700c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51914/0 (socket says 172.21.15.99:51914) 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 -- 172.21.15.99:0/1128211460 learned_addr learned my addr 172.21.15.99:0/1128211460 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:43.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6e800700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 0x7f9f700b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6de00700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f700d1740 0x7f9f700b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 -- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f700d1740 msgr2=0x7f9f700b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f700d1740 0x7f9f700b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 -- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 msgr2=0x7f9f700b78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 0x7f9f700b78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 -- 172.21.15.99:0/1128211460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0094a0 con 0x7f9f7015d690 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6de00700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f700d1740 0x7f9f700b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.330+0000 7f9f6d400700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f7015d690 0x7f9f700c94d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009fd0 tx=0x7f9f5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.331+0000 7f9f66000700 1 -- 172.21.15.99:0/1128211460 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c01b870 con 0x7f9f7015d690 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.331+0000 7f9f66000700 1 -- 172.21.15.99:0/1128211460 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c01b9d0 con 0x7f9f7015d690 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.331+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f700b6ae0 con 0x7f9f7015d690 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.331+0000 7f9f66000700 1 -- 172.21.15.99:0/1128211460 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c01bd60 con 0x7f9f7015d690 2024-03-29T09:09:43.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.331+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f700b6210 con 0x7f9f7015d690 2024-03-29T09:09:43.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.332+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f700c6e90 con 0x7f9f7015d690 2024-03-29T09:09:43.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.333+0000 7f9f66000700 1 -- 172.21.15.99:0/1128211460 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f5c023070 con 0x7f9f7015d690 2024-03-29T09:09:43.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.334+0000 7f9f66000700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f5006ac90 0x7f9f5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:43.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.334+0000 7f9f66000700 1 -- 172.21.15.99:0/1128211460 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f5c08a770 con 0x7f9f7015d690 2024-03-29T09:09:43.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.334+0000 7f9f6de00700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f5006ac90 0x7f9f5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:43.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.335+0000 7f9f6de00700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f5006ac90 0x7f9f5006d150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9f600011e0 tx=0x7f9f60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:43.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.338+0000 7f9f66000700 1 -- 172.21.15.99:0/1128211460 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f700c6e90 con 0x7f9f7015d690 2024-03-29T09:09:43.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.590+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9f700c6e90 con 0x7f9f7015d690 2024-03-29T09:09:43.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.590+0000 7f9f66000700 1 -- 172.21.15.99:0/1128211460 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9f700c6e90 con 0x7f9f7015d690 2024-03-29T09:09:43.591 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T09:09:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f5006ac90 msgr2=0x7f9f5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f5006ac90 0x7f9f5006d150 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9f600011e0 tx=0x7f9f60009040 comp rx=0 tx=0).stop 2024-03-29T09:09:43.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f7015d690 msgr2=0x7f9f700c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f7015d690 0x7f9f700c94d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009fd0 tx=0x7f9f5c004750 comp rx=0 tx=0).stop 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 shutdown_connections 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f5006ac90 0x7f9f5006d150 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f7015da70 0x7f9f700b78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f700d1740 0x7f9f700b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 --2- 172.21.15.99:0/1128211460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f7015d690 0x7f9f700c94d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 >> 172.21.15.99:0/1128211460 conn(0x7f9f7009c480 msgr2=0x7f9f700b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 shutdown_connections 2024-03-29T09:09:43.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:43.593+0000 7f9f75200700 1 -- 172.21.15.99:0/1128211460 wait complete. 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:09:43.611 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-29T09:09:43.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:43.612 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:43.655 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42204sec preferred_lft 42204sec 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:43.656 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::3caa:acff:feac:fccd/64 scope link 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run.smithi099.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5832:eeff:fe0b:ee92/64 scope link 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:43.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:43.657 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:09:43.757 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:09:43.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:43.760 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:09:43.805 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:09:43.805 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:09:43.805 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:09:43.805 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:09:43.805 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:09:43.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:43.806 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:43.806 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:43.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:09:43.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:09:43.806 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:09:43.806 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:44.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:44.006 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:09:44.047 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:09:44.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:44.048 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list-id 2024-03-29T09:09:44.130 INFO:teuthology.orchestra.run.smithi099.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-29T09:09:44.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:44.131 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:44.131 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:44.132 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:09:44.132 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-29T09:09:44.132 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:44.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:44.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T09:09:44.369 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:44.369 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether c2:86:84:d2:8f:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::c086:84ff:fed2:8fc0/64 scope link 2024-03-29T09:09:44.370 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:44.373 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether c2:86:84:d2:8f:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T09:09:44.438 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T09:09:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::c086:84ff:fed2:8fc0/64 scope link 2024-03-29T09:09:44.439 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:44.441 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-29T09:09:44.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:44.441 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:44.441 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:44.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-29T09:09:44.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T09:09:44.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-29T09:09:44.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-29T09:09:44.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-29T09:09:44.442 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:44.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:44.677 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:09:44.677 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:09:44.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.1 up 2024-03-29T09:09:44.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T09:09:44.677 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:09:44.842 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-29T09:09:44.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:44.842 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-29T09:09:44.884 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-29T09:09:44.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:44.884 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-29T09:09:44.965 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:09:45.072 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:45.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:45.134 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:45.137 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:45.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:45.137 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:09:45.179 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:09:45.179 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-03-29T09:09:45.179 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-29T09:09:45.222 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:09:45.288 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:45.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:45.353 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:45.356 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:45.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:45.356 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:09:45.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi099.stderr:2024-03-29T09:09:45.385+0000 7f9096af95c0 -1 init, newargv = 0x55a6697db3c0 newargc=15 2024-03-29T09:09:45.385 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi099.stderr:ceph-fuse[53995]: starting ceph client 2024-03-29T09:09:45.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi099.stderr:ceph-fuse[53995]: starting fuse 2024-03-29T09:09:45.401 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:09:45.401 INFO:teuthology.orchestra.run.smithi099.stdout:46 2024-03-29T09:09:45.401 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 46] 2024-03-29T09:09:45.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:09:45.402 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:09:45.403 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.2")) 2024-03-29T09:09:45.404 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:09:45.580 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.2.53995.asok 2024-03-29T09:09:45.759 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.53995.asok 2024-03-29T09:09:45.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:45.759 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.53995.asok status 2024-03-29T09:09:45.978 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:09:45.978 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:09:45.978 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:09:45.978 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:09:45.978 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "2", 2024-03-29T09:09:45.978 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:09:45.978 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "53995", 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 4842, 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 4842 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:09:45.979 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.2:0", 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2309115166 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.2:0", 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2309115166 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.4842 192.168.0.2:0/2309115166", 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.2:0/2309115166", 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 8, 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 26, 2024-03-29T09:09:45.980 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:09:45.981 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:09:45.981 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:09:45.981 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:09:45.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:09:46.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.184+0000 7f4d75400700 1 Processor -- start 2024-03-29T09:09:46.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.184+0000 7f4d75400700 1 -- start start 2024-03-29T09:09:46.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d75400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d700d1b20 0x7f4d700d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:46.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d75400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:46.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d75400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d700cddc0 con 0x7f4d701608e0 2024-03-29T09:09:46.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d75400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d700ce0a0 con 0x7f4d700d1b20 2024-03-29T09:09:46.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d75400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700ce380 con 0x7f4d70160cc0 2024-03-29T09:09:46.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d6f200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:46.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d6de00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4d700d1b20 0x7f4d700d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.185+0000 7f4d6e800700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4d701608e0 0x7f4d700d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:40506/0 (socket says 172.21.15.99:40506) 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6e800700 1 -- 172.21.15.99:0/1081790245 learned_addr learned my addr 172.21.15.99:0/1081790245 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6de00700 1 -- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 msgr2=0x7f4d700ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6de00700 1 --2- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6de00700 1 -- 172.21.15.99:0/1081790245 >> v1:172.21.15.133:6789/0 conn(0x7f4d701608e0 legacy=0x7f4d700d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6de00700 1 -- 172.21.15.99:0/1081790245 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d700ce660 con 0x7f4d700d1b20 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6f200700 1 --2- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6d400700 1 -- 172.21.15.99:0/1081790245 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054815666 0 0) 0x7f4d700cddc0 con 0x7f4d701608e0 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.186+0000 7f4d6de00700 1 --2- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d700d1b20 0x7f4d700d0150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009840 tx=0x7f4d5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f128b5eb8aeaa75 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d6d400700 1 -- 172.21.15.99:0/1081790245 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c011830 con 0x7f4d700d1b20 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d6d400700 1 -- 172.21.15.99:0/1081790245 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c011990 con 0x7f4d700d1b20 2024-03-29T09:09:46.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d6d400700 1 -- 172.21.15.99:0/1081790245 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c011d20 con 0x7f4d700d1b20 2024-03-29T09:09:46.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 -- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d700d1b20 msgr2=0x7f4d700d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:46.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 --2- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d700d1b20 0x7f4d700d0150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009840 tx=0x7f4d5c009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:46.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 -- 172.21.15.99:0/1081790245 shutdown_connections 2024-03-29T09:09:46.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 --2- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 --2- 172.21.15.99:0/1081790245 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d700d1b20 0x7f4d700d0150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 -- 172.21.15.99:0/1081790245 >> 172.21.15.99:0/1081790245 conn(0x7f4d7009c490 msgr2=0x7f4d700d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:46.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 -- 172.21.15.99:0/1081790245 shutdown_connections 2024-03-29T09:09:46.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.187+0000 7f4d75400700 1 -- 172.21.15.99:0/1081790245 wait complete. 2024-03-29T09:09:46.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.190+0000 7f4d75400700 1 Processor -- start 2024-03-29T09:09:46.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.190+0000 7f4d75400700 1 -- start start 2024-03-29T09:09:46.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d75400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d700d1b20 0x7f4d70156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:46.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d75400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d701608e0 0x7f4d700b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:46.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d75400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d75400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d70157820 con 0x7f4d700d1b20 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d75400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700b41b0 con 0x7f4d701608e0 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d75400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d700cb310 con 0x7f4d70160cc0 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d6e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d700d1b20 0x7f4d70156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d6e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d700d1b20 0x7f4d70156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:51926/0 (socket says 172.21.15.99:51926) 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d6e800700 1 -- 172.21.15.99:0/711744121 learned_addr learned my addr 172.21.15.99:0/711744121 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d6de00700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d701608e0 0x7f4d700b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.191+0000 7f4d6f200700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d6e800700 1 -- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 msgr2=0x7f4d700b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d6e800700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700b3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d6e800700 1 -- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d701608e0 msgr2=0x7f4d700b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d6e800700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d701608e0 0x7f4d700b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d6e800700 1 -- 172.21.15.99:0/711744121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68009a70 con 0x7f4d700d1b20 2024-03-29T09:09:46.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d6e800700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d700d1b20 0x7f4d70156460 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d600011e0 tx=0x7f4d6000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:46.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d66a00700 1 -- 172.21.15.99:0/711744121 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004c80 con 0x7f4d700d1b20 2024-03-29T09:09:46.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d66a00700 1 -- 172.21.15.99:0/711744121 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d60004e00 con 0x7f4d700d1b20 2024-03-29T09:09:46.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c0094a0 con 0x7f4d700d1b20 2024-03-29T09:09:46.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d66a00700 1 -- 172.21.15.99:0/711744121 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60007310 con 0x7f4d700d1b20 2024-03-29T09:09:46.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.192+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d700cb8d0 con 0x7f4d700d1b20 2024-03-29T09:09:46.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.193+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d700677a0 con 0x7f4d700d1b20 2024-03-29T09:09:46.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.194+0000 7f4d66a00700 1 -- 172.21.15.99:0/711744121 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4d60007470 con 0x7f4d700d1b20 2024-03-29T09:09:46.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.195+0000 7f4d66a00700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4d5006ac90 0x7f4d5006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:46.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.195+0000 7f4d66a00700 1 -- 172.21.15.99:0/711744121 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4d600892c0 con 0x7f4d700d1b20 2024-03-29T09:09:46.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.195+0000 7f4d6de00700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4d5006ac90 0x7f4d5006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:46.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.196+0000 7f4d6de00700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4d5006ac90 0x7f4d5006d150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c000c00 tx=0x7f4d5c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:46.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.199+0000 7f4d66a00700 1 -- 172.21.15.99:0/711744121 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4d60054140 con 0x7f4d700d1b20 2024-03-29T09:09:46.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.452+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d700d0700 con 0x7f4d700d1b20 2024-03-29T09:09:46.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.452+0000 7f4d66a00700 1 -- 172.21.15.99:0/711744121 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f4d60057760 con 0x7f4d700d1b20 2024-03-29T09:09:46.452 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T09:09:46.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4d5006ac90 msgr2=0x7f4d5006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:46.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4d5006ac90 0x7f4d5006d150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c000c00 tx=0x7f4d5c008030 comp rx=0 tx=0).stop 2024-03-29T09:09:46.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d700d1b20 msgr2=0x7f4d70156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d700d1b20 0x7f4d70156460 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4d600011e0 tx=0x7f4d6000ca30 comp rx=0 tx=0).stop 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 shutdown_connections 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4d5006ac90 0x7f4d5006d150 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d70160cc0 0x7f4d700b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d700d1b20 0x7f4d70156460 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 --2- 172.21.15.99:0/711744121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d701608e0 0x7f4d700b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 >> 172.21.15.99:0/711744121 conn(0x7f4d7009c490 msgr2=0x7f4d7015e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.454+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 shutdown_connections 2024-03-29T09:09:46.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:46.455+0000 7f4d75400700 1 -- 172.21.15.99:0/711744121 wait complete. 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:09:46.473 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T09:09:46.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:46.474 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:46.522 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-03-29T09:09:46.523 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:46.523 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T09:09:46.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:46.524 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:46.524 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:46.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-03-29T09:09:46.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-03-29T09:09:46.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-03-29T09:09:46.524 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T09:09:46.524 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:46.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:46.939 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:09:47.003 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-03-29T09:09:47.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:47.006 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-03-29T09:09:47.066 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-03-29T09:09:47.066 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:09:47.066 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:09:47.066 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:09:47.066 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:09:47.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:47.067 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:47.067 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:47.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:09:47.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:09:47.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:09:47.068 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:47.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:47.718 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-29T09:09:47.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:47.760 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-03-29T09:09:47.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:47.842 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:47.842 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:47.842 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T09:09:47.843 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T09:09:47.843 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:48.020 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T09:09:48.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:48.020 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:48.020 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:48.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T09:09:48.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T09:09:48.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T09:09:48.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T09:09:48.021 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T09:09:48.021 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:48.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:48.309 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:48.309 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:48.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-03-29T09:09:48.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T09:09:48.309 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:48.474 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T09:09:48.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:48.474 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T09:09:48.515 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T09:09:48.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:48.516 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T09:09:48.596 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-03-29T09:09:48.701 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:48.701 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:48.768 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:48.771 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:48.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:48.772 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-03-29T09:09:48.813 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:09:48.813 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T09:09:48.857 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-03-29T09:09:48.924 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:48.924 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:48.987 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:48.990 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:48.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:48.991 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-03-29T09:09:49.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi133.stderr:2024-03-29T09:09:49.023+0000 7ff96bb6d5c0 -1 init, newargv = 0x55f6df348010 newargc=15 2024-03-29T09:09:49.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi133.stderr:ceph-fuse[48761]: starting ceph client 2024-03-29T09:09:49.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi133.stderr:ceph-fuse[48761]: starting fuse 2024-03-29T09:09:49.034 INFO:teuthology.orchestra.run.smithi133.stdout:44 2024-03-29T09:09:49.034 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:09:49.034 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-03-29T09:09:49.035 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.1")) 2024-03-29T09:09:49.036 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-03-29T09:09:49.212 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.1.48761.asok 2024-03-29T09:09:49.391 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.48761.asok 2024-03-29T09:09:49.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:49.391 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.48761.asok status 2024-03-29T09:09:49.610 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "1", 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "48761", 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-03-29T09:09:49.611 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4822, 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4822 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1295481659 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-03-29T09:09:49.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1295481659 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4822 192.168.0.1:0/1295481659", 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/1295481659", 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 8, 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 26, 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-03-29T09:09:49.613 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-29T09:09:49.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:09:49.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.823+0000 7f869f800700 1 Processor -- start 2024-03-29T09:09:49.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.823+0000 7f869f800700 1 -- start start 2024-03-29T09:09:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.824+0000 7f869f800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 0x7f86980d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.824+0000 7f869f800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8698160cc0 0x7f86980ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.824+0000 7f869f800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86980cddc0 con 0x7f86981608e0 2024-03-29T09:09:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.824+0000 7f869f800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86980ce0a0 con 0x7f8698160cc0 2024-03-29T09:09:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.824+0000 7f869f800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86980ce380 con 0x7f86980d1b20 2024-03-29T09:09:49.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.824+0000 7f869ce00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f86981608e0 0x7f86980d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56058/0 (socket says 172.21.15.99:56058) 2024-03-29T09:09:49.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.824+0000 7f869ce00700 1 -- 172.21.15.99:0/134259754 learned_addr learned my addr 172.21.15.99:0/134259754 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:49.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321107027 0 0) 0x7f86980cddc0 con 0x7f86981608e0 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f869d800700 1 --2- 172.21.15.99:0/134259754 >> v2:172.21.15.133:3300/0 conn(0x7f8698160cc0 0x7f86980ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697e00700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 0x7f86980d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86800035b0 con 0x7f86981608e0 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2674358843 0 0) 0x7f86800035b0 con 0x7f86981608e0 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 msgr2=0x7f86980d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 0x7f86980d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 >> v2:172.21.15.133:3300/0 conn(0x7f8698160cc0 msgr2=0x7f86980ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 --2- 172.21.15.99:0/134259754 >> v2:172.21.15.133:3300/0 conn(0x7f8698160cc0 0x7f86980ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86980ce660 con 0x7f86981608e0 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f868c001670 con 0x7f86981608e0 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 >> v1:172.21.15.99:6789/0 conn(0x7f86981608e0 legacy=0x7f86980d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:49.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680003a00 0x7f8680003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 0x7f8680004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f8680005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980ce0a0 con 0x7f8680004d90 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980ce380 con 0x7f8680003a00 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868000ba10 con 0x7f86980d1b20 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f869d800700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f8680005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f869ce00700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680003a00 0x7f8680003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.825+0000 7f8697e00700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 0x7f8680004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f869d800700 1 -- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 msgr2=0x7f8680004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f869d800700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86980d1b20 0x7f8680004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f869d800700 1 -- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680003a00 msgr2=0x7f8680003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:49.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f869d800700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680003a00 0x7f8680003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f869d800700 1 -- 172.21.15.99:0/134259754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800035b0 con 0x7f8680004d90 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f869ce00700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8680003a00 0x7f8680003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f869d800700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f8680005210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8690009fc0 tx=0x7f8680005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399398e6abe92622 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869000ec30 con 0x7f8680004d90 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869000f040 con 0x7f8680004d90 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.826+0000 7f8697400700 1 -- 172.21.15.99:0/134259754 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690014480 con 0x7f8680004d90 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.827+0000 7f869f800700 1 -- 172.21.15.99:0/134259754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 msgr2=0x7f8680005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.827+0000 7f869f800700 1 --2- 172.21.15.99:0/134259754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f8680005210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8690009fc0 tx=0x7f8680005880 comp rx=0 tx=0).stop 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.827+0000 7f869d800700 1 -- 172.21.15.99:0/134259754 reap_dead start 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.827+0000 7f869f800700 1 -- 172.21.15.99:0/134259754 shutdown_connections 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.827+0000 7f869f800700 1 -- 172.21.15.99:0/134259754 >> 172.21.15.99:0/134259754 conn(0x7f869809c490 msgr2=0x7f86980d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:49.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.827+0000 7f869f800700 1 -- 172.21.15.99:0/134259754 shutdown_connections 2024-03-29T09:09:49.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.827+0000 7f869f800700 1 -- 172.21.15.99:0/134259754 wait complete. 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.829+0000 7f869f800700 1 Processor -- start 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.829+0000 7f869f800700 1 -- start start 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869f800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f86981586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869f800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86981559e0 0x7f86980ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869f800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86981f20a0 0x7f86981f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869f800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86981573d0 con 0x7f8680004d90 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869f800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86980a0e70 con 0x7f86981f20a0 2024-03-29T09:09:49.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869f800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8698157620 con 0x7f86981559e0 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f86981586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f86981586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58506/0 (socket says 172.21.15.99:58506) 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 -- 172.21.15.99:0/264017399 learned_addr learned my addr 172.21.15.99:0/264017399 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869d800700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86981f20a0 0x7f86981f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f8697e00700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86981559e0 0x7f86980ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 -- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86981559e0 msgr2=0x7f86980ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86981559e0 0x7f86980ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 -- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86981f20a0 msgr2=0x7f86981f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86981f20a0 0x7f86981f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.830+0000 7f869ce00700 1 -- 172.21.15.99:0/264017399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8698157900 con 0x7f8680004d90 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.831+0000 7f869ce00700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f86981586c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f868c007380 tx=0x7f868c007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.831+0000 7f8694c00700 1 -- 172.21.15.99:0/264017399 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00f8d0 con 0x7f8680004d90 2024-03-29T09:09:49.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.831+0000 7f8694c00700 1 -- 172.21.15.99:0/264017399 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00fa30 con 0x7f8680004d90 2024-03-29T09:09:49.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.831+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8698157be0 con 0x7f8680004d90 2024-03-29T09:09:49.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.831+0000 7f8694c00700 1 -- 172.21.15.99:0/264017399 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c00fda0 con 0x7f8680004d90 2024-03-29T09:09:49.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.831+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86981581a0 con 0x7f8680004d90 2024-03-29T09:09:49.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.832+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86980677a0 con 0x7f8680004d90 2024-03-29T09:09:49.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.833+0000 7f8694c00700 1 -- 172.21.15.99:0/264017399 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f868c01d460 con 0x7f8680004d90 2024-03-29T09:09:49.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.834+0000 7f8694c00700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f867806ac90 0x7f867806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:49.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.834+0000 7f8694c00700 1 -- 172.21.15.99:0/264017399 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f868c0899f0 con 0x7f8680004d90 2024-03-29T09:09:49.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.834+0000 7f8697e00700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f867806ac90 0x7f867806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:49.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.834+0000 7f8697e00700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f867806ac90 0x7f867806d150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8698155e20 tx=0x7f8684008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:49.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:49.838+0000 7f8694c00700 1 -- 172.21.15.99:0/264017399 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f868c0547e0 con 0x7f8680004d90 2024-03-29T09:09:50.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.086+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f86980d0700 con 0x7f8680004d90 2024-03-29T09:09:50.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.087+0000 7f8694c00700 1 -- 172.21.15.99:0/264017399 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f868c057e00 con 0x7f8680004d90 2024-03-29T09:09:50.087 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-29T09:09:50.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f867806ac90 msgr2=0x7f867806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f867806ac90 0x7f867806d150 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8698155e20 tx=0x7f8684008040 comp rx=0 tx=0).stop 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 msgr2=0x7f86981586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f86981586c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f868c007380 tx=0x7f868c007740 comp rx=0 tx=0).stop 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 shutdown_connections 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f867806ac90 0x7f867806d150 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86981f20a0 0x7f86981f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8680004d90 0x7f86981586c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 --2- 172.21.15.99:0/264017399 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86981559e0 0x7f86980ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 >> 172.21.15.99:0/264017399 conn(0x7f869809c490 msgr2=0x7f86980b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 shutdown_connections 2024-03-29T09:09:50.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:50.089+0000 7f869f800700 1 -- 172.21.15.99:0/264017399 wait complete. 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:09:50.109 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-29T09:09:50.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.110 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-03-29T09:09:50.154 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:50.154 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:50.154 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:50.154 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.154 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42169sec preferred_lft 42169sec 2024-03-29T09:09:50.155 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether da:2c:79:b4:78:65 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::1453:c6ff:fead:8df0/64 scope link 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether da:2c:79:b4:78:65 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::d82c:79ff:feb4:7865/64 scope link tentative 2024-03-29T09:09:50.156 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.157 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:09:50.260 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-03-29T09:09:50.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.264 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-03-29T09:09:50.307 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-03-29T09:09:50.307 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:09:50.307 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:09:50.308 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:09:50.308 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:09:50.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.309 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:50.309 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:50.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:09:50.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:09:50.309 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:09:50.309 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:50.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.510 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-29T09:09:50.550 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T09:09:50.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.551 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-03-29T09:09:50.632 INFO:teuthology.orchestra.run.smithi133.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-29T09:09:50.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.633 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:50.633 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:50.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T09:09:50.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-29T09:09:50.634 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:50.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.807 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T09:09:50.872 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:50.872 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:0f:8f:0d:9e:6f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c0f:8fff:fe0d:9e6f/64 scope link 2024-03-29T09:09:50.873 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.876 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T09:09:50.941 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:09:50.941 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:09:50.941 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:09:50.941 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.941 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-29T09:09:50.942 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.942 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:09:50.942 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0e:0f:8f:0d:9e:6f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T09:09:50.942 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T09:09:50.942 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.942 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::c0f:8fff:fe0d:9e6f/64 scope link 2024-03-29T09:09:50.942 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:09:50.944 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-29T09:09:50.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-29T09:09:50.945 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:51.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:51.180 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-29T09:09:51.180 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-29T09:09:51.180 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.1 up 2024-03-29T09:09:51.180 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T09:09:51.180 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-29T09:09:51.345 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-29T09:09:51.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:51.345 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-29T09:09:51.388 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-29T09:09:51.388 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:09:51.388 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-29T09:09:51.469 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-03-29T09:09:51.575 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:51.575 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:51.637 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:51.640 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:51.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:51.640 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-03-29T09:09:51.681 INFO:teuthology.orchestra.run.smithi133.stdout:44 2024-03-29T09:09:51.681 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [44] 2024-03-29T09:09:51.682 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-29T09:09:51.725 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2024-03-29T09:09:51.790 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:09:51.790 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:09:51.854 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:09:51.857 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:09:51.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:51.858 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2024-03-29T09:09:51.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi133.stderr:2024-03-29T09:09:51.889+0000 7f20db9b75c0 -1 init, newargv = 0x563381d04550 newargc=15 2024-03-29T09:09:51.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi133.stderr:ceph-fuse[49378]: starting ceph client 2024-03-29T09:09:51.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi133.stderr:ceph-fuse[49378]: starting fuse 2024-03-29T09:09:51.904 INFO:teuthology.orchestra.run.smithi133.stdout:44 2024-03-29T09:09:51.904 INFO:teuthology.orchestra.run.smithi133.stdout:46 2024-03-29T09:09:51.904 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44, 46] 2024-03-29T09:09:51.905 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:09:51.905 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2024-03-29T09:09:51.905 DEBUG:teuthology.orchestra.run.smithi133:> import re 2024-03-29T09:09:51.905 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-03-29T09:09:51.905 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2024-03-29T09:09:51.905 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:51.906 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2024-03-29T09:09:51.906 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-29T09:09:51.906 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2024-03-29T09:09:51.906 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-29T09:09:51.906 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:51.906 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2024-03-29T09:09:51.907 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2024-03-29T09:09:51.907 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2024-03-29T09:09:51.907 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2024-03-29T09:09:51.907 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:51.907 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2024-03-29T09:09:51.907 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2024-03-29T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2024-03-29T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi133:> return f 2024-03-29T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-29T09:09:51.909 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.3")) 2024-03-29T09:09:51.909 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-03-29T09:09:52.084 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.3.49378.asok 2024-03-29T09:09:52.263 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.49378.asok 2024-03-29T09:09:52.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.49378.asok status 2024-03-29T09:09:52.486 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-29T09:09:52.486 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2024-03-29T09:09:52.486 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:09:52.486 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:09:52.486 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "3", 2024-03-29T09:09:52.486 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "49378", 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4834, 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 4834 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:52.487 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.2:0", 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1196928255 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.2:0", 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 1196928255 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.4834 192.168.0.2:0/1196928255", 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.2:0/1196928255", 2024-03-29T09:09:52.488 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2024-03-29T09:09:52.489 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 8, 2024-03-29T09:09:52.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 26, 2024-03-29T09:09:52.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:09:52.489 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2024-03-29T09:09:52.489 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2024-03-29T09:09:52.489 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-29T09:09:52.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.500 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:09:52.500 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T09:09:52.549 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:09:52.549 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T09:09:52.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:09:52.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.619 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:09:52.619 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-29T09:09:52.630 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:09:52.663 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:09:52.663 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-29T09:09:52.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.663 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-29T09:09:52.666 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:09:52.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.731 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-03-29T09:09:52.731 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T09:09:52.773 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-03-29T09:09:52.773 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T09:09:52.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.773 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T09:09:52.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.879 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2024-03-29T09:09:52.879 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-29T09:09:52.923 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2024-03-29T09:09:52.923 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-29T09:09:52.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:09:52.923 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-29T09:09:53.028 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-29T09:09:53.085 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:53.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.276+0000 7f58c5a00700 1 Processor -- start 2024-03-29T09:09:53.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.276+0000 7f58c5a00700 1 -- start start 2024-03-29T09:09:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.277+0000 7f58c5a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f58c0070540 0x7f58c015d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.277+0000 7f58c5a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c00ccf60 con 0x7f58c0153d20 2024-03-29T09:09:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.277+0000 7f58c5a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c0071750 con 0x7f58c00709d0 2024-03-29T09:09:53.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.277+0000 7f58c5a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c00b12d0 con 0x7f58c0070540 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.277+0000 7f58bea00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f58c0153d20 0x7f58c00b0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56064/0 (socket says 172.21.15.99:56064) 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.277+0000 7f58bea00700 1 -- 172.21.15.99:0/1173376001 learned_addr learned my addr 172.21.15.99:0/1173376001 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58be000700 1 --2- 172.21.15.99:0/1173376001 >> v2:172.21.15.133:3300/0 conn(0x7f58c0070540 0x7f58c015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58bd600700 1 -- 172.21.15.99:0/1173376001 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3550303480 0 0) 0x7f58c00ccf60 con 0x7f58c0153d20 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58bd600700 1 -- 172.21.15.99:0/1173376001 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a40035b0 con 0x7f58c0153d20 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58bd600700 1 -- 172.21.15.99:0/1173376001 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772389817 0 0) 0x7f58c0071750 con 0x7f58c00709d0 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58bd600700 1 -- 172.21.15.99:0/1173376001 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c00ccf60 con 0x7f58c00709d0 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58be000700 1 -- 172.21.15.99:0/1173376001 >> v1:172.21.15.133:6789/0 conn(0x7f58c00709d0 legacy=0x7f58c01646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58be000700 1 -- 172.21.15.99:0/1173376001 >> v1:172.21.15.99:6789/0 conn(0x7f58c0153d20 legacy=0x7f58c00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58be000700 1 -- 172.21.15.99:0/1173376001 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c0070eb0 con 0x7f58c0070540 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58be000700 1 --2- 172.21.15.99:0/1173376001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c015d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009840 tx=0x7f58ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a99d5b770d4d1092 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:53.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.278+0000 7f58bd600700 1 -- 172.21.15.99:0/1173376001 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011830 con 0x7f58c0070540 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58bd600700 1 -- 172.21.15.99:0/1173376001 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac011990 con 0x7f58c0070540 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58bd600700 1 -- 172.21.15.99:0/1173376001 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac011d20 con 0x7f58c0070540 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58c5a00700 1 -- 172.21.15.99:0/1173376001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 msgr2=0x7f58c015d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58c5a00700 1 --2- 172.21.15.99:0/1173376001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c015d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f58ac009840 tx=0x7f58ac009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58c5a00700 1 -- 172.21.15.99:0/1173376001 shutdown_connections 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58c5a00700 1 --2- 172.21.15.99:0/1173376001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c015d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58c5a00700 1 -- 172.21.15.99:0/1173376001 >> 172.21.15.99:0/1173376001 conn(0x7f58c0066540 msgr2=0x7f58c0066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58c5a00700 1 -- 172.21.15.99:0/1173376001 shutdown_connections 2024-03-29T09:09:53.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.279+0000 7f58c5a00700 1 -- 172.21.15.99:0/1173376001 wait complete. 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.281+0000 7f58c5a00700 1 Processor -- start 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.281+0000 7f58c5a00700 1 -- start start 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58c5a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58c5a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00709d0 0x7f58c0159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58c5a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58c0153d20 0x7f58c015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58c5a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01f5430 con 0x7f58c00709d0 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58c5a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01552f0 con 0x7f58c0070540 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58c5a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01f47b0 con 0x7f58c0153d20 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58be000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00709d0 0x7f58c0159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58bea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58be000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00709d0 0x7f58c0159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58518/0 (socket says 172.21.15.99:58518) 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58bea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c0154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:42980/0 (socket says 172.21.15.99:42980) 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58be000700 1 -- 172.21.15.99:0/3298361613 learned_addr learned my addr 172.21.15.99:0/3298361613 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.282+0000 7f58bf400700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58c0153d20 0x7f58c015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58bea00700 1 -- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58c0153d20 msgr2=0x7f58c015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58bea00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58c0153d20 0x7f58c015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58bea00700 1 -- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00709d0 msgr2=0x7f58c0159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58bea00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00709d0 0x7f58c0159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58bea00700 1 -- 172.21.15.99:0/3298361613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0094a0 con 0x7f58c0070540 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58be000700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00709d0 0x7f58c0159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58bea00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c0154db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58b00011e0 tx=0x7f58b000c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58b6a00700 1 -- 172.21.15.99:0/3298361613 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0005b40 con 0x7f58c0070540 2024-03-29T09:09:53.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58b6a00700 1 -- 172.21.15.99:0/3298361613 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b0005ca0 con 0x7f58c0070540 2024-03-29T09:09:53.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c01f4a90 con 0x7f58c0070540 2024-03-29T09:09:53.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58b6a00700 1 -- 172.21.15.99:0/3298361613 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0007310 con 0x7f58c0070540 2024-03-29T09:09:53.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.283+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c01f5050 con 0x7f58c0070540 2024-03-29T09:09:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.285+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c00cb640 con 0x7f58c0070540 2024-03-29T09:09:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.285+0000 7f58b6a00700 1 -- 172.21.15.99:0/3298361613 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f58b0007680 con 0x7f58c0070540 2024-03-29T09:09:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.286+0000 7f58b6a00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f58a006ac90 0x7f58a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.286+0000 7f58b6a00700 1 -- 172.21.15.99:0/3298361613 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f58b0088c40 con 0x7f58c0070540 2024-03-29T09:09:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.286+0000 7f58be000700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f58a006ac90 0x7f58a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.287+0000 7f58be000700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f58a006ac90 0x7f58a006d150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58c00cd620 tx=0x7f58ac01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:53.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.291+0000 7f58b6a00700 1 -- 172.21.15.99:0/3298361613 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f58c00cb640 con 0x7f58c0070540 2024-03-29T09:09:53.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.501+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58c00cb640 con 0x7f58c0070540 2024-03-29T09:09:53.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.503+0000 7f58b6a00700 1 -- 172.21.15.99:0/3298361613 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f58c00cb640 con 0x7f58c0070540 2024-03-29T09:09:53.504 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:53.504 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f58a006ac90 msgr2=0x7f58a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f58a006ac90 0x7f58a006d150 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f58c00cd620 tx=0x7f58ac01c040 comp rx=0 tx=0).stop 2024-03-29T09:09:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 msgr2=0x7f58c0154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c0154db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f58b00011e0 tx=0x7f58b000c6e0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 shutdown_connections 2024-03-29T09:09:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f58a006ac90 0x7f58a006d150 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58c0153d20 0x7f58c015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58c0070540 0x7f58c0154db0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 --2- 172.21.15.99:0/3298361613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f58c00709d0 0x7f58c0159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 >> 172.21.15.99:0/3298361613 conn(0x7f58c0066540 msgr2=0x7f58c00b01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 shutdown_connections 2024-03-29T09:09:53.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.506+0000 7f58c5a00700 1 -- 172.21.15.99:0/3298361613 wait complete. 2024-03-29T09:09:53.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:09:53.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:09:53.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.716+0000 7f66dec00700 1 Processor -- start 2024-03-29T09:09:53.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.717+0000 7f66dec00700 1 -- start start 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66dec00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66d80b3cc0 0x7f66d80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66dec00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80d3810 con 0x7f66d80d3380 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66dec00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66d80d1520 con 0x7f66d8160770 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66dec00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66d80b4140 con 0x7f66d80b3cc0 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66d7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66d80b3cc0 0x7f66d80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66d7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66d80b3cc0 0x7f66d80d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58534/0 (socket says 172.21.15.99:58534) 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66d7400700 1 -- 172.21.15.99:0/2767358884 learned_addr learned my addr 172.21.15.99:0/2767358884 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:53.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66dcc00700 1 --1- 172.21.15.99:0/2767358884 >> v1:172.21.15.99:6789/0 conn(0x7f66d80d3380 0x7f66d80ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56070/0 (socket says 172.21.15.99:56070) 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.719+0000 7f66d7400700 1 -- 172.21.15.99:0/2767358884 >> v1:172.21.15.133:6789/0 conn(0x7f66d8160770 legacy=0x7f66d80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.719+0000 7f66d7400700 1 -- 172.21.15.99:0/2767358884 >> v1:172.21.15.99:6789/0 conn(0x7f66d80d3380 legacy=0x7f66d80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.718+0000 7f66d6a00700 1 -- 172.21.15.99:0/2767358884 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603795156 0 0) 0x7f66d80d1520 con 0x7f66d8160770 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.719+0000 7f66d7400700 1 -- 172.21.15.99:0/2767358884 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80d1800 con 0x7f66d80b3cc0 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.719+0000 7f66d7400700 1 --2- 172.21.15.99:0/2767358884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d80b3cc0 0x7f66d80d2e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098f0 tx=0x7f66c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec47c19d04d7e51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.719+0000 7f66d6a00700 1 -- 172.21.15.99:0/2767358884 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4004230 con 0x7f66d80b3cc0 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.719+0000 7f66d6a00700 1 -- 172.21.15.99:0/2767358884 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c4004390 con 0x7f66d80b3cc0 2024-03-29T09:09:53.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.719+0000 7f66d6a00700 1 -- 172.21.15.99:0/2767358884 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c40036a0 con 0x7f66d80b3cc0 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.720+0000 7f66dec00700 1 -- 172.21.15.99:0/2767358884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d80b3cc0 msgr2=0x7f66d80d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.720+0000 7f66dec00700 1 --2- 172.21.15.99:0/2767358884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d80b3cc0 0x7f66d80d2e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f66c40098f0 tx=0x7f66c4009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.720+0000 7f66dec00700 1 -- 172.21.15.99:0/2767358884 shutdown_connections 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.720+0000 7f66dec00700 1 --2- 172.21.15.99:0/2767358884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d80b3cc0 0x7f66d80d2e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.720+0000 7f66dec00700 1 -- 172.21.15.99:0/2767358884 >> 172.21.15.99:0/2767358884 conn(0x7f66d808afd0 msgr2=0x7f66d809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.720+0000 7f66dec00700 1 -- 172.21.15.99:0/2767358884 shutdown_connections 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.720+0000 7f66dec00700 1 -- 172.21.15.99:0/2767358884 wait complete. 2024-03-29T09:09:53.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.721+0000 7f66dec00700 1 Processor -- start 2024-03-29T09:09:53.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.722+0000 7f66dec00700 1 -- start start 2024-03-29T09:09:53.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.722+0000 7f66dec00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66d80b3cc0 0x7f66d81f2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.722+0000 7f66dec00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66d80d3380 0x7f66d80b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.722+0000 7f66dec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d8160770 0x7f66d80b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.722+0000 7f66dec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80b6da0 con 0x7f66d8160770 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.722+0000 7f66dec00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8156570 con 0x7f66d80b3cc0 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.722+0000 7f66dec00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66d80b6190 con 0x7f66d80d3380 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d8160770 0x7f66d80b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d8160770 0x7f66d80b7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58542/0 (socket says 172.21.15.99:58542) 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 -- 172.21.15.99:0/2498590396 learned_addr learned my addr 172.21.15.99:0/2498590396 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66d7e00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66d80b3cc0 0x7f66d81f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66d7400700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66d80d3380 0x7f66d80b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 -- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66d80d3380 msgr2=0x7f66d80b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66d80d3380 0x7f66d80b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 -- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66d80b3cc0 msgr2=0x7f66d81f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66d80b3cc0 0x7f66d81f2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 -- 172.21.15.99:0/2498590396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4009550 con 0x7f66d8160770 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.723+0000 7f66dcc00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d8160770 0x7f66d80b7d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f66d0002000 tx=0x7f66d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.724+0000 7f66cfe00700 1 -- 172.21.15.99:0/2498590396 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d000ac90 con 0x7f66d8160770 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.724+0000 7f66cfe00700 1 -- 172.21.15.99:0/2498590396 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d000adf0 con 0x7f66d8160770 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.724+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d80b6470 con 0x7f66d8160770 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.724+0000 7f66cfe00700 1 -- 172.21.15.99:0/2498590396 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0007310 con 0x7f66d8160770 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.724+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d80b6a30 con 0x7f66d8160770 2024-03-29T09:09:53.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.725+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d80b6f10 con 0x7f66d8160770 2024-03-29T09:09:53.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.726+0000 7f66cfe00700 1 -- 172.21.15.99:0/2498590396 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f66d0007470 con 0x7f66d8160770 2024-03-29T09:09:53.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.727+0000 7f66cfe00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f66b806acc0 0x7f66b806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:53.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.727+0000 7f66cfe00700 1 -- 172.21.15.99:0/2498590396 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f66d0089500 con 0x7f66d8160770 2024-03-29T09:09:53.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.727+0000 7f66d7e00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f66b806acc0 0x7f66b806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:53.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.727+0000 7f66d7e00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f66b806acc0 0x7f66b806d180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000f60 tx=0x7f66c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:53.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.731+0000 7f66cfe00700 1 -- 172.21.15.99:0/2498590396 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f66d00542f0 con 0x7f66d8160770 2024-03-29T09:09:53.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.933+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66d80678b0 con 0x7f66d8160770 2024-03-29T09:09:53.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.934+0000 7f66cfe00700 1 -- 172.21.15.99:0/2498590396 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66d0057910 con 0x7f66d8160770 2024-03-29T09:09:53.934 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:09:53.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f66b806acc0 msgr2=0x7f66b806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f66b806acc0 0x7f66b806d180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f66c8000f60 tx=0x7f66c8009040 comp rx=0 tx=0).stop 2024-03-29T09:09:53.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d8160770 msgr2=0x7f66d80b7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d8160770 0x7f66d80b7d40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f66d0002000 tx=0x7f66d000a340 comp rx=0 tx=0).stop 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 shutdown_connections 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f66b806acc0 0x7f66b806d180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66d8160770 0x7f66d80b7d40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66d80b3cc0 0x7f66d81f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 --2- 172.21.15.99:0/2498590396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66d80d3380 0x7f66d80b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 >> 172.21.15.99:0/2498590396 conn(0x7f66d808afd0 msgr2=0x7f66d806bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 shutdown_connections 2024-03-29T09:09:53.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:53.936+0000 7f66dec00700 1 -- 172.21.15.99:0/2498590396 wait complete. 2024-03-29T09:09:53.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:09:54.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.148+0000 7f867e000700 1 Processor -- start 2024-03-29T09:09:54.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.148+0000 7f867e000700 1 -- start start 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f867e000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86780b3cc0 0x7f86780d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f867e000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86780d3380 0x7f86780ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f867e000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780d3810 con 0x7f8678160770 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f867e000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780b4140 con 0x7f86780d3380 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f867e000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780d1520 con 0x7f86780b3cc0 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86780d3380 0x7f86780ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f86780d3380 0x7f86780ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58546/0 (socket says 172.21.15.99:58546) 2024-03-29T09:09:54.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677e00700 1 -- 172.21.15.99:0/3023791122 learned_addr learned my addr 172.21.15.99:0/3023791122 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:54.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677400700 1 --1- 172.21.15.99:0/3023791122 >> v1:172.21.15.133:6789/0 conn(0x7f8678160770 0x7f86780d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:45220/0 (socket says 172.21.15.99:45220) 2024-03-29T09:09:54.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f866fe00700 1 --2- 172.21.15.99:0/3023791122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86780b3cc0 0x7f86780d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677e00700 1 -- 172.21.15.99:0/3023791122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86780b3cc0 msgr2=0x7f86780d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:54.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677e00700 1 --2- 172.21.15.99:0/3023791122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86780b3cc0 0x7f86780d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677e00700 1 -- 172.21.15.99:0/3023791122 >> v1:172.21.15.133:6789/0 conn(0x7f8678160770 legacy=0x7f86780d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.150+0000 7f8677e00700 1 -- 172.21.15.99:0/3023791122 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780d1800 con 0x7f86780d3380 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.151+0000 7f8677e00700 1 --2- 172.21.15.99:0/3023791122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780d3380 0x7f86780ce080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8670009e50 tx=0x7f867000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428ef2db63998f2c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.151+0000 7f8676a00700 1 -- 172.21.15.99:0/3023791122 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867000e070 con 0x7f86780d3380 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.151+0000 7f8676a00700 1 -- 172.21.15.99:0/3023791122 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867000b910 con 0x7f86780d3380 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.151+0000 7f8676a00700 1 -- 172.21.15.99:0/3023791122 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867000bcc0 con 0x7f86780d3380 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.151+0000 7f867e000700 1 -- 172.21.15.99:0/3023791122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780d3380 msgr2=0x7f86780ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.151+0000 7f867e000700 1 --2- 172.21.15.99:0/3023791122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780d3380 0x7f86780ce080 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8670009e50 tx=0x7f867000b330 comp rx=0 tx=0).stop 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.152+0000 7f867e000700 1 -- 172.21.15.99:0/3023791122 shutdown_connections 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.152+0000 7f867e000700 1 --2- 172.21.15.99:0/3023791122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780d3380 0x7f86780ce080 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.152+0000 7f867e000700 1 --2- 172.21.15.99:0/3023791122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86780b3cc0 0x7f86780d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.152+0000 7f867e000700 1 -- 172.21.15.99:0/3023791122 >> 172.21.15.99:0/3023791122 conn(0x7f867808afd0 msgr2=0x7f867809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:54.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.152+0000 7f867e000700 1 -- 172.21.15.99:0/3023791122 shutdown_connections 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.152+0000 7f867e000700 1 -- 172.21.15.99:0/3023791122 wait complete. 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.154+0000 7f867e000700 1 Processor -- start 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.154+0000 7f867e000700 1 -- start start 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.155+0000 7f867e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780b3cc0 0x7f86780b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.155+0000 7f867e000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86780d3380 0x7f86780ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.155+0000 7f867e000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8678160770 0x7f86780caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.155+0000 7f867e000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780b0ed0 con 0x7f86780b3cc0 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.155+0000 7f867e000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780cd180 con 0x7f86780d3380 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.155+0000 7f867e000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780b8100 con 0x7f8678160770 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780b3cc0 0x7f86780b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780b3cc0 0x7f86780b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58554/0 (socket says 172.21.15.99:58554) 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 -- 172.21.15.99:0/2028187610 learned_addr learned my addr 172.21.15.99:0/2028187610 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f866fe00700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86780d3380 0x7f86780ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677e00700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8678160770 0x7f86780caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 -- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8678160770 msgr2=0x7f86780caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8678160770 0x7f86780caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 -- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86780d3380 msgr2=0x7f86780ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86780d3380 0x7f86780ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 -- 172.21.15.99:0/2028187610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670009b30 con 0x7f86780b3cc0 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f8677400700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780b3cc0 0x7f86780b07d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f866800c530 tx=0x7f866800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f866f400700 1 -- 172.21.15.99:0/2028187610 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668005cb0 con 0x7f86780b3cc0 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.156+0000 7f866f400700 1 -- 172.21.15.99:0/2028187610 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668005e10 con 0x7f86780b3cc0 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.157+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86780b7530 con 0x7f86780b3cc0 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.157+0000 7f866f400700 1 -- 172.21.15.99:0/2028187610 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680072d0 con 0x7f86780b3cc0 2024-03-29T09:09:54.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.157+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86780b7ab0 con 0x7f86780b3cc0 2024-03-29T09:09:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.158+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86780678b0 con 0x7f86780b3cc0 2024-03-29T09:09:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.159+0000 7f866f400700 1 -- 172.21.15.99:0/2028187610 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8668007430 con 0x7f86780b3cc0 2024-03-29T09:09:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.159+0000 7f866f400700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f865406ac90 0x7f865406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.159+0000 7f866f400700 1 -- 172.21.15.99:0/2028187610 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8668088990 con 0x7f86780b3cc0 2024-03-29T09:09:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.160+0000 7f866fe00700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f865406ac90 0x7f865406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.160+0000 7f866fe00700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f865406ac90 0x7f865406d150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86780cca80 tx=0x7f8660008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:54.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.164+0000 7f866f400700 1 -- 172.21.15.99:0/2028187610 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f86680538d0 con 0x7f86780b3cc0 2024-03-29T09:09:54.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.364+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86780cef80 con 0x7f86780b3cc0 2024-03-29T09:09:54.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.365+0000 7f866f400700 1 -- 172.21.15.99:0/2028187610 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8668017020 con 0x7f86780b3cc0 2024-03-29T09:09:54.365 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f865406ac90 msgr2=0x7f865406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f865406ac90 0x7f865406d150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f86780cca80 tx=0x7f8660008040 comp rx=0 tx=0).stop 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780b3cc0 msgr2=0x7f86780b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780b3cc0 0x7f86780b07d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f866800c530 tx=0x7f866800c8f0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 shutdown_connections 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f865406ac90 0x7f865406d150 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8678160770 0x7f86780caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f86780b3cc0 0x7f86780b07d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 --2- 172.21.15.99:0/2028187610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86780d3380 0x7f86780ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 >> 172.21.15.99:0/2028187610 conn(0x7f867808afd0 msgr2=0x7f867806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 shutdown_connections 2024-03-29T09:09:54.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.367+0000 7f867e000700 1 -- 172.21.15.99:0/2028187610 wait complete. 2024-03-29T09:09:54.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:09:54.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.578+0000 7f9b8c400700 1 Processor -- start 2024-03-29T09:09:54.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.578+0000 7f9b8c400700 1 -- start start 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.579+0000 7f9b8c400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9b84160770 0x7f9b840b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.579+0000 7f9b8c400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d3550 0x7f9b840ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.579+0000 7f9b8c400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b840d1520 con 0x7f9b840d0d60 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.579+0000 7f9b8c400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b840d1800 con 0x7f9b84160770 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.579+0000 7f9b8c400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840d1ae0 con 0x7f9b840d3550 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9b84160770 0x7f9b840b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9b84160770 0x7f9b840b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58556/0 (socket says 172.21.15.99:58556) 2024-03-29T09:09:54.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 -- 172.21.15.99:0/3117217795 learned_addr learned my addr 172.21.15.99:0/3117217795 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:54.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89000700 1 --1- 172.21.15.99:0/3117217795 >> v1:172.21.15.99:6789/0 conn(0x7f9b840d0d60 0x7f9b840d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56078/0 (socket says 172.21.15.99:56078) 2024-03-29T09:09:54.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 -- 172.21.15.99:0/3117217795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d3550 msgr2=0x7f9b840ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:54.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 --2- 172.21.15.99:0/3117217795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d3550 0x7f9b840ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 -- 172.21.15.99:0/3117217795 >> v1:172.21.15.99:6789/0 conn(0x7f9b840d0d60 legacy=0x7f9b840d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 -- 172.21.15.99:0/3117217795 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b840d1dc0 con 0x7f9b84160770 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b89a00700 1 --2- 172.21.15.99:0/3117217795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840b3cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098f0 tx=0x7f9b78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c58684d6f26985 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.580+0000 7f9b83e00700 1 -- 172.21.15.99:0/3117217795 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800b8a0 con 0x7f9b84160770 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b83e00700 1 -- 172.21.15.99:0/3117217795 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7800ba00 con 0x7f9b84160770 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b83e00700 1 -- 172.21.15.99:0/3117217795 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800bd90 con 0x7f9b84160770 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 -- 172.21.15.99:0/3117217795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 msgr2=0x7f9b840b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 --2- 172.21.15.99:0/3117217795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840b3cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098f0 tx=0x7f9b78009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 -- 172.21.15.99:0/3117217795 shutdown_connections 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 --2- 172.21.15.99:0/3117217795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d3550 0x7f9b840ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 --2- 172.21.15.99:0/3117217795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840b3cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 -- 172.21.15.99:0/3117217795 >> 172.21.15.99:0/3117217795 conn(0x7f9b8408afd0 msgr2=0x7f9b8409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:54.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 -- 172.21.15.99:0/3117217795 shutdown_connections 2024-03-29T09:09:54.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.581+0000 7f9b8c400700 1 -- 172.21.15.99:0/3117217795 wait complete. 2024-03-29T09:09:54.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.583+0000 7f9b8c400700 1 Processor -- start 2024-03-29T09:09:54.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.583+0000 7f9b8c400700 1 -- start start 2024-03-29T09:09:54.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8c400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d0d60 0x7f9b840b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8c400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b840d3550 0x7f9b841515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8c400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8c400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840b7920 con 0x7f9b84160770 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8c400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840b6260 con 0x7f9b840d3550 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8c400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840cbd50 con 0x7f9b840d0d60 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b89000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b840d3550 0x7f9b841515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b89a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d0d60 0x7f9b840b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58568/0 (socket says 172.21.15.99:58568) 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.584+0000 7f9b8a400700 1 -- 172.21.15.99:0/2787913654 learned_addr learned my addr 172.21.15.99:0/2787913654 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8a400700 1 -- 172.21.15.99:0/2787913654 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d0d60 msgr2=0x7f9b840b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8a400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d0d60 0x7f9b840b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8a400700 1 -- 172.21.15.99:0/2787913654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b840d3550 msgr2=0x7f9b841515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8a400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b840d3550 0x7f9b841515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8a400700 1 -- 172.21.15.99:0/2787913654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78009550 con 0x7f9b84160770 2024-03-29T09:09:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8a400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840ccdf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00b9e0 tx=0x7f9b7c00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b81600700 1 -- 172.21.15.99:0/2787913654 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00d8e0 con 0x7f9b84160770 2024-03-29T09:09:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b81600700 1 -- 172.21.15.99:0/2787913654 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c00da80 con 0x7f9b84160770 2024-03-29T09:09:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b840cc030 con 0x7f9b84160770 2024-03-29T09:09:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b81600700 1 -- 172.21.15.99:0/2787913654 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00de30 con 0x7f9b84160770 2024-03-29T09:09:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.585+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b840cc5f0 con 0x7f9b84160770 2024-03-29T09:09:54.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.586+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b840678b0 con 0x7f9b84160770 2024-03-29T09:09:54.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.587+0000 7f9b81600700 1 -- 172.21.15.99:0/2787913654 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b7c015420 con 0x7f9b84160770 2024-03-29T09:09:54.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.588+0000 7f9b81600700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b6406ac90 0x7f9b6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:54.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.588+0000 7f9b81600700 1 -- 172.21.15.99:0/2787913654 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9b7c011070 con 0x7f9b84160770 2024-03-29T09:09:54.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.588+0000 7f9b89a00700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b6406ac90 0x7f9b6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:54.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.589+0000 7f9b89a00700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b6406ac90 0x7f9b6406d150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b78014040 tx=0x7f9b78008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:54.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.592+0000 7f9b81600700 1 -- 172.21.15.99:0/2787913654 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b7c04f580 con 0x7f9b84160770 2024-03-29T09:09:54.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.796+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b8406d130 con 0x7f9b84160770 2024-03-29T09:09:54.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.796+0000 7f9b81600700 1 -- 172.21.15.99:0/2787913654 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b7c0543a0 con 0x7f9b84160770 2024-03-29T09:09:54.796 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:09:54.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b6406ac90 msgr2=0x7f9b6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b6406ac90 0x7f9b6406d150 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b78014040 tx=0x7f9b78008030 comp rx=0 tx=0).stop 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 msgr2=0x7f9b840ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840ccdf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00b9e0 tx=0x7f9b7c00bcf0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 shutdown_connections 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b6406ac90 0x7f9b6406d150 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b84160770 0x7f9b840ccdf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b840d0d60 0x7f9b840b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 --2- 172.21.15.99:0/2787913654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b840d3550 0x7f9b841515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 >> 172.21.15.99:0/2787913654 conn(0x7f9b8408afd0 msgr2=0x7f9b8406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 shutdown_connections 2024-03-29T09:09:54.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:54.799+0000 7f9b8c400700 1 -- 172.21.15.99:0/2787913654 wait complete. 2024-03-29T09:09:54.817 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:09:55.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.009+0000 7f6e73c00700 1 Processor -- start 2024-03-29T09:09:55.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.009+0000 7f6e73c00700 1 -- start start 2024-03-29T09:09:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.010+0000 7f6e73c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e6c160770 0x7f6e6c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e73c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0d1520 con 0x7f6e6c0d3550 2024-03-29T09:09:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e73c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0d1800 con 0x7f6e6c0d0d60 2024-03-29T09:09:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e73c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c0d1ae0 con 0x7f6e6c160770 2024-03-29T09:09:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e71c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6e6c0d3550 0x7f6e6c0ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56084/0 (socket says 172.21.15.99:56084) 2024-03-29T09:09:55.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e71c00700 1 -- 172.21.15.99:0/1597704187 learned_addr learned my addr 172.21.15.99:0/1597704187 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:55.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395220030 0 0) 0x7f6e6c0d1520 con 0x7f6e6c0d3550 2024-03-29T09:09:55.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e71200700 1 --2- 172.21.15.99:0/1597704187 >> v2:172.21.15.133:3300/0 conn(0x7f6e6c160770 0x7f6e6c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.011+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e540035b0 con 0x7f6e6c0d3550 2024-03-29T09:09:55.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1672901961 0 0) 0x7f6e540035b0 con 0x7f6e6c0d3550 2024-03-29T09:09:55.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 >> v2:172.21.15.133:3300/0 conn(0x7f6e6c160770 msgr2=0x7f6e6c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 --2- 172.21.15.99:0/1597704187 >> v2:172.21.15.133:3300/0 conn(0x7f6e6c160770 0x7f6e6c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 >> v1:172.21.15.133:6789/0 conn(0x7f6e6c0d0d60 legacy=0x7f6e6c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0d1dc0 con 0x7f6e6c0d3550 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f6e640019f0 con 0x7f6e6c0d3550 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 >> v1:172.21.15.99:6789/0 conn(0x7f6e6c0d3550 legacy=0x7f6e6c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54003a00 0x7f6e54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e540048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54005170 0x7f6e5400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0d1ae0 con 0x7f6e54004400 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c0b0 con 0x7f6e54005170 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5400c270 con 0x7f6e54003a00 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e540048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e71200700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54003a00 0x7f6e54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e71c00700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54005170 0x7f6e5400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6be00700 1 -- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54003a00 msgr2=0x7f6e54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e54003a00 0x7f6e54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6be00700 1 -- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54005170 msgr2=0x7f6e5400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.012+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e54005170 0x7f6e5400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e6be00700 1 -- 172.21.15.99:0/1597704187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540035b0 con 0x7f6e54004400 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e71c00700 1 -- 172.21.15.99:0/1597704187 reap_dead start 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e540048d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e580018d0 tx=0x7f6e54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e15042fb3398f73 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e580052d0 con 0x7f6e54004400 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e58005430 con 0x7f6e54004400 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e6b400700 1 -- 172.21.15.99:0/1597704187 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e580038a0 con 0x7f6e54004400 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e73c00700 1 -- 172.21.15.99:0/1597704187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 msgr2=0x7f6e540048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.013+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e540048d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6e580018d0 tx=0x7f6e54005880 comp rx=0 tx=0).stop 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.014+0000 7f6e73c00700 1 -- 172.21.15.99:0/1597704187 shutdown_connections 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.014+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1597704187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e540048d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.014+0000 7f6e73c00700 1 -- 172.21.15.99:0/1597704187 >> 172.21.15.99:0/1597704187 conn(0x7f6e6c08afd0 msgr2=0x7f6e6c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:55.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.014+0000 7f6e73c00700 1 -- 172.21.15.99:0/1597704187 shutdown_connections 2024-03-29T09:09:55.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.014+0000 7f6e73c00700 1 -- 172.21.15.99:0/1597704187 wait complete. 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.016+0000 7f6e73c00700 1 Processor -- start 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.016+0000 7f6e73c00700 1 -- start start 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.017+0000 7f6e73c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e6c0b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.017+0000 7f6e73c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e6c155b10 0x7f6e6c0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.017+0000 7f6e73c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c155f90 0x7f6e6c1f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.017+0000 7f6e73c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0b7250 con 0x7f6e54004400 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.017+0000 7f6e73c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0b7530 con 0x7f6e6c155f90 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.017+0000 7f6e73c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0b7810 con 0x7f6e6c155b10 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e6c0b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e6c0b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58586/0 (socket says 172.21.15.99:58586) 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 -- 172.21.15.99:0/1554075195 learned_addr learned my addr 172.21.15.99:0/1554075195 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c155f90 0x7f6e6c1f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e6c155b10 0x7f6e6c0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 -- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e6c155b10 msgr2=0x7f6e6c0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e6c155b10 0x7f6e6c0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 -- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c155f90 msgr2=0x7f6e6c1f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c155f90 0x7f6e6c1f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 -- 172.21.15.99:0/1554075195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0b7af0 con 0x7f6e54004400 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e6c155b10 0x7f6e6c0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e71200700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e6c0b8010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e600119d0 tx=0x7f6e60011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e68c00700 1 -- 172.21.15.99:0/1554075195 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e60009630 con 0x7f6e54004400 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e68c00700 1 -- 172.21.15.99:0/1554075195 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e60004240 con 0x7f6e54004400 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.018+0000 7f6e68c00700 1 -- 172.21.15.99:0/1554075195 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6000f730 con 0x7f6e54004400 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.019+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c1f3bb0 con 0x7f6e54004400 2024-03-29T09:09:55.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.019+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c0c7be0 con 0x7f6e54004400 2024-03-29T09:09:55.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.020+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c0b7e60 con 0x7f6e54004400 2024-03-29T09:09:55.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.021+0000 7f6e68c00700 1 -- 172.21.15.99:0/1554075195 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6e6000f890 con 0x7f6e54004400 2024-03-29T09:09:55.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.021+0000 7f6e68c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e4c06ac90 0x7f6e4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.021+0000 7f6e68c00700 1 -- 172.21.15.99:0/1554075195 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6e60089cd0 con 0x7f6e54004400 2024-03-29T09:09:55.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.022+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e4c06ac90 0x7f6e4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.022+0000 7f6e6be00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e4c06ac90 0x7f6e4c06d150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e58015040 tx=0x7f6e58005000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.026+0000 7f6e68c00700 1 -- 172.21.15.99:0/1554075195 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6e60054b60 con 0x7f6e54004400 2024-03-29T09:09:55.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.231+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6e6c0b7390 con 0x7f6e54004400 2024-03-29T09:09:55.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.232+0000 7f6e68c00700 1 -- 172.21.15.99:0/1554075195 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6e6c0b7390 con 0x7f6e54004400 2024-03-29T09:09:55.232 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.234+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e4c06ac90 msgr2=0x7f6e4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.234+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e4c06ac90 0x7f6e4c06d150 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e58015040 tx=0x7f6e58005000 comp rx=0 tx=0).stop 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.234+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 msgr2=0x7f6e6c0b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e6c0b8010 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e600119d0 tx=0x7f6e60011d90 comp rx=0 tx=0).stop 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 shutdown_connections 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e4c06ac90 0x7f6e4c06d150 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e6c155f90 0x7f6e6c1f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e54004400 0x7f6e6c0b8010 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 --2- 172.21.15.99:0/1554075195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e6c155b10 0x7f6e6c0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 >> 172.21.15.99:0/1554075195 conn(0x7f6e6c08afd0 msgr2=0x7f6e6c15e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:55.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 shutdown_connections 2024-03-29T09:09:55.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.235+0000 7f6e73c00700 1 -- 172.21.15.99:0/1554075195 wait complete. 2024-03-29T09:09:55.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:55.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.442+0000 7f7f9b600700 1 Processor -- start 2024-03-29T09:09:55.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.442+0000 7f7f9b600700 1 -- start start 2024-03-29T09:09:55.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.443+0000 7f7f9b600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 0x7f7f940d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.443+0000 7f7f9b600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940cddc0 con 0x7f7f941608e0 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.443+0000 7f7f9b600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f940ce0a0 con 0x7f7f94160cc0 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.443+0000 7f7f9b600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940ce380 con 0x7f7f940d1b20 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.443+0000 7f7f98c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7f941608e0 0x7f7f940d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56100/0 (socket says 172.21.15.99:56100) 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f98c00700 1 -- 172.21.15.99:0/1806294703 learned_addr learned my addr 172.21.15.99:0/1806294703 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93e00700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 0x7f7f940d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706495807 0 0) 0x7f7f940cddc0 con 0x7f7f941608e0 2024-03-29T09:09:55.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f7c0035b0 con 0x7f7f941608e0 2024-03-29T09:09:55.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195700759 0 0) 0x7f7f940ce0a0 con 0x7f7f94160cc0 2024-03-29T09:09:55.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f940cddc0 con 0x7f7f94160cc0 2024-03-29T09:09:55.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 134172523 0 0) 0x7f7f7c0035b0 con 0x7f7f941608e0 2024-03-29T09:09:55.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 msgr2=0x7f7f940d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 0x7f7f940d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 >> v1:172.21.15.133:6789/0 conn(0x7f7f94160cc0 legacy=0x7f7f940afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f940ce660 con 0x7f7f941608e0 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f7f88001670 con 0x7f7f941608e0 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 >> v1:172.21.15.99:6789/0 conn(0x7f7f941608e0 legacy=0x7f7f940d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7c003bc0 0x7f7f7c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 0x7f7f7c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c004d90 0x7f7f7c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f940ce380 con 0x7f7f7c003bc0 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c00bac0 con 0x7f7f7c004d90 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.444+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c00bc30 con 0x7f7f940d1b20 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f98c00700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7c003bc0 0x7f7f7c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f93e00700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 0x7f7f7c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f99600700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c004d90 0x7f7f7c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f98c00700 1 -- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 msgr2=0x7f7f7c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f98c00700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f940d1b20 0x7f7f7c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f98c00700 1 -- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c004d90 msgr2=0x7f7f7c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f98c00700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c004d90 0x7f7f7c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f98c00700 1 -- 172.21.15.99:0/1806294703 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0035b0 con 0x7f7f7c003bc0 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.445+0000 7f7f98c00700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7c003bc0 0x7f7f7c003fc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800c210 tx=0x7f7f7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2c002506cca390 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8800cd50 con 0x7f7f7c003bc0 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88005960 con 0x7f7f7c003bc0 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f93400700 1 -- 172.21.15.99:0/1806294703 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88005d10 con 0x7f7f7c003bc0 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f9b600700 1 -- 172.21.15.99:0/1806294703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7c003bc0 msgr2=0x7f7f7c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f9b600700 1 --2- 172.21.15.99:0/1806294703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f7c003bc0 0x7f7f7c003fc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800c210 tx=0x7f7f7c005880 comp rx=0 tx=0).stop 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f99600700 1 -- 172.21.15.99:0/1806294703 reap_dead start 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f9b600700 1 -- 172.21.15.99:0/1806294703 shutdown_connections 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f9b600700 1 -- 172.21.15.99:0/1806294703 >> 172.21.15.99:0/1806294703 conn(0x7f7f9409c490 msgr2=0x7f7f940d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f9b600700 1 -- 172.21.15.99:0/1806294703 shutdown_connections 2024-03-29T09:09:55.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.446+0000 7f7f9b600700 1 -- 172.21.15.99:0/1806294703 wait complete. 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.448+0000 7f7f9b600700 1 Processor -- start 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.448+0000 7f7f9b600700 1 -- start start 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f9b600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c003bc0 0x7f7f94146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f9b600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f940ca220 0x7f7f941582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f9b600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f94158800 0x7f7f941487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f9b600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94157e30 con 0x7f7f940ca220 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f9b600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f94157500 con 0x7f7f7c003bc0 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f9b600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f941577e0 con 0x7f7f94158800 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f93e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f940ca220 0x7f7f941582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f93e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f940ca220 0x7f7f941582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58598/0 (socket says 172.21.15.99:58598) 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f93e00700 1 -- 172.21.15.99:0/1659319567 learned_addr learned my addr 172.21.15.99:0/1659319567 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:55.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f98c00700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c003bc0 0x7f7f94146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.449+0000 7f7f99600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f94158800 0x7f7f941487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f93e00700 1 -- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f94158800 msgr2=0x7f7f941487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f93e00700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f94158800 0x7f7f941487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f93e00700 1 -- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c003bc0 msgr2=0x7f7f94146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f93e00700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c003bc0 0x7f7f94146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f93e00700 1 -- 172.21.15.99:0/1659319567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f94157ac0 con 0x7f7f940ca220 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f99600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f94158800 0x7f7f941487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f93e00700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f940ca220 0x7f7f941582c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f80011aa0 tx=0x7f7f80011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f90c00700 1 -- 172.21.15.99:0/1659319567 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000f940 con 0x7f7f940ca220 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f90c00700 1 -- 172.21.15.99:0/1659319567 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8000faa0 con 0x7f7f940ca220 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f90c00700 1 -- 172.21.15.99:0/1659319567 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000fe50 con 0x7f7f940ca220 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f94146a60 con 0x7f7f940ca220 2024-03-29T09:09:55.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.450+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f94155800 con 0x7f7f940ca220 2024-03-29T09:09:55.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.451+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9409c8a0 con 0x7f7f940ca220 2024-03-29T09:09:55.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.452+0000 7f7f90c00700 1 -- 172.21.15.99:0/1659319567 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7f80014440 con 0x7f7f940ca220 2024-03-29T09:09:55.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.453+0000 7f7f90c00700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7f7406ac90 0x7f7f7406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.453+0000 7f7f90c00700 1 -- 172.21.15.99:0/1659319567 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7f80089ee0 con 0x7f7f940ca220 2024-03-29T09:09:55.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.453+0000 7f7f98c00700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7f7406ac90 0x7f7f7406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.454+0000 7f7f98c00700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7f7406ac90 0x7f7f7406d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f880018b0 tx=0x7f7f88007f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.458+0000 7f7f90c00700 1 -- 172.21.15.99:0/1659319567 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7f80054e40 con 0x7f7f940ca220 2024-03-29T09:09:55.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.665+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f94157c50 con 0x7f7f940ca220 2024-03-29T09:09:55.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.666+0000 7f7f90c00700 1 -- 172.21.15.99:0/1659319567 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f7f80022020 con 0x7f7f940ca220 2024-03-29T09:09:55.666 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:55.666 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.668+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7f7406ac90 msgr2=0x7f7f7406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.668+0000 7f7f9b600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7f7406ac90 0x7f7f7406d150 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f880018b0 tx=0x7f7f88007f90 comp rx=0 tx=0).stop 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f940ca220 msgr2=0x7f7f941582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f940ca220 0x7f7f941582c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7f80011aa0 tx=0x7f7f80011db0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 shutdown_connections 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7f7406ac90 0x7f7f7406d150 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f94158800 0x7f7f941487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c003bc0 0x7f7f94146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 --2- 172.21.15.99:0/1659319567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f940ca220 0x7f7f941582c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 >> 172.21.15.99:0/1659319567 conn(0x7f7f9409c490 msgr2=0x7f7f940cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:55.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 shutdown_connections 2024-03-29T09:09:55.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.669+0000 7f7f9b600700 1 -- 172.21.15.99:0/1659319567 wait complete. 2024-03-29T09:09:55.687 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:09:55.687 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:09:55.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.878+0000 7eff34000700 1 Processor -- start 2024-03-29T09:09:55.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.878+0000 7eff34000700 1 -- start start 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff34000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff2c0adc40 0x7eff2c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff34000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff2c072120 0x7eff2c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff34000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2c06c580 con 0x7eff2c06d230 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff34000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c15c7a0 con 0x7eff2c072120 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff34000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2c06c8d0 con 0x7eff2c0adc40 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff30c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff2c072120 0x7eff2c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff30c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7eff2c072120 0x7eff2c06ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58608/0 (socket says 172.21.15.99:58608) 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff30c00700 1 -- 172.21.15.99:0/4150083667 learned_addr learned my addr 172.21.15.99:0/4150083667 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff31600700 1 --2- 172.21.15.99:0/4150083667 >> v2:172.21.15.133:3300/0 conn(0x7eff2c0adc40 0x7eff2c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff30c00700 1 -- 172.21.15.99:0/4150083667 >> v2:172.21.15.133:3300/0 conn(0x7eff2c0adc40 msgr2=0x7eff2c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff30c00700 1 --2- 172.21.15.99:0/4150083667 >> v2:172.21.15.133:3300/0 conn(0x7eff2c0adc40 0x7eff2c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff30c00700 1 -- 172.21.15.99:0/4150083667 >> v1:172.21.15.133:6789/0 conn(0x7eff2c06d230 legacy=0x7eff2c155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:55.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.880+0000 7eff30c00700 1 -- 172.21.15.99:0/4150083667 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0abf10 con 0x7eff2c072120 2024-03-29T09:09:55.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.881+0000 7eff30c00700 1 --2- 172.21.15.99:0/4150083667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c06ccf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0b4230 tx=0x7eff18009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b271823faa78e915 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.881+0000 7eff2be00700 1 -- 172.21.15.99:0/4150083667 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18004230 con 0x7eff2c072120 2024-03-29T09:09:55.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.881+0000 7eff2be00700 1 -- 172.21.15.99:0/4150083667 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff18004390 con 0x7eff2c072120 2024-03-29T09:09:55.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.881+0000 7eff2be00700 1 -- 172.21.15.99:0/4150083667 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180036a0 con 0x7eff2c072120 2024-03-29T09:09:55.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.881+0000 7eff34000700 1 -- 172.21.15.99:0/4150083667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 msgr2=0x7eff2c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.881+0000 7eff34000700 1 --2- 172.21.15.99:0/4150083667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c06ccf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0b4230 tx=0x7eff18009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.882+0000 7eff34000700 1 -- 172.21.15.99:0/4150083667 shutdown_connections 2024-03-29T09:09:55.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.882+0000 7eff34000700 1 --2- 172.21.15.99:0/4150083667 >> v2:172.21.15.133:3300/0 conn(0x7eff2c0adc40 0x7eff2c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.882+0000 7eff34000700 1 --2- 172.21.15.99:0/4150083667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c06ccf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.882+0000 7eff34000700 1 -- 172.21.15.99:0/4150083667 >> 172.21.15.99:0/4150083667 conn(0x7eff2c066540 msgr2=0x7eff2c066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:55.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.882+0000 7eff34000700 1 -- 172.21.15.99:0/4150083667 shutdown_connections 2024-03-29T09:09:55.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.882+0000 7eff34000700 1 -- 172.21.15.99:0/4150083667 wait complete. 2024-03-29T09:09:55.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.883+0000 7eff34000700 1 Processor -- start 2024-03-29T09:09:55.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.883+0000 7eff34000700 1 -- start start 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.884+0000 7eff34000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff2c06d230 0x7eff2c153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.884+0000 7eff34000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.884+0000 7eff34000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff2c0adc40 0x7eff2c09be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.884+0000 7eff34000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c145a60 con 0x7eff2c072120 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.884+0000 7eff34000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0ccfa0 con 0x7eff2c06d230 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.884+0000 7eff34000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c0cd280 con 0x7eff2c0adc40 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58624/0 (socket says 172.21.15.99:58624) 2024-03-29T09:09:55.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 -- 172.21.15.99:0/2256238083 learned_addr learned my addr 172.21.15.99:0/2256238083 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff31600700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff2c06d230 0x7eff2c153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff32000700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff2c0adc40 0x7eff2c09be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 -- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff2c0adc40 msgr2=0x7eff2c09be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff2c0adc40 0x7eff2c09be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 -- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff2c06d230 msgr2=0x7eff2c153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff2c06d230 0x7eff2c153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 -- 172.21.15.99:0/2256238083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18009550 con 0x7eff2c072120 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff31600700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff2c06d230 0x7eff2c153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff30c00700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c153790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff18009130 tx=0x7eff18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff29600700 1 -- 172.21.15.99:0/2256238083 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18003d40 con 0x7eff2c072120 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff29600700 1 -- 172.21.15.99:0/2256238083 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1801f070 con 0x7eff2c072120 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c0cd560 con 0x7eff2c072120 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.885+0000 7eff29600700 1 -- 172.21.15.99:0/2256238083 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff18011870 con 0x7eff2c072120 2024-03-29T09:09:55.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.886+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c09c390 con 0x7eff2c072120 2024-03-29T09:09:55.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.887+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c145c10 con 0x7eff2c072120 2024-03-29T09:09:55.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.887+0000 7eff29600700 1 -- 172.21.15.99:0/2256238083 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7eff18021030 con 0x7eff2c072120 2024-03-29T09:09:55.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.889+0000 7eff29600700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff0c06acc0 0x7eff0c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:55.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.889+0000 7eff29600700 1 -- 172.21.15.99:0/2256238083 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7eff1808ac00 con 0x7eff2c072120 2024-03-29T09:09:55.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.889+0000 7eff31600700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff0c06acc0 0x7eff0c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:55.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.889+0000 7eff31600700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff0c06acc0 0x7eff0c06d180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff2c142140 tx=0x7eff20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:55.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:55.893+0000 7eff29600700 1 -- 172.21.15.99:0/2256238083 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7eff2c145c10 con 0x7eff2c072120 2024-03-29T09:09:56.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.098+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff2c0678b0 con 0x7eff2c072120 2024-03-29T09:09:56.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.098+0000 7eff29600700 1 -- 172.21.15.99:0/2256238083 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff18055a80 con 0x7eff2c072120 2024-03-29T09:09:56.098 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.100+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff0c06acc0 msgr2=0x7eff0c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.100+0000 7eff34000700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff0c06acc0 0x7eff0c06d180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff2c142140 tx=0x7eff20008040 comp rx=0 tx=0).stop 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.100+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 msgr2=0x7eff2c153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.100+0000 7eff34000700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c153790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff18009130 tx=0x7eff18004750 comp rx=0 tx=0).stop 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 shutdown_connections 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff0c06acc0 0x7eff0c06d180 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7eff2c0adc40 0x7eff2c09be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff2c06d230 0x7eff2c153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 --2- 172.21.15.99:0/2256238083 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff2c072120 0x7eff2c153790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 >> 172.21.15.99:0/2256238083 conn(0x7eff2c066540 msgr2=0x7eff2c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 shutdown_connections 2024-03-29T09:09:56.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.101+0000 7eff34000700 1 -- 172.21.15.99:0/2256238083 wait complete. 2024-03-29T09:09:56.119 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:09:56.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.310+0000 7fab5fc00700 1 Processor -- start 2024-03-29T09:09:56.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.310+0000 7fab5fc00700 1 -- start start 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.311+0000 7fab5fc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fab5806dda0 0x7fab580b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.311+0000 7fab5fc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab5815a240 0x7fab5815dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab5fc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580ac990 con 0x7fab58159e60 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab5fc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5815e230 con 0x7fab5806dda0 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab5fc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5815e510 con 0x7fab5815a240 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab57e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fab5806dda0 0x7fab580b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab5d200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fab58159e60 0x7fab5806d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56112/0 (socket says 172.21.15.99:56112) 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab5d200700 1 -- 172.21.15.99:0/2823359515 learned_addr learned my addr 172.21.15.99:0/2823359515 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab5dc00700 1 --2- 172.21.15.99:0/2823359515 >> v2:172.21.15.133:3300/0 conn(0x7fab5815a240 0x7fab5815dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab57e00700 1 -- 172.21.15.99:0/2823359515 >> v2:172.21.15.133:3300/0 conn(0x7fab5815a240 msgr2=0x7fab5815dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab57e00700 1 --2- 172.21.15.99:0/2823359515 >> v2:172.21.15.133:3300/0 conn(0x7fab5815a240 0x7fab5815dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab57e00700 1 -- 172.21.15.99:0/2823359515 >> v1:172.21.15.99:6789/0 conn(0x7fab58159e60 legacy=0x7fab5806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab57e00700 1 -- 172.21.15.99:0/2823359515 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5815e7f0 con 0x7fab5806dda0 2024-03-29T09:09:56.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.312+0000 7fab57400700 1 -- 172.21.15.99:0/2823359515 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896898459 0 0) 0x7fab580ac990 con 0x7fab58159e60 2024-03-29T09:09:56.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab57e00700 1 --2- 172.21.15.99:0/2823359515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5806dda0 0x7fab580b2b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab580b4260 tx=0x7fab44009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5b387074424fff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:56.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab57400700 1 -- 172.21.15.99:0/2823359515 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44011830 con 0x7fab5806dda0 2024-03-29T09:09:56.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab57400700 1 -- 172.21.15.99:0/2823359515 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44011990 con 0x7fab5806dda0 2024-03-29T09:09:56.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab57400700 1 -- 172.21.15.99:0/2823359515 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44011d20 con 0x7fab5806dda0 2024-03-29T09:09:56.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 -- 172.21.15.99:0/2823359515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5806dda0 msgr2=0x7fab580b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 --2- 172.21.15.99:0/2823359515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5806dda0 0x7fab580b2b60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab580b4260 tx=0x7fab44009b20 comp rx=0 tx=0).stop 2024-03-29T09:09:56.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 -- 172.21.15.99:0/2823359515 shutdown_connections 2024-03-29T09:09:56.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 --2- 172.21.15.99:0/2823359515 >> v2:172.21.15.133:3300/0 conn(0x7fab5815a240 0x7fab5815dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 --2- 172.21.15.99:0/2823359515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab5806dda0 0x7fab580b2b60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 -- 172.21.15.99:0/2823359515 >> 172.21.15.99:0/2823359515 conn(0x7fab58066540 msgr2=0x7fab58066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:56.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 -- 172.21.15.99:0/2823359515 shutdown_connections 2024-03-29T09:09:56.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.313+0000 7fab5fc00700 1 -- 172.21.15.99:0/2823359515 wait complete. 2024-03-29T09:09:56.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.315+0000 7fab5fc00700 1 Processor -- start 2024-03-29T09:09:56.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.315+0000 7fab5fc00700 1 -- start start 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab5fc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5806dda0 0x7fab58152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab5fc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58159e60 0x7fab5815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab5fc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab5815a240 0x7fab5809af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab5fc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab58153000 con 0x7fab58159e60 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab5fc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5815ac70 con 0x7fab5806dda0 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab5fc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5809a950 con 0x7fab5815a240 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab57e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58159e60 0x7fab5815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab57e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58159e60 0x7fab5815cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58630/0 (socket says 172.21.15.99:58630) 2024-03-29T09:09:56.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab57e00700 1 -- 172.21.15.99:0/3510502474 learned_addr learned my addr 172.21.15.99:0/3510502474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.316+0000 7fab5d200700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5806dda0 0x7fab58152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab5dc00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab5815a240 0x7fab5809af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab57e00700 1 -- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab5815a240 msgr2=0x7fab5809af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab57e00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab5815a240 0x7fab5809af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab57e00700 1 -- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5806dda0 msgr2=0x7fab58152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab57e00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5806dda0 0x7fab58152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab57e00700 1 -- 172.21.15.99:0/3510502474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440094a0 con 0x7fab58159e60 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab57e00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58159e60 0x7fab5815cb10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fab440090b0 tx=0x7fab44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab54c00700 1 -- 172.21.15.99:0/3510502474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab440036a0 con 0x7fab58159e60 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab54c00700 1 -- 172.21.15.99:0/3510502474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4401e070 con 0x7fab58159e60 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab54c00700 1 -- 172.21.15.99:0/3510502474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44003a20 con 0x7fab58159e60 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5809ac30 con 0x7fab58159e60 2024-03-29T09:09:56.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.317+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5815a900 con 0x7fab58159e60 2024-03-29T09:09:56.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.318+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5815ae00 con 0x7fab58159e60 2024-03-29T09:09:56.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.319+0000 7fab54c00700 1 -- 172.21.15.99:0/3510502474 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fab44003b80 con 0x7fab58159e60 2024-03-29T09:09:56.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.320+0000 7fab54c00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab3806ac90 0x7fab3806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.320+0000 7fab54c00700 1 -- 172.21.15.99:0/3510502474 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fab4408a6b0 con 0x7fab58159e60 2024-03-29T09:09:56.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.320+0000 7fab5d200700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab3806ac90 0x7fab3806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.321+0000 7fab5d200700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab3806ac90 0x7fab3806d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab4c007fd0 tx=0x7fab4c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.324+0000 7fab54c00700 1 -- 172.21.15.99:0/3510502474 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fab440555d0 con 0x7fab58159e60 2024-03-29T09:09:56.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.527+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab580678b0 con 0x7fab58159e60 2024-03-29T09:09:56.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.528+0000 7fab54c00700 1 -- 172.21.15.99:0/3510502474 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab44058bf0 con 0x7fab58159e60 2024-03-29T09:09:56.528 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab3806ac90 msgr2=0x7fab3806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab3806ac90 0x7fab3806d150 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fab4c007fd0 tx=0x7fab4c007de0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58159e60 msgr2=0x7fab5815cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58159e60 0x7fab5815cb10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fab440090b0 tx=0x7fab44004750 comp rx=0 tx=0).stop 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 shutdown_connections 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab3806ac90 0x7fab3806d150 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab5815a240 0x7fab5809af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab5806dda0 0x7fab58152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 --2- 172.21.15.99:0/3510502474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab58159e60 0x7fab5815cb10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 >> 172.21.15.99:0/3510502474 conn(0x7fab58066540 msgr2=0x7fab580b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 shutdown_connections 2024-03-29T09:09:56.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.530+0000 7fab5fc00700 1 -- 172.21.15.99:0/3510502474 wait complete. 2024-03-29T09:09:56.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:09:56.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.741+0000 7f6015e00700 1 Processor -- start 2024-03-29T09:09:56.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.741+0000 7f6015e00700 1 -- start start 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.742+0000 7f6015e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f60100b3cc0 0x7f60100d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.742+0000 7f6015e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f60100ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.742+0000 7f6015e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60100d3810 con 0x7f6010160770 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.742+0000 7f6015e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60100b4140 con 0x7f60100b3cc0 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.742+0000 7f6015e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60100d1520 con 0x7f60100d3380 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600f400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f60100ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600ea00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6010160770 0x7f60100d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:45258/0 (socket says 172.21.15.99:45258) 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600ea00700 1 -- 172.21.15.99:0/2703853058 learned_addr learned my addr 172.21.15.99:0/2703853058 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:56.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600e000700 1 --2- 172.21.15.99:0/2703853058 >> v2:172.21.15.99:3300/0 conn(0x7f60100b3cc0 0x7f60100d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600f400700 1 -- 172.21.15.99:0/2703853058 >> v2:172.21.15.99:3300/0 conn(0x7f60100b3cc0 msgr2=0x7f60100d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600f400700 1 --2- 172.21.15.99:0/2703853058 >> v2:172.21.15.99:3300/0 conn(0x7f60100b3cc0 0x7f60100d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600f400700 1 -- 172.21.15.99:0/2703853058 >> v1:172.21.15.133:6789/0 conn(0x7f6010160770 legacy=0x7f60100d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.743+0000 7f600f400700 1 -- 172.21.15.99:0/2703853058 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100d1800 con 0x7f60100d3380 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.744+0000 7f600f400700 1 --2- 172.21.15.99:0/2703853058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f60100ce080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6008009d90 tx=0x7f600800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f0917269fa1440 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.744+0000 7f600d600700 1 -- 172.21.15.99:0/2703853058 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600800e070 con 0x7f60100d3380 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.744+0000 7f600d600700 1 -- 172.21.15.99:0/2703853058 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600800bb20 con 0x7f60100d3380 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.744+0000 7f600d600700 1 -- 172.21.15.99:0/2703853058 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008012430 con 0x7f60100d3380 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.744+0000 7f6015e00700 1 -- 172.21.15.99:0/2703853058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 msgr2=0x7f60100ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.744+0000 7f6015e00700 1 --2- 172.21.15.99:0/2703853058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f60100ce080 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6008009d90 tx=0x7f600800b330 comp rx=0 tx=0).stop 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.745+0000 7f6015e00700 1 -- 172.21.15.99:0/2703853058 shutdown_connections 2024-03-29T09:09:56.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.745+0000 7f6015e00700 1 --2- 172.21.15.99:0/2703853058 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f60100ce080 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.745+0000 7f6015e00700 1 --2- 172.21.15.99:0/2703853058 >> v2:172.21.15.99:3300/0 conn(0x7f60100b3cc0 0x7f60100d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.745+0000 7f6015e00700 1 -- 172.21.15.99:0/2703853058 >> 172.21.15.99:0/2703853058 conn(0x7f601008afd0 msgr2=0x7f601009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:56.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.745+0000 7f6015e00700 1 -- 172.21.15.99:0/2703853058 shutdown_connections 2024-03-29T09:09:56.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.745+0000 7f6015e00700 1 -- 172.21.15.99:0/2703853058 wait complete. 2024-03-29T09:09:56.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.746+0000 7f6015e00700 1 Processor -- start 2024-03-29T09:09:56.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.746+0000 7f6015e00700 1 -- start start 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.747+0000 7f6015e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60100b3cc0 0x7f60101545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.747+0000 7f6015e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f601006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.747+0000 7f6015e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6010160770 0x7f601006c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.747+0000 7f6015e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60100a0d70 con 0x7f60100b3cc0 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.747+0000 7f6015e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60100b7670 con 0x7f6010160770 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.747+0000 7f6015e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6010154b10 con 0x7f60100d3380 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60100b3cc0 0x7f60101545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60100b3cc0 0x7f60101545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58654/0 (socket says 172.21.15.99:58654) 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 -- 172.21.15.99:0/3798637899 learned_addr learned my addr 172.21.15.99:0/3798637899 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:56.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600e000700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f601006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600f400700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6010160770 0x7f601006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 -- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 msgr2=0x7f601006bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f601006bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 -- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6010160770 msgr2=0x7f601006c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6010160770 0x7f601006c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 -- 172.21.15.99:0/3798637899 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008009a70 con 0x7f60100b3cc0 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600f400700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6010160770 0x7f601006c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.748+0000 7f600ea00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60100b3cc0 0x7f60101545d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f600000c480 tx=0x7f600000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.749+0000 7f6007400700 1 -- 172.21.15.99:0/3798637899 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000005d70 con 0x7f60100b3cc0 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.749+0000 7f6007400700 1 -- 172.21.15.99:0/3798637899 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60000070d0 con 0x7f60100b3cc0 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.749+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60100b69f0 con 0x7f60100b3cc0 2024-03-29T09:09:56.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.749+0000 7f6007400700 1 -- 172.21.15.99:0/3798637899 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000007440 con 0x7f60100b3cc0 2024-03-29T09:09:56.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.749+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60100b6fb0 con 0x7f60100b3cc0 2024-03-29T09:09:56.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.750+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60100678b0 con 0x7f60100b3cc0 2024-03-29T09:09:56.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.751+0000 7f6007400700 1 -- 172.21.15.99:0/3798637899 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f60000075a0 con 0x7f60100b3cc0 2024-03-29T09:09:56.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.752+0000 7f6007400700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5ff006ac90 0x7f5ff006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:56.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.752+0000 7f6007400700 1 -- 172.21.15.99:0/3798637899 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6000088900 con 0x7f60100b3cc0 2024-03-29T09:09:56.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.752+0000 7f600e000700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5ff006ac90 0x7f5ff006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:56.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.753+0000 7f600e000700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5ff006ac90 0x7f5ff006d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009be0 tx=0x7f5ffc009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:56.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.756+0000 7f6007400700 1 -- 172.21.15.99:0/3798637899 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6000053910 con 0x7f60100b3cc0 2024-03-29T09:09:56.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.962+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60100b45b0 con 0x7f60100b3cc0 2024-03-29T09:09:56.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.962+0000 7f6007400700 1 -- 172.21.15.99:0/3798637899 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6000056f30 con 0x7f60100b3cc0 2024-03-29T09:09:56.962 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.964+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5ff006ac90 msgr2=0x7f5ff006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.964+0000 7f6015e00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5ff006ac90 0x7f5ff006d150 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009be0 tx=0x7f5ffc009210 comp rx=0 tx=0).stop 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.964+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60100b3cc0 msgr2=0x7f60101545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.964+0000 7f6015e00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60100b3cc0 0x7f60101545d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f600000c480 tx=0x7f600000c790 comp rx=0 tx=0).stop 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 shutdown_connections 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5ff006ac90 0x7f5ff006d150 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6010160770 0x7f601006c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f60100b3cc0 0x7f60101545d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 --2- 172.21.15.99:0/3798637899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f60100d3380 0x7f601006bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 >> 172.21.15.99:0/3798637899 conn(0x7f601008afd0 msgr2=0x7f6010073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:56.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 shutdown_connections 2024-03-29T09:09:56.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:56.965+0000 7f6015e00700 1 -- 172.21.15.99:0/3798637899 wait complete. 2024-03-29T09:09:56.983 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:09:57.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.177+0000 7f3272c00700 1 Processor -- start 2024-03-29T09:09:57.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.177+0000 7f3272c00700 1 -- start start 2024-03-29T09:09:57.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.178+0000 7f3272c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3272c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f326c0d3550 0x7f326c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3272c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c0d1520 con 0x7f326c0d0d60 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3272c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c0d1800 con 0x7f326c0d3550 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3272c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0d1ae0 con 0x7f326c160770 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3270c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f326c0d3550 0x7f326c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3270c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f326c0d3550 0x7f326c0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58664/0 (socket says 172.21.15.99:58664) 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3270c00700 1 -- 172.21.15.99:0/160330816 learned_addr learned my addr 172.21.15.99:0/160330816 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f326be00700 1 --2- 172.21.15.99:0/160330816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3270c00700 1 -- 172.21.15.99:0/160330816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 msgr2=0x7f326c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3270c00700 1 --2- 172.21.15.99:0/160330816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3270c00700 1 -- 172.21.15.99:0/160330816 >> v1:172.21.15.133:6789/0 conn(0x7f326c0d0d60 legacy=0x7f326c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.179+0000 7f3270c00700 1 -- 172.21.15.99:0/160330816 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c0d1dc0 con 0x7f326c0d3550 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3270c00700 1 --2- 172.21.15.99:0/160330816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ce080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3264009e50 tx=0x7f326400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61138f8f125b184 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:57.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f326aa00700 1 -- 172.21.15.99:0/160330816 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400e070 con 0x7f326c0d3550 2024-03-29T09:09:57.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f326aa00700 1 -- 172.21.15.99:0/160330816 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326400b9c0 con 0x7f326c0d3550 2024-03-29T09:09:57.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f326aa00700 1 -- 172.21.15.99:0/160330816 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400bd70 con 0x7f326c0d3550 2024-03-29T09:09:57.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3272c00700 1 -- 172.21.15.99:0/160330816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 msgr2=0x7f326c0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3272c00700 1 --2- 172.21.15.99:0/160330816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ce080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3264009e50 tx=0x7f326400b330 comp rx=0 tx=0).stop 2024-03-29T09:09:57.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3272c00700 1 -- 172.21.15.99:0/160330816 shutdown_connections 2024-03-29T09:09:57.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3272c00700 1 --2- 172.21.15.99:0/160330816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ce080 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3272c00700 1 --2- 172.21.15.99:0/160330816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3272c00700 1 -- 172.21.15.99:0/160330816 >> 172.21.15.99:0/160330816 conn(0x7f326c08afd0 msgr2=0x7f326c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:57.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.180+0000 7f3272c00700 1 -- 172.21.15.99:0/160330816 shutdown_connections 2024-03-29T09:09:57.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.181+0000 7f3272c00700 1 -- 172.21.15.99:0/160330816 wait complete. 2024-03-29T09:09:57.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.182+0000 7f3272c00700 1 Processor -- start 2024-03-29T09:09:57.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.182+0000 7f3272c00700 1 -- start start 2024-03-29T09:09:57.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.183+0000 7f3272c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326c0d0d60 0x7f326c14d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.183+0000 7f3272c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.183+0000 7f3272c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.183+0000 7f3272c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c14ce70 con 0x7f326c0d3550 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.183+0000 7f3272c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c0b04d0 con 0x7f326c0d0d60 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.183+0000 7f3272c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c14d8a0 con 0x7f326c160770 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.183+0000 7f326b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58676/0 (socket says 172.21.15.99:58676) 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 -- 172.21.15.99:0/2942615121 learned_addr learned my addr 172.21.15.99:0/2942615121 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f3270c00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326be00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326c0d0d60 0x7f326c14d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 -- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 msgr2=0x7f326c0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 -- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326c0d0d60 msgr2=0x7f326c14d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326c0d0d60 0x7f326c14d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 -- 172.21.15.99:0/2942615121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3264009b30 con 0x7f326c0d3550 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f326b400700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ca260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f325800c510 tx=0x7f325800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f3263e00700 1 -- 172.21.15.99:0/2942615121 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580052d0 con 0x7f326c0d3550 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f3263e00700 1 -- 172.21.15.99:0/2942615121 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258005430 con 0x7f326c0d3550 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.184+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c0b0740 con 0x7f326c0d3550 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.185+0000 7f3263e00700 1 -- 172.21.15.99:0/2942615121 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580038c0 con 0x7f326c0d3550 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.185+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c0b0d00 con 0x7f326c0d3550 2024-03-29T09:09:57.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.186+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c0b90a0 con 0x7f326c0d3550 2024-03-29T09:09:57.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.187+0000 7f3263e00700 1 -- 172.21.15.99:0/2942615121 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3258003a20 con 0x7f326c0d3550 2024-03-29T09:09:57.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.187+0000 7f3263e00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f324c06ac90 0x7f324c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.187+0000 7f3263e00700 1 -- 172.21.15.99:0/2942615121 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3258088aa0 con 0x7f326c0d3550 2024-03-29T09:09:57.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.188+0000 7f326be00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f324c06ac90 0x7f324c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.188+0000 7f326be00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f324c06ac90 0x7f324c06d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f325c006b20 tx=0x7f325c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:57.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.192+0000 7f3263e00700 1 -- 172.21.15.99:0/2942615121 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3258053890 con 0x7f326c0d3550 2024-03-29T09:09:57.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.394+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f326c0b4020 con 0x7f326c0d3550 2024-03-29T09:09:57.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.395+0000 7f3263e00700 1 -- 172.21.15.99:0/2942615121 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3258017020 con 0x7f326c0d3550 2024-03-29T09:09:57.395 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f324c06ac90 msgr2=0x7f324c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f324c06ac90 0x7f324c06d150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f325c006b20 tx=0x7f325c008040 comp rx=0 tx=0).stop 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 msgr2=0x7f326c0ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ca260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f325800c510 tx=0x7f325800c820 comp rx=0 tx=0).stop 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 shutdown_connections 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f324c06ac90 0x7f324c06d150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f326c160770 0x7f326c0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f326c0d0d60 0x7f326c14d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 --2- 172.21.15.99:0/2942615121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f326c0d3550 0x7f326c0ca260 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.397+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 >> 172.21.15.99:0/2942615121 conn(0x7f326c08afd0 msgr2=0x7f326c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.398+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 shutdown_connections 2024-03-29T09:09:57.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.398+0000 7f3272c00700 1 -- 172.21.15.99:0/2942615121 wait complete. 2024-03-29T09:09:57.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:57.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.607+0000 7fb577800700 1 Processor -- start 2024-03-29T09:09:57.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.607+0000 7fb577800700 1 -- start start 2024-03-29T09:09:57.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.608+0000 7fb577800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb57015d690 0x7fb5700cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb577800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5700afde0 con 0x7fb5700d1740 2024-03-29T09:09:57.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb577800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5700d01d0 con 0x7fb57015da70 2024-03-29T09:09:57.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb577800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5700d04b0 con 0x7fb57015d690 2024-03-29T09:09:57.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb574e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb57015d690 0x7fb5700cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb575800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb5700d1740 0x7fb5700b0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:56114/0 (socket says 172.21.15.99:56114) 2024-03-29T09:09:57.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb575800700 1 -- 172.21.15.99:0/2460946041 learned_addr learned my addr 172.21.15.99:0/2460946041 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb56f400700 1 -- 172.21.15.99:0/2460946041 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151301985 0 0) 0x7fb5700afde0 con 0x7fb5700d1740 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb56f400700 1 -- 172.21.15.99:0/2460946041 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5580035b0 con 0x7fb5700d1740 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb574e00700 1 -- 172.21.15.99:0/2460946041 >> v1:172.21.15.133:6789/0 conn(0x7fb57015da70 legacy=0x7fb5700cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb574e00700 1 -- 172.21.15.99:0/2460946041 >> v1:172.21.15.99:6789/0 conn(0x7fb5700d1740 legacy=0x7fb5700b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.609+0000 7fb574e00700 1 -- 172.21.15.99:0/2460946041 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5700b0030 con 0x7fb57015d690 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb574e00700 1 --2- 172.21.15.99:0/2460946041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb5700cfb60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb564009840 tx=0x7fb564009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6719386db6faa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb56f400700 1 -- 172.21.15.99:0/2460946041 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011830 con 0x7fb57015d690 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb56f400700 1 -- 172.21.15.99:0/2460946041 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564011990 con 0x7fb57015d690 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb56f400700 1 -- 172.21.15.99:0/2460946041 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564011d20 con 0x7fb57015d690 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb577800700 1 -- 172.21.15.99:0/2460946041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 msgr2=0x7fb5700cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb577800700 1 --2- 172.21.15.99:0/2460946041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb5700cfb60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb564009840 tx=0x7fb564009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:57.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb577800700 1 -- 172.21.15.99:0/2460946041 shutdown_connections 2024-03-29T09:09:57.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb577800700 1 --2- 172.21.15.99:0/2460946041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb5700cfb60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.610+0000 7fb577800700 1 -- 172.21.15.99:0/2460946041 >> 172.21.15.99:0/2460946041 conn(0x7fb57009c480 msgr2=0x7fb5700d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:57.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.611+0000 7fb577800700 1 -- 172.21.15.99:0/2460946041 shutdown_connections 2024-03-29T09:09:57.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.611+0000 7fb577800700 1 -- 172.21.15.99:0/2460946041 wait complete. 2024-03-29T09:09:57.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.612+0000 7fb577800700 1 Processor -- start 2024-03-29T09:09:57.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.612+0000 7fb577800700 1 -- start start 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.613+0000 7fb577800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5700d1740 0x7fb5700c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.613+0000 7fb577800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb57015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.613+0000 7fb577800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57015da70 0x7fb57015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.613+0000 7fb577800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57015f4d0 con 0x7fb57015d690 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.613+0000 7fb577800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700c7120 con 0x7fb57015da70 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.613+0000 7fb577800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700c7400 con 0x7fb5700d1740 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb57015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb57015b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58688/0 (socket says 172.21.15.99:58688) 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 -- 172.21.15.99:0/1365631972 learned_addr learned my addr 172.21.15.99:0/1365631972 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb575800700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57015da70 0x7fb57015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb574e00700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5700d1740 0x7fb5700c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 -- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5700d1740 msgr2=0x7fb5700c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5700d1740 0x7fb5700c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 -- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57015da70 msgr2=0x7fb57015a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57015da70 0x7fb57015a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 -- 172.21.15.99:0/1365631972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640094a0 con 0x7fb57015d690 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56fe00700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb57015b310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c4e0 tx=0x7fb55c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.614+0000 7fb56cc00700 1 -- 172.21.15.99:0/1365631972 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0052d0 con 0x7fb57015d690 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.615+0000 7fb56cc00700 1 -- 172.21.15.99:0/1365631972 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c005430 con 0x7fb57015d690 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.615+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5700c76e0 con 0x7fb57015d690 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.615+0000 7fb56cc00700 1 -- 172.21.15.99:0/1365631972 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0038c0 con 0x7fb57015d690 2024-03-29T09:09:57.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.615+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5700b8f70 con 0x7fb57015d690 2024-03-29T09:09:57.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.616+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57015f640 con 0x7fb57015d690 2024-03-29T09:09:57.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.617+0000 7fb56cc00700 1 -- 172.21.15.99:0/1365631972 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb55c003a20 con 0x7fb57015d690 2024-03-29T09:09:57.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.617+0000 7fb56cc00700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55006ac90 0x7fb55006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:57.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.617+0000 7fb56cc00700 1 -- 172.21.15.99:0/1365631972 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb55c088930 con 0x7fb57015d690 2024-03-29T09:09:57.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.618+0000 7fb574e00700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55006ac90 0x7fb55006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:57.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.618+0000 7fb574e00700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55006ac90 0x7fb55006d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5640090b0 tx=0x7fb56401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:57.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.622+0000 7fb56cc00700 1 -- 172.21.15.99:0/1365631972 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb55c053850 con 0x7fb57015d690 2024-03-29T09:09:57.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.825+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5700677a0 con 0x7fb57015d690 2024-03-29T09:09:57.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.827+0000 7fb56cc00700 1 -- 172.21.15.99:0/1365631972 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7fb55c056e70 con 0x7fb57015d690 2024-03-29T09:09:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:57.827 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:57.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55006ac90 msgr2=0x7fb55006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55006ac90 0x7fb55006d150 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5640090b0 tx=0x7fb56401c040 comp rx=0 tx=0).stop 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 msgr2=0x7fb57015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb57015b310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00c4e0 tx=0x7fb55c00c7f0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 shutdown_connections 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55006ac90 0x7fb55006d150 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57015da70 0x7fb57015a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5700d1740 0x7fb5700c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 --2- 172.21.15.99:0/1365631972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57015d690 0x7fb57015b310 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 >> 172.21.15.99:0/1365631972 conn(0x7fb57009c480 msgr2=0x7fb5700d3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 shutdown_connections 2024-03-29T09:09:57.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:57.829+0000 7fb577800700 1 -- 172.21.15.99:0/1365631972 wait complete. 2024-03-29T09:09:57.847 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:09:57.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:09:58.040 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.040+0000 7f8bff200700 1 Processor -- start 2024-03-29T09:09:58.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.041+0000 7f8bff200700 1 -- start start 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bff200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80b3cc0 0x7f8bf80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bff200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bf80d3380 0x7f8bf80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bff200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf80d3810 con 0x7f8bf8160770 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bff200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80b4140 con 0x7f8bf80d3380 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bff200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80d1520 con 0x7f8bf80b3cc0 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bf7400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80b3cc0 0x7f8bf80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bf7400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80b3cc0 0x7f8bf80d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:40456/0 (socket says 172.21.15.99:40456) 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bf7400700 1 -- 172.21.15.99:0/3218143905 learned_addr learned my addr 172.21.15.99:0/3218143905 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:58.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.042+0000 7f8bfd200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bf80d3380 0x7f8bf80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf7400700 1 -- 172.21.15.99:0/3218143905 >> v2:172.21.15.133:3300/0 conn(0x7f8bf80d3380 msgr2=0x7f8bf80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:58.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf7400700 1 --2- 172.21.15.99:0/3218143905 >> v2:172.21.15.133:3300/0 conn(0x7f8bf80d3380 0x7f8bf80ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf7400700 1 -- 172.21.15.99:0/3218143905 >> v1:172.21.15.133:6789/0 conn(0x7f8bf8160770 legacy=0x7f8bf80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf6a00700 1 -- 172.21.15.99:0/3218143905 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237092928 0 0) 0x7f8bf80d3810 con 0x7f8bf8160770 2024-03-29T09:09:58.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf7400700 1 -- 172.21.15.99:0/3218143905 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80d1800 con 0x7f8bf80b3cc0 2024-03-29T09:09:58.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf7400700 1 --2- 172.21.15.99:0/3218143905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80b3cc0 0x7f8bf80d2e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098f0 tx=0x7f8be4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd795c3bdf6afff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf6a00700 1 -- 172.21.15.99:0/3218143905 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011830 con 0x7f8bf80b3cc0 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf6a00700 1 -- 172.21.15.99:0/3218143905 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be4011990 con 0x7f8bf80b3cc0 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.043+0000 7f8bf6a00700 1 -- 172.21.15.99:0/3218143905 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4011d20 con 0x7f8bf80b3cc0 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 -- 172.21.15.99:0/3218143905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80b3cc0 msgr2=0x7f8bf80d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 --2- 172.21.15.99:0/3218143905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80b3cc0 0x7f8bf80d2e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098f0 tx=0x7f8be4009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 -- 172.21.15.99:0/3218143905 shutdown_connections 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 --2- 172.21.15.99:0/3218143905 >> v2:172.21.15.133:3300/0 conn(0x7f8bf80d3380 0x7f8bf80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 --2- 172.21.15.99:0/3218143905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80b3cc0 0x7f8bf80d2e40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 -- 172.21.15.99:0/3218143905 >> 172.21.15.99:0/3218143905 conn(0x7f8bf808afd0 msgr2=0x7f8bf809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 -- 172.21.15.99:0/3218143905 shutdown_connections 2024-03-29T09:09:58.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.044+0000 7f8bff200700 1 -- 172.21.15.99:0/3218143905 wait complete. 2024-03-29T09:09:58.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.046+0000 7f8bff200700 1 Processor -- start 2024-03-29T09:09:58.046 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.046+0000 7f8bff200700 1 -- start start 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bff200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bf80b3cc0 0x7f8bf80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bff200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80d3380 0x7f8bf809d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bff200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bf8160770 0x7f8bf80a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bff200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80cd470 con 0x7f8bf8160770 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bff200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80cbe90 con 0x7f8bf80b3cc0 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bff200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80cc170 con 0x7f8bf80d3380 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bfd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bf8160770 0x7f8bf80a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bf7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bf80b3cc0 0x7f8bf80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bfd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bf8160770 0x7f8bf80a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50682/0 (socket says 172.21.15.99:50682) 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bf7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bf80b3cc0 0x7f8bf80ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:37152/0 (socket says 172.21.15.99:37152) 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bfd200700 1 -- 172.21.15.99:0/4071304430 learned_addr learned my addr 172.21.15.99:0/4071304430 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:58.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.047+0000 7f8bf7400700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80d3380 0x7f8bf809d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.048+0000 7f8bfd200700 1 -- 172.21.15.99:0/4071304430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80d3380 msgr2=0x7f8bf809d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.048+0000 7f8bfd200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80d3380 0x7f8bf809d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.048+0000 7f8bfd200700 1 -- 172.21.15.99:0/4071304430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bf80b3cc0 msgr2=0x7f8bf80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.048+0000 7f8bfd200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bf80b3cc0 0x7f8bf80ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.048+0000 7f8bfd200700 1 -- 172.21.15.99:0/4071304430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4009550 con 0x7f8bf8160770 2024-03-29T09:09:58.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.048+0000 7f8bfd200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bf8160770 0x7f8bf80a07d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000efc0 tx=0x7f8bf000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.049+0000 7f8befe00700 1 -- 172.21.15.99:0/4071304430 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0009ea0 con 0x7f8bf8160770 2024-03-29T09:09:58.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.049+0000 7f8befe00700 1 -- 172.21.15.99:0/4071304430 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000f040 con 0x7f8bf8160770 2024-03-29T09:09:58.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.049+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf80cc450 con 0x7f8bf8160770 2024-03-29T09:09:58.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.049+0000 7f8befe00700 1 -- 172.21.15.99:0/4071304430 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000ced0 con 0x7f8bf8160770 2024-03-29T09:09:58.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.050+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf80cca10 con 0x7f8bf8160770 2024-03-29T09:09:58.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.051+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf80ccc00 con 0x7f8bf8160770 2024-03-29T09:09:58.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.051+0000 7f8befe00700 1 -- 172.21.15.99:0/4071304430 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8bf0014480 con 0x7f8bf8160770 2024-03-29T09:09:58.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.052+0000 7f8befe00700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8bd806ac90 0x7f8bd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.052+0000 7f8befe00700 1 -- 172.21.15.99:0/4071304430 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8bf0089f10 con 0x7f8bf8160770 2024-03-29T09:09:58.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.053+0000 7f8bf7e00700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8bd806ac90 0x7f8bd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.053+0000 7f8bf7e00700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8bd806ac90 0x7f8bd806d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8be8000f60 tx=0x7f8be8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.057+0000 7f8befe00700 1 -- 172.21.15.99:0/4071304430 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8bf0054d00 con 0x7f8bf8160770 2024-03-29T09:09:58.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.258+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bf80678b0 con 0x7f8bf8160770 2024-03-29T09:09:58.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.259+0000 7f8befe00700 1 -- 172.21.15.99:0/4071304430 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bf0018020 con 0x7f8bf8160770 2024-03-29T09:09:58.259 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:09:58.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8bd806ac90 msgr2=0x7f8bd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8bd806ac90 0x7f8bd806d150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8be8000f60 tx=0x7f8be8009040 comp rx=0 tx=0).stop 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bf8160770 msgr2=0x7f8bf80a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bf8160770 0x7f8bf80a07d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000efc0 tx=0x7f8bf000c3a0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 shutdown_connections 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8bd806ac90 0x7f8bd806d150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bf8160770 0x7f8bf80a07d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bf80b3cc0 0x7f8bf80ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 --2- 172.21.15.99:0/4071304430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bf80d3380 0x7f8bf809d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 >> 172.21.15.99:0/4071304430 conn(0x7f8bf808afd0 msgr2=0x7f8bf806c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 shutdown_connections 2024-03-29T09:09:58.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.261+0000 7f8bff200700 1 -- 172.21.15.99:0/4071304430 wait complete. 2024-03-29T09:09:58.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:09:58.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.473+0000 7fe5b4c00700 1 Processor -- start 2024-03-29T09:09:58.473 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.474+0000 7fe5b4c00700 1 -- start start 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5b4c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b0160770 0x7fe5b00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5b4c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe5b00d0d60 0x7fe5b00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5b4c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b00d3810 con 0x7fe5b00d3380 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5b4c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b00d1520 con 0x7fe5b00d0d60 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5b4c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00d1800 con 0x7fe5b0160770 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5ade00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b0160770 0x7fe5b00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5ade00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b0160770 0x7fe5b00b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:40484/0 (socket says 172.21.15.99:40484) 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5ade00700 1 -- 172.21.15.99:0/900929808 learned_addr learned my addr 172.21.15.99:0/900929808 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5ae800700 1 --1- 172.21.15.99:0/900929808 >> v1:172.21.15.133:6789/0 conn(0x7fe5b00d3380 0x7fe5b00cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:37418/0 (socket says 172.21.15.99:37418) 2024-03-29T09:09:58.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.475+0000 7fe5a7e00700 1 --2- 172.21.15.99:0/900929808 >> v2:172.21.15.133:3300/0 conn(0x7fe5b00d0d60 0x7fe5b00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ad400700 1 -- 172.21.15.99:0/900929808 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860822868 0 0) 0x7fe5b00d3810 con 0x7fe5b00d3380 2024-03-29T09:09:58.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ad400700 1 -- 172.21.15.99:0/900929808 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5940035b0 con 0x7fe5b00d3380 2024-03-29T09:09:58.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ade00700 1 -- 172.21.15.99:0/900929808 >> v2:172.21.15.133:3300/0 conn(0x7fe5b00d0d60 msgr2=0x7fe5b00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:58.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ade00700 1 --2- 172.21.15.99:0/900929808 >> v2:172.21.15.133:3300/0 conn(0x7fe5b00d0d60 0x7fe5b00d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ade00700 1 -- 172.21.15.99:0/900929808 >> v1:172.21.15.133:6789/0 conn(0x7fe5b00d3380 legacy=0x7fe5b00cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ade00700 1 -- 172.21.15.99:0/900929808 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00d1ae0 con 0x7fe5b0160770 2024-03-29T09:09:58.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ade00700 1 --2- 172.21.15.99:0/900929808 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b0160770 0x7fe5b00b3cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00098f0 tx=0x7fe5a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f7e73ae2ff7f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ad400700 1 -- 172.21.15.99:0/900929808 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0011830 con 0x7fe5b0160770 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ad400700 1 -- 172.21.15.99:0/900929808 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a0011990 con 0x7fe5b0160770 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.476+0000 7fe5ad400700 1 -- 172.21.15.99:0/900929808 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0011d20 con 0x7fe5b0160770 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 -- 172.21.15.99:0/900929808 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b0160770 msgr2=0x7fe5b00b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/900929808 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b0160770 0x7fe5b00b3cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00098f0 tx=0x7fe5a0009c00 comp rx=0 tx=0).stop 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 -- 172.21.15.99:0/900929808 shutdown_connections 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/900929808 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b0160770 0x7fe5b00b3cc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/900929808 >> v2:172.21.15.133:3300/0 conn(0x7fe5b00d0d60 0x7fe5b00d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 -- 172.21.15.99:0/900929808 >> 172.21.15.99:0/900929808 conn(0x7fe5b008afd0 msgr2=0x7fe5b009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 -- 172.21.15.99:0/900929808 shutdown_connections 2024-03-29T09:09:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.477+0000 7fe5b4c00700 1 -- 172.21.15.99:0/900929808 wait complete. 2024-03-29T09:09:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.479+0000 7fe5b4c00700 1 Processor -- start 2024-03-29T09:09:58.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.479+0000 7fe5b4c00700 1 -- start start 2024-03-29T09:09:58.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5b4c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5b00d0d60 0x7fe5b00b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5b4c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b00d3380 0x7fe5b00b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.481 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5b4c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0160770 0x7fe5b015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5b4c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b009d680 con 0x7fe5b0160770 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5b4c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00b6290 con 0x7fe5b00d0d60 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5b4c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b009e080 con 0x7fe5b00d3380 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5ae800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0160770 0x7fe5b015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5ae800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0160770 0x7fe5b015c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50690/0 (socket says 172.21.15.99:50690) 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5ae800700 1 -- 172.21.15.99:0/725288512 learned_addr learned my addr 172.21.15.99:0/725288512 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5ade00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5b00d0d60 0x7fe5b00b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.480+0000 7fe5a7e00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b00d3380 0x7fe5b00b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5ae800700 1 -- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b00d3380 msgr2=0x7fe5b00b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5ae800700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b00d3380 0x7fe5b00b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5ae800700 1 -- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5b00d0d60 msgr2=0x7fe5b00b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5ae800700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5b00d0d60 0x7fe5b00b8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5ae800700 1 -- 172.21.15.99:0/725288512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a0009550 con 0x7fe5b0160770 2024-03-29T09:09:58.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5ae800700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0160770 0x7fe5b015c830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800cc50 tx=0x7fe5a800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5a6000700 1 -- 172.21.15.99:0/725288512 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800aad0 con 0x7fe5b0160770 2024-03-29T09:09:58.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5a6000700 1 -- 172.21.15.99:0/725288512 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a800ac30 con 0x7fe5b0160770 2024-03-29T09:09:58.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00b6480 con 0x7fe5b0160770 2024-03-29T09:09:58.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5a6000700 1 -- 172.21.15.99:0/725288512 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8007070 con 0x7fe5b0160770 2024-03-29T09:09:58.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.481+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b00b6a40 con 0x7fe5b0160770 2024-03-29T09:09:58.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.482+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b00737f0 con 0x7fe5b0160770 2024-03-29T09:09:58.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.483+0000 7fe5a6000700 1 -- 172.21.15.99:0/725288512 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe5a80071d0 con 0x7fe5b0160770 2024-03-29T09:09:58.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.484+0000 7fe5a6000700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe58c06ac90 0x7fe58c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.484+0000 7fe5a6000700 1 -- 172.21.15.99:0/725288512 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe5a8089480 con 0x7fe5b0160770 2024-03-29T09:09:58.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.484+0000 7fe5ade00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe58c06ac90 0x7fe58c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.485+0000 7fe5ade00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe58c06ac90 0x7fe58c06d150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0014040 tx=0x7fe5a0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.488+0000 7fe5a6000700 1 -- 172.21.15.99:0/725288512 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe5a8054300 con 0x7fe5b0160770 2024-03-29T09:09:58.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.690+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5b00678b0 con 0x7fe5b0160770 2024-03-29T09:09:58.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.691+0000 7fe5a6000700 1 -- 172.21.15.99:0/725288512 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5a8057920 con 0x7fe5b0160770 2024-03-29T09:09:58.691 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:58.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe58c06ac90 msgr2=0x7fe58c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe58c06ac90 0x7fe58c06d150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0014040 tx=0x7fe5a0008030 comp rx=0 tx=0).stop 2024-03-29T09:09:58.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0160770 msgr2=0x7fe5b015c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0160770 0x7fe5b015c830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800cc50 tx=0x7fe5a800cf60 comp rx=0 tx=0).stop 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 shutdown_connections 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe58c06ac90 0x7fe58c06d150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe5b0160770 0x7fe5b015c830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5b00d0d60 0x7fe5b00b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 --2- 172.21.15.99:0/725288512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5b00d3380 0x7fe5b00b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 >> 172.21.15.99:0/725288512 conn(0x7fe5b008afd0 msgr2=0x7fe5b006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 shutdown_connections 2024-03-29T09:09:58.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.693+0000 7fe5b4c00700 1 -- 172.21.15.99:0/725288512 wait complete. 2024-03-29T09:09:58.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:09:58.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.904+0000 7f8bbd200700 1 Processor -- start 2024-03-29T09:09:58.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.905+0000 7f8bbd200700 1 -- start start 2024-03-29T09:09:58.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bbd200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8bb8160770 0x7f8bb80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.906 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bbd200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb80d3550 0x7f8bb80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bbd200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80d1520 con 0x7f8bb80d0d60 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bbd200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb80d1800 con 0x7f8bb8160770 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bbd200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80d1ae0 con 0x7f8bb80d3550 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bb5400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8bb80d0d60 0x7f8bb80d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60858/0 (socket says 172.21.15.99:60858) 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bb5400700 1 -- 172.21.15.99:0/1840995222 learned_addr learned my addr 172.21.15.99:0/1840995222 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bb5e00700 1 --2- 172.21.15.99:0/1840995222 >> v2:172.21.15.133:3300/0 conn(0x7f8bb8160770 0x7f8bb80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bb6800700 1 --2- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb80d3550 0x7f8bb80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bb4a00700 1 -- 172.21.15.99:0/1840995222 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021355357 0 0) 0x7f8bb80d1520 con 0x7f8bb80d0d60 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.906+0000 7f8bb4a00700 1 -- 172.21.15.99:0/1840995222 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b9c0035b0 con 0x7f8bb80d0d60 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb5e00700 1 -- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb80d3550 msgr2=0x7f8bb80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb5e00700 1 --2- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb80d3550 0x7f8bb80ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb4a00700 1 -- 172.21.15.99:0/1840995222 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858945592 0 0) 0x7f8b9c0035b0 con 0x7f8bb80d0d60 2024-03-29T09:09:58.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb5e00700 1 -- 172.21.15.99:0/1840995222 >> v1:172.21.15.99:6789/0 conn(0x7f8bb80d0d60 legacy=0x7f8bb80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb5e00700 1 -- 172.21.15.99:0/1840995222 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80d1dc0 con 0x7f8bb8160770 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb5e00700 1 --2- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb8160770 0x7f8bb80b3cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009840 tx=0x7f8ba8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b3e6988900d219e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb4a00700 1 -- 172.21.15.99:0/1840995222 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4001670 con 0x7f8bb8160770 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb4a00700 1 -- 172.21.15.99:0/1840995222 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8004030 con 0x7f8bb8160770 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.907+0000 7f8bb4a00700 1 -- 172.21.15.99:0/1840995222 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80043c0 con 0x7f8bb8160770 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 -- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb8160770 msgr2=0x7f8bb80b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb8160770 0x7f8bb80b3cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009840 tx=0x7f8ba8009b50 comp rx=0 tx=0).stop 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 -- 172.21.15.99:0/1840995222 shutdown_connections 2024-03-29T09:09:58.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb80d3550 0x7f8bb80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1840995222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb8160770 0x7f8bb80b3cc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 -- 172.21.15.99:0/1840995222 >> 172.21.15.99:0/1840995222 conn(0x7f8bb808afd0 msgr2=0x7f8bb809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:58.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 -- 172.21.15.99:0/1840995222 shutdown_connections 2024-03-29T09:09:58.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.908+0000 7f8bbd200700 1 -- 172.21.15.99:0/1840995222 wait complete. 2024-03-29T09:09:58.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.910+0000 7f8bbd200700 1 Processor -- start 2024-03-29T09:09:58.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.911+0000 7f8bbd200700 1 -- start start 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bbd200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb80d0d60 0x7f8bb809e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bbd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80d3550 0x7f8bb809cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bbd200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb8160770 0x7f8bb815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bbd200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb814d4f0 con 0x7f8bb80d3550 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bbd200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb814e070 con 0x7f8bb80d0d60 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bbd200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb815c8e0 con 0x7f8bb8160770 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80d3550 0x7f8bb809cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80d3550 0x7f8bb809cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50704/0 (socket says 172.21.15.99:50704) 2024-03-29T09:09:58.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 -- 172.21.15.99:0/1266133751 learned_addr learned my addr 172.21.15.99:0/1266133751 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:58.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5e00700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb80d0d60 0x7f8bb809e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb6800700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb8160770 0x7f8bb815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 -- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb8160770 msgr2=0x7f8bb815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb8160770 0x7f8bb815c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 -- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb80d0d60 msgr2=0x7f8bb809e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:58.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb80d0d60 0x7f8bb809e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb5400700 1 -- 172.21.15.99:0/1266133751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba80094a0 con 0x7f8bb80d3550 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.912+0000 7f8bb6800700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb8160770 0x7f8bb815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.913+0000 7f8bb5400700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80d3550 0x7f8bb809cf40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c7b0 tx=0x7f8ba400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.913+0000 7f8bae000700 1 -- 172.21.15.99:0/1266133751 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4004c80 con 0x7f8bb80d3550 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.913+0000 7f8bae000700 1 -- 172.21.15.99:0/1266133751 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4004de0 con 0x7f8bb80d3550 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.913+0000 7f8bae000700 1 -- 172.21.15.99:0/1266133751 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40072f0 con 0x7f8bb80d3550 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.913+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb814d740 con 0x7f8bb80d3550 2024-03-29T09:09:58.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.913+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb814dd00 con 0x7f8bb80d3550 2024-03-29T09:09:58.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.914+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb80678b0 con 0x7f8bb80d3550 2024-03-29T09:09:58.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.915+0000 7f8bae000700 1 -- 172.21.15.99:0/1266133751 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8ba4007450 con 0x7f8bb80d3550 2024-03-29T09:09:58.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.916+0000 7f8bae000700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8b9806ac90 0x7f8b9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:58.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.916+0000 7f8bae000700 1 -- 172.21.15.99:0/1266133751 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8ba4088810 con 0x7f8bb80d3550 2024-03-29T09:09:58.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.916+0000 7f8bb5e00700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8b9806ac90 0x7f8b9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:58.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.917+0000 7f8bb5e00700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8b9806ac90 0x7f8b9806d150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80090b0 tx=0x7f8ba8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:58.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:58.920+0000 7f8bae000700 1 -- 172.21.15.99:0/1266133751 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8ba4053600 con 0x7f8bb80d3550 2024-03-29T09:09:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.124+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8bb80b9050 con 0x7f8bb80d3550 2024-03-29T09:09:59.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.125+0000 7f8bae000700 1 -- 172.21.15.99:0/1266133751 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8ba4018020 con 0x7f8bb80d3550 2024-03-29T09:09:59.125 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8b9806ac90 msgr2=0x7f8b9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8b9806ac90 0x7f8b9806d150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80090b0 tx=0x7f8ba8008030 comp rx=0 tx=0).stop 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80d3550 msgr2=0x7f8bb809cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80d3550 0x7f8bb809cf40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c7b0 tx=0x7f8ba400cac0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 shutdown_connections 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8b9806ac90 0x7f8b9806d150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8bb8160770 0x7f8bb815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bb80d0d60 0x7f8bb809e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 --2- 172.21.15.99:0/1266133751 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8bb80d3550 0x7f8bb809cf40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 >> 172.21.15.99:0/1266133751 conn(0x7f8bb808afd0 msgr2=0x7f8bb815f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:59.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.127+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 shutdown_connections 2024-03-29T09:09:59.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.128+0000 7f8bbd200700 1 -- 172.21.15.99:0/1266133751 wait complete. 2024-03-29T09:09:59.145 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:09:59.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.337+0000 7f52f7800700 1 Processor -- start 2024-03-29T09:09:59.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.338+0000 7f52f7800700 1 -- start start 2024-03-29T09:09:59.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f7800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f52f00d1760 0x7f52f00d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f7800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f00d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f7800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f52f00d12b0 0x7f52f00b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f7800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f0161ae0 con 0x7f52f00d1760 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f7800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f0160f10 con 0x7f52f00d12b0 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f7800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00d2080 con 0x7f52f00cdcb0 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f4e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f52f00d1760 0x7f52f00d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f4e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f52f00d1760 0x7f52f00d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50714/0 (socket says 172.21.15.99:50714) 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f4e00700 1 -- 172.21.15.99:0/2693353664 learned_addr learned my addr 172.21.15.99:0/2693353664 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f5800700 1 --2- 172.21.15.99:0/2693353664 >> v2:172.21.15.133:3300/0 conn(0x7f52f00d12b0 0x7f52f00b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52efe00700 1 --2- 172.21.15.99:0/2693353664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f00d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f4e00700 1 -- 172.21.15.99:0/2693353664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 msgr2=0x7f52f00d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.339+0000 7f52f4e00700 1 --2- 172.21.15.99:0/2693353664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f00d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.340+0000 7f52f4e00700 1 -- 172.21.15.99:0/2693353664 >> v2:172.21.15.133:3300/0 conn(0x7f52f00d12b0 msgr2=0x7f52f00b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.340+0000 7f52f4e00700 1 --2- 172.21.15.99:0/2693353664 >> v2:172.21.15.133:3300/0 conn(0x7f52f00d12b0 0x7f52f00b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.340+0000 7f52f4e00700 1 -- 172.21.15.99:0/2693353664 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f01611b0 con 0x7f52f00d1760 2024-03-29T09:09:59.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.340+0000 7f52f4e00700 1 --2- 172.21.15.99:0/2693353664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d1760 0x7f52f00d1b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f52f00b5d70 tx=0x7f52e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a566c8c968779bf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:59.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.340+0000 7f52ef400700 1 -- 172.21.15.99:0/2693353664 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4011830 con 0x7f52f00d1760 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.340+0000 7f52ef400700 1 -- 172.21.15.99:0/2693353664 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e4011990 con 0x7f52f00d1760 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.340+0000 7f52ef400700 1 -- 172.21.15.99:0/2693353664 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4011d20 con 0x7f52f00d1760 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 -- 172.21.15.99:0/2693353664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d1760 msgr2=0x7f52f00d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 --2- 172.21.15.99:0/2693353664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d1760 0x7f52f00d1b40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f52f00b5d70 tx=0x7f52e4009bd0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 -- 172.21.15.99:0/2693353664 shutdown_connections 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 --2- 172.21.15.99:0/2693353664 >> v2:172.21.15.133:3300/0 conn(0x7f52f00d12b0 0x7f52f00b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 --2- 172.21.15.99:0/2693353664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d1760 0x7f52f00d1b40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 --2- 172.21.15.99:0/2693353664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f00d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 -- 172.21.15.99:0/2693353664 >> 172.21.15.99:0/2693353664 conn(0x7f52f008afd0 msgr2=0x7f52f009c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:59.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 -- 172.21.15.99:0/2693353664 shutdown_connections 2024-03-29T09:09:59.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.341+0000 7f52f7800700 1 -- 172.21.15.99:0/2693353664 wait complete. 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.342+0000 7f52f7800700 1 Processor -- start 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.343+0000 7f52f7800700 1 -- start start 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.343+0000 7f52f7800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f01f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52f7800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d12b0 0x7f52f01f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52f7800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52f00d1760 0x7f52f00c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52f7800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00b1ee0 con 0x7f52f00d12b0 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52f7800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f01f6f80 con 0x7f52f00d1760 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52f7800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00b1260 con 0x7f52f00cdcb0 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d12b0 0x7f52f01f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d12b0 0x7f52f01f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50720/0 (socket says 172.21.15.99:50720) 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 -- 172.21.15.99:0/1680740988 learned_addr learned my addr 172.21.15.99:0/1680740988 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52f4e00700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f01f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52f5800700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52f00d1760 0x7f52f00c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 -- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 msgr2=0x7f52f01f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f01f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 -- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52f00d1760 msgr2=0x7f52f00c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52f00d1760 0x7f52f00c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 -- 172.21.15.99:0/1680740988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009550 con 0x7f52f00d12b0 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.344+0000 7f52efe00700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d12b0 0x7f52f01f6a40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00e870 tx=0x7f52dc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.345+0000 7f52ecc00700 1 -- 172.21.15.99:0/1680740988 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc009590 con 0x7f52f00d12b0 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.345+0000 7f52ecc00700 1 -- 172.21.15.99:0/1680740988 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc00ee70 con 0x7f52f00d12b0 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.345+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f00b1540 con 0x7f52f00d12b0 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.345+0000 7f52ecc00700 1 -- 172.21.15.99:0/1680740988 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00ca50 con 0x7f52f00d12b0 2024-03-29T09:09:59.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.345+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f00b1b00 con 0x7f52f00d12b0 2024-03-29T09:09:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.346+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f00678b0 con 0x7f52f00d12b0 2024-03-29T09:09:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.347+0000 7f52ecc00700 1 -- 172.21.15.99:0/1680740988 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f52dc00cbb0 con 0x7f52f00d12b0 2024-03-29T09:09:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.348+0000 7f52ecc00700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f52d006ac90 0x7f52d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.348+0000 7f52ecc00700 1 -- 172.21.15.99:0/1680740988 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f52dc0896d0 con 0x7f52f00d12b0 2024-03-29T09:09:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.348+0000 7f52f4e00700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f52d006ac90 0x7f52d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.348+0000 7f52f4e00700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f52d006ac90 0x7f52d006d150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52e4014040 tx=0x7f52e4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:59.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.352+0000 7f52ecc00700 1 -- 172.21.15.99:0/1680740988 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f52dc0544c0 con 0x7f52f00d12b0 2024-03-29T09:09:59.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.553+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f52f00b2db0 con 0x7f52f00d12b0 2024-03-29T09:09:59.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.554+0000 7f52ecc00700 1 -- 172.21.15.99:0/1680740988 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f52dc00ce60 con 0x7f52f00d12b0 2024-03-29T09:09:59.554 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f52d006ac90 msgr2=0x7f52d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f52d006ac90 0x7f52d006d150 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f52e4014040 tx=0x7f52e4008030 comp rx=0 tx=0).stop 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d12b0 msgr2=0x7f52f01f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d12b0 0x7f52f01f6a40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00e870 tx=0x7f52dc00ec30 comp rx=0 tx=0).stop 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 shutdown_connections 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f52d006ac90 0x7f52d006d150 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52f00d1760 0x7f52f00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52f00cdcb0 0x7f52f01f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 --2- 172.21.15.99:0/1680740988 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52f00d12b0 0x7f52f01f6a40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.556+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 >> 172.21.15.99:0/1680740988 conn(0x7f52f008afd0 msgr2=0x7f52f006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.557+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 shutdown_connections 2024-03-29T09:09:59.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.557+0000 7f52f7800700 1 -- 172.21.15.99:0/1680740988 wait complete. 2024-03-29T09:09:59.576 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:09:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.765+0000 7fe061600700 1 Processor -- start 2024-03-29T09:09:59.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.765+0000 7fe061600700 1 -- start start 2024-03-29T09:09:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.766+0000 7fe061600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 0x7fe05c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.766+0000 7fe061600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05c0cddc0 con 0x7fe05c0d1b20 2024-03-29T09:09:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.766+0000 7fe061600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05c0ce0a0 con 0x7fe05c1608e0 2024-03-29T09:09:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.766+0000 7fe061600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c0ce380 con 0x7fe05c160cc0 2024-03-29T09:09:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.766+0000 7fe05a000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe05c0d1b20 0x7fe05c0afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60864/0 (socket says 172.21.15.99:60864) 2024-03-29T09:09:59.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.766+0000 7fe05a000700 1 -- 172.21.15.99:0/796040094 learned_addr learned my addr 172.21.15.99:0/796040094 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:09:59.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2945828919 0 0) 0x7fe05c0cddc0 con 0x7fe05c0d1b20 2024-03-29T09:09:59.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0400035b0 con 0x7fe05c0d1b20 2024-03-29T09:09:59.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe05b400700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 0x7fe05c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909558378 0 0) 0x7fe0400035b0 con 0x7fe05c0d1b20 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 msgr2=0x7fe05c0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 0x7fe05c0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 >> v1:172.21.15.133:6789/0 conn(0x7fe05c1608e0 legacy=0x7fe05c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c0ce660 con 0x7fe05c0d1b20 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fe048001670 con 0x7fe05c0d1b20 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 >> v1:172.21.15.99:6789/0 conn(0x7fe05c0d1b20 legacy=0x7fe05c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe040003a30 0x7fe040003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 0x7fe040004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040004d90 0x7fe040005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c0ce380 con 0x7fe040003a30 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04000bc30 con 0x7fe040004d90 2024-03-29T09:09:59.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04000bdc0 con 0x7fe05c160cc0 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe05aa00700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe040003a30 0x7fe040003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe05a000700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 0x7fe040004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.767+0000 7fe05b400700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040004d90 0x7fe040005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe05aa00700 1 -- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 msgr2=0x7fe040004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe05aa00700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c160cc0 0x7fe040004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe05aa00700 1 -- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040004d90 msgr2=0x7fe040005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe05aa00700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040004d90 0x7fe040005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe05aa00700 1 -- 172.21.15.99:0/796040094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400035b0 con 0x7fe040003a30 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe05aa00700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe040003a30 0x7fe040003e50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe04c001830 tx=0x7fe040005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb27b9276d773c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0012f0 con 0x7fe040003a30 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c0052f0 con 0x7fe040003a30 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.768+0000 7fe059600700 1 -- 172.21.15.99:0/796040094 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c006860 con 0x7fe040003a30 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.769+0000 7fe061600700 1 -- 172.21.15.99:0/796040094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe040003a30 msgr2=0x7fe040003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.769+0000 7fe061600700 1 --2- 172.21.15.99:0/796040094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe040003a30 0x7fe040003e50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe04c001830 tx=0x7fe040005880 comp rx=0 tx=0).stop 2024-03-29T09:09:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.769+0000 7fe05b400700 1 -- 172.21.15.99:0/796040094 reap_dead start 2024-03-29T09:09:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.769+0000 7fe061600700 1 -- 172.21.15.99:0/796040094 shutdown_connections 2024-03-29T09:09:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.769+0000 7fe061600700 1 -- 172.21.15.99:0/796040094 >> 172.21.15.99:0/796040094 conn(0x7fe05c09c490 msgr2=0x7fe05c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.769+0000 7fe061600700 1 -- 172.21.15.99:0/796040094 shutdown_connections 2024-03-29T09:09:59.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.769+0000 7fe061600700 1 -- 172.21.15.99:0/796040094 wait complete. 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.771+0000 7fe061600700 1 Processor -- start 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.771+0000 7fe061600700 1 -- start start 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe061600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040003a30 0x7fe05c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe061600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1f6f60 0x7fe05c15c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe061600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c15caf0 0x7fe05c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe061600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c15c190 con 0x7fe05c1f6f60 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe061600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c0c9580 con 0x7fe040003a30 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe061600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05c1f73e0 con 0x7fe05c15caf0 2024-03-29T09:09:59.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1f6f60 0x7fe05c15c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1f6f60 0x7fe05c15c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50728/0 (socket says 172.21.15.99:50728) 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 -- 172.21.15.99:0/4094363627 learned_addr learned my addr 172.21.15.99:0/4094363627 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05aa00700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040003a30 0x7fe05c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 -- 172.21.15.99:0/4094363627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c15caf0 msgr2=0x7fe05c0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c15caf0 0x7fe05c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 -- 172.21.15.99:0/4094363627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040003a30 msgr2=0x7fe05c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040003a30 0x7fe05c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.772+0000 7fe05a000700 1 -- 172.21.15.99:0/4094363627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c15b7f0 con 0x7fe05c1f6f60 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.773+0000 7fe05a000700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1f6f60 0x7fe05c15c5b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe0480072c0 tx=0x7fe048007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.773+0000 7fe052a00700 1 -- 172.21.15.99:0/4094363627 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048001450 con 0x7fe05c1f6f60 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.773+0000 7fe052a00700 1 -- 172.21.15.99:0/4094363627 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800f770 con 0x7fe05c1f6f60 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.773+0000 7fe052a00700 1 -- 172.21.15.99:0/4094363627 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800fb20 con 0x7fe05c1f6f60 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.773+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05c15bad0 con 0x7fe05c1f6f60 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.773+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05c15b4a0 con 0x7fe05c1f6f60 2024-03-29T09:09:59.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.774+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05c0677a0 con 0x7fe05c1f6f60 2024-03-29T09:09:59.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.775+0000 7fe052a00700 1 -- 172.21.15.99:0/4094363627 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe04800fc80 con 0x7fe05c1f6f60 2024-03-29T09:09:59.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.776+0000 7fe052a00700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe03c06ac90 0x7fe03c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:09:59.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.776+0000 7fe052a00700 1 -- 172.21.15.99:0/4094363627 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe048089630 con 0x7fe05c1f6f60 2024-03-29T09:09:59.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.776+0000 7fe05aa00700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe03c06ac90 0x7fe03c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:09:59.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.777+0000 7fe05aa00700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe03c06ac90 0x7fe03c06d150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe04c014040 tx=0x7fe04c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:09:59.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.780+0000 7fe052a00700 1 -- 172.21.15.99:0/4094363627 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe048054560 con 0x7fe05c1f6f60 2024-03-29T09:09:59.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.985+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe05c0d0700 con 0x7fe05c1f6f60 2024-03-29T09:09:59.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.987+0000 7fe052a00700 1 -- 172.21.15.99:0/4094363627 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7fe048057b80 con 0x7fe05c1f6f60 2024-03-29T09:09:59.987 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:09:59.987 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:09:59.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe03c06ac90 msgr2=0x7fe03c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe03c06ac90 0x7fe03c06d150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe04c014040 tx=0x7fe04c009030 comp rx=0 tx=0).stop 2024-03-29T09:09:59.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1f6f60 msgr2=0x7fe05c15c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:09:59.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1f6f60 0x7fe05c15c5b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe0480072c0 tx=0x7fe048007680 comp rx=0 tx=0).stop 2024-03-29T09:09:59.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 shutdown_connections 2024-03-29T09:09:59.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe03c06ac90 0x7fe03c06d150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c15caf0 0x7fe05c0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe040003a30 0x7fe05c1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 --2- 172.21.15.99:0/4094363627 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe05c1f6f60 0x7fe05c15c5b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:09:59.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 >> 172.21.15.99:0/4094363627 conn(0x7fe05c09c490 msgr2=0x7fe05c06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:09:59.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 shutdown_connections 2024-03-29T09:09:59.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:09:59.989+0000 7fe061600700 1 -- 172.21.15.99:0/4094363627 wait complete. 2024-03-29T09:10:00.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:00.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:00.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.203+0000 7f6c3f600700 1 Processor -- start 2024-03-29T09:10:00.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.203+0000 7f6c3f600700 1 -- start start 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.204+0000 7f6c3f600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c380adc40 0x7f6c3815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.204+0000 7f6c3f600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380ae020 0x7f6c38155070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.204+0000 7f6c3f600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3806c580 con 0x7f6c38072120 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.204+0000 7f6c3f600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3815c7a0 con 0x7f6c380adc40 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.204+0000 7f6c3f600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3806c8d0 con 0x7f6c380ae020 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c3cc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6c380adc40 0x7f6c3815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c37e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6c38072120 0x7f6c3806ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60868/0 (socket says 172.21.15.99:60868) 2024-03-29T09:10:00.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c37e00700 1 -- 172.21.15.99:0/2068643757 learned_addr learned my addr 172.21.15.99:0/2068643757 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c3cc00700 1 -- 172.21.15.99:0/2068643757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380ae020 msgr2=0x7f6c38155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c3cc00700 1 --2- 172.21.15.99:0/2068643757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380ae020 0x7f6c38155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c3cc00700 1 -- 172.21.15.99:0/2068643757 >> v1:172.21.15.99:6789/0 conn(0x7f6c38072120 legacy=0x7f6c3806ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c3cc00700 1 -- 172.21.15.99:0/2068643757 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380abf10 con 0x7f6c380adc40 2024-03-29T09:10:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c37400700 1 -- 172.21.15.99:0/2068643757 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930656247 0 0) 0x7f6c3806c580 con 0x7f6c38072120 2024-03-29T09:10:00.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.205+0000 7f6c3cc00700 1 --2- 172.21.15.99:0/2068643757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c380adc40 0x7f6c3815c1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009840 tx=0x7f6c2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fe9c1d397c934b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c37400700 1 -- 172.21.15.99:0/2068643757 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c011830 con 0x7f6c380adc40 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c37400700 1 -- 172.21.15.99:0/2068643757 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c011990 con 0x7f6c380adc40 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c37400700 1 -- 172.21.15.99:0/2068643757 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c011d20 con 0x7f6c380adc40 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c3f600700 1 -- 172.21.15.99:0/2068643757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c380adc40 msgr2=0x7f6c3815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c3f600700 1 --2- 172.21.15.99:0/2068643757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c380adc40 0x7f6c3815c1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009840 tx=0x7f6c2c009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c3f600700 1 -- 172.21.15.99:0/2068643757 shutdown_connections 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c3f600700 1 --2- 172.21.15.99:0/2068643757 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380ae020 0x7f6c38155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c3f600700 1 --2- 172.21.15.99:0/2068643757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c380adc40 0x7f6c3815c1c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.206+0000 7f6c3f600700 1 -- 172.21.15.99:0/2068643757 >> 172.21.15.99:0/2068643757 conn(0x7f6c38066540 msgr2=0x7f6c38066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:00.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.207+0000 7f6c3f600700 1 -- 172.21.15.99:0/2068643757 shutdown_connections 2024-03-29T09:10:00.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.207+0000 7f6c3f600700 1 -- 172.21.15.99:0/2068643757 wait complete. 2024-03-29T09:10:00.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.209+0000 7f6c3f600700 1 Processor -- start 2024-03-29T09:10:00.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.209+0000 7f6c3f600700 1 -- start start 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.210+0000 7f6c3f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c38072120 0x7f6c381543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.210+0000 7f6c3f600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380adc40 0x7f6c38158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.210+0000 7f6c3f600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c380ae020 0x7f6c38158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.210+0000 7f6c3f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3809a330 con 0x7f6c38072120 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.210+0000 7f6c3f600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38154920 con 0x7f6c380ae020 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.210+0000 7f6c3f600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38099a40 con 0x7f6c380adc40 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c38072120 0x7f6c381543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c38072120 0x7f6c381543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50734/0 (socket says 172.21.15.99:50734) 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 -- 172.21.15.99:0/387941184 learned_addr learned my addr 172.21.15.99:0/387941184 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c37e00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380adc40 0x7f6c38158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3d600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c380ae020 0x7f6c38158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 -- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380adc40 msgr2=0x7f6c38158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380adc40 0x7f6c38158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 -- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c380ae020 msgr2=0x7f6c38158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c380ae020 0x7f6c38158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 -- 172.21.15.99:0/387941184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0094a0 con 0x7f6c38072120 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3d600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c380ae020 0x7f6c38158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c37e00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380adc40 0x7f6c38158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c3cc00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c38072120 0x7f6c381543e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c3806e930 tx=0x7f6c2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.211+0000 7f6c34c00700 1 -- 172.21.15.99:0/387941184 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0036a0 con 0x7f6c38072120 2024-03-29T09:10:00.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.212+0000 7f6c34c00700 1 -- 172.21.15.99:0/387941184 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c01f070 con 0x7f6c38072120 2024-03-29T09:10:00.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.212+0000 7f6c34c00700 1 -- 172.21.15.99:0/387941184 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c003a40 con 0x7f6c38072120 2024-03-29T09:10:00.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.212+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c38157210 con 0x7f6c38072120 2024-03-29T09:10:00.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.212+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c38099fc0 con 0x7f6c38072120 2024-03-29T09:10:00.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.213+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3809a180 con 0x7f6c38072120 2024-03-29T09:10:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.214+0000 7f6c34c00700 1 -- 172.21.15.99:0/387941184 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6c2c003ba0 con 0x7f6c38072120 2024-03-29T09:10:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.214+0000 7f6c34c00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6c1806ac90 0x7f6c1806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.215+0000 7f6c34c00700 1 -- 172.21.15.99:0/387941184 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6c2c021070 con 0x7f6c38072120 2024-03-29T09:10:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.215+0000 7f6c37e00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6c1806ac90 0x7f6c1806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.215+0000 7f6c37e00700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6c1806ac90 0x7f6c1806d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c3815a6e0 tx=0x7f6c24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:00.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.219+0000 7f6c34c00700 1 -- 172.21.15.99:0/387941184 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6c2c056010 con 0x7f6c38072120 2024-03-29T09:10:00.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.421+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c380678b0 con 0x7f6c38072120 2024-03-29T09:10:00.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.421+0000 7f6c34c00700 1 -- 172.21.15.99:0/387941184 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c2c02b0a0 con 0x7f6c38072120 2024-03-29T09:10:00.421 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6c1806ac90 msgr2=0x7f6c1806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6c1806ac90 0x7f6c1806d150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c3815a6e0 tx=0x7f6c24009040 comp rx=0 tx=0).stop 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c38072120 msgr2=0x7f6c381543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c38072120 0x7f6c381543e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6c3806e930 tx=0x7f6c2c004750 comp rx=0 tx=0).stop 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 shutdown_connections 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6c1806ac90 0x7f6c1806d150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c380ae020 0x7f6c38158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6c38072120 0x7f6c381543e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 --2- 172.21.15.99:0/387941184 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6c380adc40 0x7f6c38158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 >> 172.21.15.99:0/387941184 conn(0x7f6c38066540 msgr2=0x7f6c38070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:00.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 shutdown_connections 2024-03-29T09:10:00.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.424+0000 7f6c3f600700 1 -- 172.21.15.99:0/387941184 wait complete. 2024-03-29T09:10:00.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:00.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.635+0000 7fb89fe00700 1 Processor -- start 2024-03-29T09:10:00.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.636+0000 7fb89fe00700 1 -- start start 2024-03-29T09:10:00.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89fe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb898160770 0x7fb8980b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89fe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb8980d0d60 0x7fb8980d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89fe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8980d3380 0x7fb8980cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89fe00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8980d3810 con 0x7fb8980d0d60 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89fe00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8980d1520 con 0x7fb898160770 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89fe00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980d1800 con 0x7fb8980d3380 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb8980d0d60 0x7fb8980d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb8980d0d60 0x7fb8980d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50736/0 (socket says 172.21.15.99:50736) 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 -- 172.21.15.99:0/4229496838 learned_addr learned my addr 172.21.15.99:0/4229496838 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89d400700 1 --2- 172.21.15.99:0/4229496838 >> v2:172.21.15.133:3300/0 conn(0x7fb898160770 0x7fb8980b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89de00700 1 --2- 172.21.15.99:0/4229496838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8980d3380 0x7fb8980cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 -- 172.21.15.99:0/4229496838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8980d3380 msgr2=0x7fb8980cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 --2- 172.21.15.99:0/4229496838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8980d3380 0x7fb8980cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 -- 172.21.15.99:0/4229496838 >> v2:172.21.15.133:3300/0 conn(0x7fb898160770 msgr2=0x7fb8980b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 --2- 172.21.15.99:0/4229496838 >> v2:172.21.15.133:3300/0 conn(0x7fb898160770 0x7fb8980b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.637+0000 7fb89ca00700 1 -- 172.21.15.99:0/4229496838 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980d1ae0 con 0x7fb8980d0d60 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.638+0000 7fb89de00700 1 --2- 172.21.15.99:0/4229496838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8980d3380 0x7fb8980cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.638+0000 7fb89ca00700 1 --2- 172.21.15.99:0/4229496838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d0d60 0x7fb8980d2e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb8980b5d20 tx=0x7fb884009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac11164b7b5bd203 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.638+0000 7fb897e00700 1 -- 172.21.15.99:0/4229496838 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400b870 con 0x7fb8980d0d60 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.638+0000 7fb897e00700 1 -- 172.21.15.99:0/4229496838 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88400b9d0 con 0x7fb8980d0d60 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.638+0000 7fb897e00700 1 -- 172.21.15.99:0/4229496838 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400bd60 con 0x7fb8980d0d60 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.638+0000 7fb89fe00700 1 -- 172.21.15.99:0/4229496838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d0d60 msgr2=0x7fb8980d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.638+0000 7fb89fe00700 1 --2- 172.21.15.99:0/4229496838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d0d60 0x7fb8980d2e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb8980b5d20 tx=0x7fb884009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.639+0000 7fb89fe00700 1 -- 172.21.15.99:0/4229496838 shutdown_connections 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.639+0000 7fb89fe00700 1 --2- 172.21.15.99:0/4229496838 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8980d3380 0x7fb8980cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.639+0000 7fb89fe00700 1 --2- 172.21.15.99:0/4229496838 >> v2:172.21.15.133:3300/0 conn(0x7fb898160770 0x7fb8980b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.639+0000 7fb89fe00700 1 --2- 172.21.15.99:0/4229496838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d0d60 0x7fb8980d2e40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.639+0000 7fb89fe00700 1 -- 172.21.15.99:0/4229496838 >> 172.21.15.99:0/4229496838 conn(0x7fb89808afd0 msgr2=0x7fb89809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.639+0000 7fb89fe00700 1 -- 172.21.15.99:0/4229496838 shutdown_connections 2024-03-29T09:10:00.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.639+0000 7fb89fe00700 1 -- 172.21.15.99:0/4229496838 wait complete. 2024-03-29T09:10:00.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.641+0000 7fb89fe00700 1 Processor -- start 2024-03-29T09:10:00.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.641+0000 7fb89fe00700 1 -- start start 2024-03-29T09:10:00.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89fe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8980d0d60 0x7fb8981f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d3380 0x7fb898153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89fe00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb898160770 0x7fb8980b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89fe00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8981f2940 con 0x7fb8980d3380 2024-03-29T09:10:00.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89fe00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8980b8cc0 con 0x7fb8980d0d60 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89fe00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8981f60d0 con 0x7fb898160770 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d3380 0x7fb898153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d3380 0x7fb898153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50742/0 (socket says 172.21.15.99:50742) 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 -- 172.21.15.99:0/1477910522 learned_addr learned my addr 172.21.15.99:0/1477910522 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89d400700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8980d0d60 0x7fb8981f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89de00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb898160770 0x7fb8980b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 -- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb898160770 msgr2=0x7fb8980b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb898160770 0x7fb8980b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 -- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8980d0d60 msgr2=0x7fb8981f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8980d0d60 0x7fb8981f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89ca00700 1 -- 172.21.15.99:0/1477910522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb884009550 con 0x7fb8980d3380 2024-03-29T09:10:00.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.642+0000 7fb89de00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb898160770 0x7fb8980b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.643+0000 7fb89ca00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d3380 0x7fb898153fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb884000c00 tx=0x7fb884004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.643+0000 7fb895600700 1 -- 172.21.15.99:0/1477910522 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884004540 con 0x7fb8980d3380 2024-03-29T09:10:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.643+0000 7fb895600700 1 -- 172.21.15.99:0/1477910522 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8840036a0 con 0x7fb8980d3380 2024-03-29T09:10:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.643+0000 7fb895600700 1 -- 172.21.15.99:0/1477910522 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884003a50 con 0x7fb8980d3380 2024-03-29T09:10:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.643+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8981f63b0 con 0x7fb8980d3380 2024-03-29T09:10:00.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.643+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8980b8fa0 con 0x7fb8980d3380 2024-03-29T09:10:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.644+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8980678b0 con 0x7fb8980d3380 2024-03-29T09:10:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.645+0000 7fb895600700 1 -- 172.21.15.99:0/1477910522 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb884021070 con 0x7fb8980d3380 2024-03-29T09:10:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.646+0000 7fb895600700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb87806ac90 0x7fb87806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.646+0000 7fb895600700 1 -- 172.21.15.99:0/1477910522 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb88408acb0 con 0x7fb8980d3380 2024-03-29T09:10:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.646+0000 7fb89d400700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb87806ac90 0x7fb87806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:00.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.647+0000 7fb89d400700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb87806ac90 0x7fb87806d150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8980b5f50 tx=0x7fb88c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:00.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.650+0000 7fb895600700 1 -- 172.21.15.99:0/1477910522 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb884055cf0 con 0x7fb8980d3380 2024-03-29T09:10:00.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.852+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb89806c950 con 0x7fb8980d3380 2024-03-29T09:10:00.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.853+0000 7fb895600700 1 -- 172.21.15.99:0/1477910522 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8840150a0 con 0x7fb8980d3380 2024-03-29T09:10:00.853 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb87806ac90 msgr2=0x7fb87806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb87806ac90 0x7fb87806d150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8980b5f50 tx=0x7fb88c008040 comp rx=0 tx=0).stop 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d3380 msgr2=0x7fb898153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d3380 0x7fb898153fc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb884000c00 tx=0x7fb884004750 comp rx=0 tx=0).stop 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 shutdown_connections 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb87806ac90 0x7fb87806d150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb898160770 0x7fb8980b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8980d0d60 0x7fb8981f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.855+0000 7fb89fe00700 1 --2- 172.21.15.99:0/1477910522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8980d3380 0x7fb898153fc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.856+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 >> 172.21.15.99:0/1477910522 conn(0x7fb89808afd0 msgr2=0x7fb89806ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:00.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.856+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 shutdown_connections 2024-03-29T09:10:00.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:00.856+0000 7fb89fe00700 1 -- 172.21.15.99:0/1477910522 wait complete. 2024-03-29T09:10:00.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:01.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.069+0000 7fdab3a00700 1 Processor -- start 2024-03-29T09:10:01.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.069+0000 7fdab3a00700 1 -- start start 2024-03-29T09:10:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.070+0000 7fdab3a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdaac0b3cc0 0x7fdaac0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.070+0000 7fdab3a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdaac0d3380 0x7fdaac0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.070+0000 7fdab3a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaac0d3810 con 0x7fdaac160770 2024-03-29T09:10:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.070+0000 7fdab3a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaac0b4140 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.070+0000 7fdab3a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaac0d1520 con 0x7fdaac0d3380 2024-03-29T09:10:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdaac0b3cc0 0x7fdaac0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdaac0b3cc0 0x7fdaac0d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50754/0 (socket says 172.21.15.99:50754) 2024-03-29T09:10:01.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 -- 172.21.15.99:0/2521282601 learned_addr learned my addr 172.21.15.99:0/2521282601 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:01.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdab1a00700 1 --2- 172.21.15.99:0/2521282601 >> v2:172.21.15.133:3300/0 conn(0x7fdaac0d3380 0x7fdaac0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 -- 172.21.15.99:0/2521282601 >> v2:172.21.15.133:3300/0 conn(0x7fdaac0d3380 msgr2=0x7fdaac0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:01.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 --2- 172.21.15.99:0/2521282601 >> v2:172.21.15.133:3300/0 conn(0x7fdaac0d3380 0x7fdaac0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 -- 172.21.15.99:0/2521282601 >> v1:172.21.15.133:6789/0 conn(0x7fdaac160770 legacy=0x7fdaac0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 -- 172.21.15.99:0/2521282601 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac0d1800 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.071+0000 7fdaabe00700 1 --2- 172.21.15.99:0/2521282601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0d2e40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0b3230 tx=0x7fda98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f244af9f2ddf8a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdaab400700 1 -- 172.21.15.99:0/2521282601 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9800b870 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdaab400700 1 -- 172.21.15.99:0/2521282601 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9800b9d0 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdaab400700 1 -- 172.21.15.99:0/2521282601 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9800bd60 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 -- 172.21.15.99:0/2521282601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 msgr2=0x7fdaac0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 --2- 172.21.15.99:0/2521282601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0d2e40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0b3230 tx=0x7fda98009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 -- 172.21.15.99:0/2521282601 shutdown_connections 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 --2- 172.21.15.99:0/2521282601 >> v2:172.21.15.133:3300/0 conn(0x7fdaac0d3380 0x7fdaac0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 --2- 172.21.15.99:0/2521282601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0d2e40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 -- 172.21.15.99:0/2521282601 >> 172.21.15.99:0/2521282601 conn(0x7fdaac08afd0 msgr2=0x7fdaac09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:01.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 -- 172.21.15.99:0/2521282601 shutdown_connections 2024-03-29T09:10:01.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.072+0000 7fdab3a00700 1 -- 172.21.15.99:0/2521282601 wait complete. 2024-03-29T09:10:01.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.074+0000 7fdab3a00700 1 Processor -- start 2024-03-29T09:10:01.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.074+0000 7fdab3a00700 1 -- start start 2024-03-29T09:10:01.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab3a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab3a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac0d3380 0x7fdaac0b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab3a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdaac160770 0x7fdaac15c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab3a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac0b7ae0 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab3a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac15cc90 con 0x7fdaac0d3380 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab3a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac0b7d10 con 0x7fdaac160770 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab1000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab1000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50768/0 (socket says 172.21.15.99:50768) 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.075+0000 7fdab1000700 1 -- 172.21.15.99:0/3735025049 learned_addr learned my addr 172.21.15.99:0/3735025049 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdaabe00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac0d3380 0x7fdaac0b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab1a00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdaac160770 0x7fdaac15c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab1000700 1 -- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdaac160770 msgr2=0x7fdaac15c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab1000700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdaac160770 0x7fdaac15c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab1000700 1 -- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac0d3380 msgr2=0x7fdaac0b75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab1000700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac0d3380 0x7fdaac0b75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab1000700 1 -- 172.21.15.99:0/3735025049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda98009550 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab1000700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0cbe50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000c450 tx=0x7fdaa000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdaa8c00700 1 -- 172.21.15.99:0/3735025049 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00052d0 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdaa8c00700 1 -- 172.21.15.99:0/3735025049 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa0005430 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdaa8c00700 1 -- 172.21.15.99:0/3735025049 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00038c0 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac0b7ff0 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.076+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac0b85b0 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.078+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaac0678b0 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.078+0000 7fdaa8c00700 1 -- 172.21.15.99:0/3735025049 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdaa0003a20 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.079+0000 7fdaa8c00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fda8c06ac90 0x7fda8c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.079+0000 7fdaa8c00700 1 -- 172.21.15.99:0/3735025049 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdaa0088950 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.080+0000 7fdaabe00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fda8c06ac90 0x7fda8c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.080+0000 7fdaabe00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fda8c06ac90 0x7fda8c06d150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda98000c00 tx=0x7fda98008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.084+0000 7fdaa8c00700 1 -- 172.21.15.99:0/3735025049 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdaa0053870 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.288+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdaac0b3390 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.288+0000 7fdaa8c00700 1 -- 172.21.15.99:0/3735025049 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdaa0017020 con 0x7fdaac0b3cc0 2024-03-29T09:10:01.289 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:01.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fda8c06ac90 msgr2=0x7fda8c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fda8c06ac90 0x7fda8c06d150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda98000c00 tx=0x7fda98008030 comp rx=0 tx=0).stop 2024-03-29T09:10:01.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 msgr2=0x7fdaac0cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0cbe50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000c450 tx=0x7fdaa000c760 comp rx=0 tx=0).stop 2024-03-29T09:10:01.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 shutdown_connections 2024-03-29T09:10:01.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fda8c06ac90 0x7fda8c06d150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdaac160770 0x7fdaac15c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdaac0b3cc0 0x7fdaac0cbe50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 --2- 172.21.15.99:0/3735025049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdaac0d3380 0x7fdaac0b75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 >> 172.21.15.99:0/3735025049 conn(0x7fdaac08afd0 msgr2=0x7fdaac06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:01.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 shutdown_connections 2024-03-29T09:10:01.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.291+0000 7fdab3a00700 1 -- 172.21.15.99:0/3735025049 wait complete. 2024-03-29T09:10:01.309 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:01.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.508+0000 7f2d89000700 1 Processor -- start 2024-03-29T09:10:01.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.508+0000 7f2d89000700 1 -- start start 2024-03-29T09:10:01.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.509+0000 7f2d89000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d840b3cc0 0x7f2d840d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.509+0000 7f2d89000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840d3380 0x7f2d840ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.509+0000 7f2d89000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d840d3810 con 0x7f2d84160770 2024-03-29T09:10:01.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.509+0000 7f2d89000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d840b4140 con 0x7f2d840b3cc0 2024-03-29T09:10:01.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.509+0000 7f2d89000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840d1520 con 0x7f2d840d3380 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d82800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840d3380 0x7f2d840ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d81400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d840b3cc0 0x7f2d840d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d81e00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2d84160770 0x7f2d840d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:37450/0 (socket says 172.21.15.99:37450) 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d81e00700 1 -- 172.21.15.99:0/3623006228 learned_addr learned my addr 172.21.15.99:0/3623006228 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d82800700 1 -- 172.21.15.99:0/3623006228 >> v2:172.21.15.133:3300/0 conn(0x7f2d840b3cc0 msgr2=0x7f2d840d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d82800700 1 --2- 172.21.15.99:0/3623006228 >> v2:172.21.15.133:3300/0 conn(0x7f2d840b3cc0 0x7f2d840d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d82800700 1 -- 172.21.15.99:0/3623006228 >> v1:172.21.15.133:6789/0 conn(0x7f2d84160770 legacy=0x7f2d840d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d82800700 1 -- 172.21.15.99:0/3623006228 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840d1800 con 0x7f2d840d3380 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d80a00700 1 -- 172.21.15.99:0/3623006228 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272061667 0 0) 0x7f2d840d3810 con 0x7f2d84160770 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.510+0000 7f2d81400700 1 --2- 172.21.15.99:0/3623006228 >> v2:172.21.15.133:3300/0 conn(0x7f2d840b3cc0 0x7f2d840d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.511+0000 7f2d82800700 1 --2- 172.21.15.99:0/3623006228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840d3380 0x7f2d840ce080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009d90 tx=0x7f2d7c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0af98a40de1618b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.511+0000 7f2d80a00700 1 -- 172.21.15.99:0/3623006228 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00e070 con 0x7f2d840d3380 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.511+0000 7f2d80a00700 1 -- 172.21.15.99:0/3623006228 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c00bb20 con 0x7f2d840d3380 2024-03-29T09:10:01.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.511+0000 7f2d80a00700 1 -- 172.21.15.99:0/3623006228 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c012430 con 0x7f2d840d3380 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 -- 172.21.15.99:0/3623006228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840d3380 msgr2=0x7f2d840ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 --2- 172.21.15.99:0/3623006228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840d3380 0x7f2d840ce080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009d90 tx=0x7f2d7c00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 -- 172.21.15.99:0/3623006228 shutdown_connections 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 --2- 172.21.15.99:0/3623006228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840d3380 0x7f2d840ce080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 --2- 172.21.15.99:0/3623006228 >> v2:172.21.15.133:3300/0 conn(0x7f2d840b3cc0 0x7f2d840d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 -- 172.21.15.99:0/3623006228 >> 172.21.15.99:0/3623006228 conn(0x7f2d8408afd0 msgr2=0x7f2d8409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 -- 172.21.15.99:0/3623006228 shutdown_connections 2024-03-29T09:10:01.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.512+0000 7f2d89000700 1 -- 172.21.15.99:0/3623006228 wait complete. 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.513+0000 7f2d89000700 1 Processor -- start 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.514+0000 7f2d89000700 1 -- start start 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.514+0000 7f2d89000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840b3cc0 0x7f2d840c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.514+0000 7f2d89000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d840d3380 0x7f2d840b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.514+0000 7f2d89000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d84160770 0x7f2d840b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.514+0000 7f2d89000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840c9670 con 0x7f2d840d3380 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.514+0000 7f2d89000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840b1750 con 0x7f2d84160770 2024-03-29T09:10:01.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.514+0000 7f2d89000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8414af80 con 0x7f2d840b3cc0 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d840d3380 0x7f2d840b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d840d3380 0x7f2d840b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50772/0 (socket says 172.21.15.99:50772) 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 -- 172.21.15.99:0/349172447 learned_addr learned my addr 172.21.15.99:0/349172447 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81e00700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840b3cc0 0x7f2d840c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d82800700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d84160770 0x7f2d840b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 -- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840b3cc0 msgr2=0x7f2d840c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840b3cc0 0x7f2d840c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 -- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d84160770 msgr2=0x7f2d840b6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d84160770 0x7f2d840b6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 -- 172.21.15.99:0/349172447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700094a0 con 0x7f2d840d3380 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81e00700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840b3cc0 0x7f2d840c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.515+0000 7f2d81400700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d840d3380 0x7f2d840b5d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000ec10 tx=0x7f2d70009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.516+0000 7f2d7a000700 1 -- 172.21.15.99:0/349172447 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000c940 con 0x7f2d840d3380 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.516+0000 7f2d7a000700 1 -- 172.21.15.99:0/349172447 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000caa0 con 0x7f2d840d3380 2024-03-29T09:10:01.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.516+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c009a70 con 0x7f2d840d3380 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.516+0000 7f2d7a000700 1 -- 172.21.15.99:0/349172447 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000ce10 con 0x7f2d840d3380 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.516+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d840b1190 con 0x7f2d840d3380 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.517+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d840678b0 con 0x7f2d840d3380 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.518+0000 7f2d7a000700 1 -- 172.21.15.99:0/349172447 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2d70013470 con 0x7f2d840d3380 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.519+0000 7f2d7a000700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d6406ac90 0x7f2d6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.519+0000 7f2d7a000700 1 -- 172.21.15.99:0/349172447 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2d700898c0 con 0x7f2d840d3380 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.519+0000 7f2d81e00700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d6406ac90 0x7f2d6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.520+0000 7f2d81e00700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d6406ac90 0x7f2d6406d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000f60 tx=0x7f2d74007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.523+0000 7f2d7a000700 1 -- 172.21.15.99:0/349172447 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2d70054740 con 0x7f2d840d3380 2024-03-29T09:10:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.724+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2d840d37f0 con 0x7f2d840d3380 2024-03-29T09:10:01.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.725+0000 7f2d7a000700 1 -- 172.21.15.99:0/349172447 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2d70057d60 con 0x7f2d840d3380 2024-03-29T09:10:01.725 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:01.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d6406ac90 msgr2=0x7f2d6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d6406ac90 0x7f2d6406d150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000f60 tx=0x7f2d74007d30 comp rx=0 tx=0).stop 2024-03-29T09:10:01.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d840d3380 msgr2=0x7f2d840b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d840d3380 0x7f2d840b5d50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000ec10 tx=0x7f2d70009ec0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 shutdown_connections 2024-03-29T09:10:01.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d6406ac90 0x7f2d6406d150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d84160770 0x7f2d840b6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d840b3cc0 0x7f2d840c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 --2- 172.21.15.99:0/349172447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d840d3380 0x7f2d840b5d50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 >> 172.21.15.99:0/349172447 conn(0x7f2d8408afd0 msgr2=0x7f2d840b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 shutdown_connections 2024-03-29T09:10:01.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.727+0000 7f2d89000700 1 -- 172.21.15.99:0/349172447 wait complete. 2024-03-29T09:10:01.762 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2024-03-29T09:10:01.762 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-03-29T09:10:01.762 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-03-29T09:10:01.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.953+0000 7f11e1a00700 1 Processor -- start 2024-03-29T09:10:01.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.953+0000 7f11e1a00700 1 -- start start 2024-03-29T09:10:01.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11e1a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f11dc15d690 0x7f11dc0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11e1a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f11dc15da70 0x7f11dc0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11e1a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc0d1740 0x7f11dc0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11e1a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11dc0d2110 con 0x7f11dc15d690 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11e1a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11dc0afde0 con 0x7f11dc15da70 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11e1a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc0cf660 con 0x7f11dc0d1740 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11daa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f11dc15d690 0x7f11dc0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11daa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f11dc15d690 0x7f11dc0cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50774/0 (socket says 172.21.15.99:50774) 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11daa00700 1 -- 172.21.15.99:0/2258776864 learned_addr learned my addr 172.21.15.99:0/2258776864 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11d3e00700 1 --2- 172.21.15.99:0/2258776864 >> v2:172.21.15.133:3300/0 conn(0x7f11dc15da70 0x7f11dc0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.954+0000 7f11db400700 1 --2- 172.21.15.99:0/2258776864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc0d1740 0x7f11dc0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11daa00700 1 -- 172.21.15.99:0/2258776864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc0d1740 msgr2=0x7f11dc0d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11daa00700 1 --2- 172.21.15.99:0/2258776864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc0d1740 0x7f11dc0d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11daa00700 1 -- 172.21.15.99:0/2258776864 >> v2:172.21.15.133:3300/0 conn(0x7f11dc15da70 msgr2=0x7f11dc0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:01.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11daa00700 1 --2- 172.21.15.99:0/2258776864 >> v2:172.21.15.133:3300/0 conn(0x7f11dc15da70 0x7f11dc0cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11daa00700 1 -- 172.21.15.99:0/2258776864 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0d01d0 con 0x7f11dc15d690 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11daa00700 1 --2- 172.21.15.99:0/2258776864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc0cfb60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0d0c50 tx=0x7f11cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914d19fef499614c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11da000700 1 -- 172.21.15.99:0/2258776864 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc011830 con 0x7f11dc15d690 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11da000700 1 -- 172.21.15.99:0/2258776864 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc011990 con 0x7f11dc15d690 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.955+0000 7f11da000700 1 -- 172.21.15.99:0/2258776864 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc011d20 con 0x7f11dc15d690 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 -- 172.21.15.99:0/2258776864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 msgr2=0x7f11dc0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 --2- 172.21.15.99:0/2258776864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc0cfb60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0d0c50 tx=0x7f11cc009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 -- 172.21.15.99:0/2258776864 shutdown_connections 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 --2- 172.21.15.99:0/2258776864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc0d1740 0x7f11dc0d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 --2- 172.21.15.99:0/2258776864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc0cfb60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 --2- 172.21.15.99:0/2258776864 >> v2:172.21.15.133:3300/0 conn(0x7f11dc15da70 0x7f11dc0cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 -- 172.21.15.99:0/2258776864 >> 172.21.15.99:0/2258776864 conn(0x7f11dc09c480 msgr2=0x7f11dc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 -- 172.21.15.99:0/2258776864 shutdown_connections 2024-03-29T09:10:01.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.956+0000 7f11e1a00700 1 -- 172.21.15.99:0/2258776864 wait complete. 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.958+0000 7f11e1a00700 1 Processor -- start 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.958+0000 7f11e1a00700 1 -- start start 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11e1a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11dc0d1740 0x7f11dc151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11e1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11e1a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc15da70 0x7f11dc15fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11e1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc1524d0 con 0x7f11dc15d690 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11e1a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc152ee0 con 0x7f11dc0d1740 2024-03-29T09:10:01.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11e1a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc15dee0 con 0x7f11dc15da70 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50786/0 (socket says 172.21.15.99:50786) 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 -- 172.21.15.99:0/3084067194 learned_addr learned my addr 172.21.15.99:0/3084067194 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11daa00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11dc0d1740 0x7f11dc151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 -- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc15da70 msgr2=0x7f11dc15fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11db400700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc15da70 0x7f11dc15fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc15da70 0x7f11dc15fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 -- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11dc0d1740 msgr2=0x7f11dc151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11dc0d1740 0x7f11dc151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11d3e00700 1 -- 172.21.15.99:0/3084067194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc009550 con 0x7f11dc15d690 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11daa00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11dc0d1740 0x7f11dc151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.959+0000 7f11db400700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc15da70 0x7f11dc15fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.960+0000 7f11d3e00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc150090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11c400e870 tx=0x7f11c400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.960+0000 7f11d2a00700 1 -- 172.21.15.99:0/3084067194 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4009590 con 0x7f11dc15d690 2024-03-29T09:10:01.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.960+0000 7f11d2a00700 1 -- 172.21.15.99:0/3084067194 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c400ee70 con 0x7f11dc15d690 2024-03-29T09:10:01.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.960+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11dc15e1c0 con 0x7f11dc15d690 2024-03-29T09:10:01.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.960+0000 7f11d2a00700 1 -- 172.21.15.99:0/3084067194 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c400ca50 con 0x7f11dc15d690 2024-03-29T09:10:01.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.960+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11dc15e780 con 0x7f11dc15d690 2024-03-29T09:10:01.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.961+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11dc15edd0 con 0x7f11dc15d690 2024-03-29T09:10:01.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.962+0000 7f11d2a00700 1 -- 172.21.15.99:0/3084067194 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11c400cbb0 con 0x7f11dc15d690 2024-03-29T09:10:01.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.963+0000 7f11d2a00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11b806ac90 0x7f11b806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:01.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.963+0000 7f11d2a00700 1 -- 172.21.15.99:0/3084067194 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f11c4089720 con 0x7f11dc15d690 2024-03-29T09:10:01.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.963+0000 7f11daa00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11b806ac90 0x7f11b806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:01.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.964+0000 7f11daa00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11b806ac90 0x7f11b806d150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009c10 tx=0x7f11cc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:01.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:01.967+0000 7f11d2a00700 1 -- 172.21.15.99:0/3084067194 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11c4054510 con 0x7f11dc15d690 2024-03-29T09:10:02.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:02.249+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f11dc0c6880 con 0x7f11dc15d690 2024-03-29T09:10:03.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.072+0000 7f11d2a00700 1 -- 172.21.15.99:0/3084067194 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v77) v1 ==== 150+0+0 (secure 0 0 0) 0x7f11c400ce60 con 0x7f11dc15d690 2024-03-29T09:10:03.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11b806ac90 msgr2=0x7f11b806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11b806ac90 0x7f11b806d150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009c10 tx=0x7f11cc008030 comp rx=0 tx=0).stop 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 msgr2=0x7f11dc150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc150090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11c400e870 tx=0x7f11c400ec30 comp rx=0 tx=0).stop 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 shutdown_connections 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11b806ac90 0x7f11b806d150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11dc15da70 0x7f11dc15fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11dc0d1740 0x7f11dc151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 --2- 172.21.15.99:0/3084067194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11dc15d690 0x7f11dc150090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 >> 172.21.15.99:0/3084067194 conn(0x7f11dc09c480 msgr2=0x7f11dc0ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 shutdown_connections 2024-03-29T09:10:03.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.076+0000 7f11e1a00700 1 -- 172.21.15.99:0/3084067194 wait complete. 2024-03-29T09:10:03.097 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T09:10:03.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.293+0000 7fe3c3400700 1 Processor -- start 2024-03-29T09:10:03.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.293+0000 7fe3c3400700 1 -- start start 2024-03-29T09:10:03.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.294+0000 7fe3c3400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe3bc0b3cc0 0x7fe3bc0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:03.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.294+0000 7fe3c3400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc0d3810 con 0x7fe3bc160770 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.294+0000 7fe3c3400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc0d1520 con 0x7fe3bc0d3380 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.294+0000 7fe3c3400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc0b4140 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.294+0000 7fe3c0a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe3bc160770 0x7fe3bc0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60870/0 (socket says 172.21.15.99:60870) 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3c0a00700 1 -- 172.21.15.99:0/3252124660 learned_addr learned my addr 172.21.15.99:0/3252124660 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bbe00700 1 --2- 172.21.15.99:0/3252124660 >> v2:172.21.15.133:3300/0 conn(0x7fe3bc0b3cc0 0x7fe3bc0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401463962 0 0) 0x7fe3bc0d3810 con 0x7fe3bc160770 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3a40035b0 con 0x7fe3bc160770 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371159138 0 0) 0x7fe3bc0d1520 con 0x7fe3bc0d3380 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3bc0d3810 con 0x7fe3bc0d3380 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3478706548 0 0) 0x7fe3a40035b0 con 0x7fe3bc160770 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bbe00700 1 -- 172.21.15.99:0/3252124660 >> v1:172.21.15.133:6789/0 conn(0x7fe3bc0d3380 legacy=0x7fe3bc0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bbe00700 1 -- 172.21.15.99:0/3252124660 >> v1:172.21.15.99:6789/0 conn(0x7fe3bc160770 legacy=0x7fe3bc0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.295+0000 7fe3bbe00700 1 -- 172.21.15.99:0/3252124660 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc0d1800 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.296+0000 7fe3bbe00700 1 --2- 172.21.15.99:0/3252124660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc0d2e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009840 tx=0x7fe3a8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=829bc35af6005ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:03.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.296+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0001670 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.296+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a8004030 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.296+0000 7fe3bb400700 1 -- 172.21.15.99:0/3252124660 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a80043c0 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.296+0000 7fe3c3400700 1 -- 172.21.15.99:0/3252124660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0b3cc0 msgr2=0x7fe3bc0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.296+0000 7fe3c3400700 1 --2- 172.21.15.99:0/3252124660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc0d2e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009840 tx=0x7fe3a8009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.297+0000 7fe3c3400700 1 -- 172.21.15.99:0/3252124660 shutdown_connections 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.297+0000 7fe3c3400700 1 --2- 172.21.15.99:0/3252124660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc0d2e40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.297+0000 7fe3c3400700 1 -- 172.21.15.99:0/3252124660 >> 172.21.15.99:0/3252124660 conn(0x7fe3bc08afd0 msgr2=0x7fe3bc09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.297+0000 7fe3c3400700 1 -- 172.21.15.99:0/3252124660 shutdown_connections 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.297+0000 7fe3c3400700 1 -- 172.21.15.99:0/3252124660 wait complete. 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.298+0000 7fe3c3400700 1 Processor -- start 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.298+0000 7fe3c3400700 1 -- start start 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.299+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.299+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0d3380 0x7fe3bc0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:03.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.299+0000 7fe3c3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3bc160770 0x7fe3bc06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.299+0000 7fe3c3400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc14eeb0 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.299+0000 7fe3c3400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0caab0 con 0x7fe3bc0d3380 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.299+0000 7fe3c3400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0cad90 con 0x7fe3bc160770 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc14e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50796/0 (socket says 172.21.15.99:50796) 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 -- 172.21.15.99:0/962132231 learned_addr learned my addr 172.21.15.99:0/962132231 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3bbe00700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0d3380 0x7fe3bc0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c1400700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3bc160770 0x7fe3bc06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 -- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3bc160770 msgr2=0x7fe3bc06bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3bc160770 0x7fe3bc06bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 -- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0d3380 msgr2=0x7fe3bc0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0d3380 0x7fe3bc0ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 -- 172.21.15.99:0/962132231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80094a0 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c0a00700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc14e970 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c520 tx=0x7fe3b000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3b8c00700 1 -- 172.21.15.99:0/962132231 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0004c80 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3b8c00700 1 -- 172.21.15.99:0/962132231 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b0004de0 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc0cb070 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.300+0000 7fe3b8c00700 1 -- 172.21.15.99:0/962132231 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0007310 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.301+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc0ca7a0 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.302+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc0678b0 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.302+0000 7fe3b8c00700 1 -- 172.21.15.99:0/962132231 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe3b0007470 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.304+0000 7fe3b8c00700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe39c06ac90 0x7fe39c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.304+0000 7fe3b8c00700 1 -- 172.21.15.99:0/962132231 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe3b0089440 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.304+0000 7fe3bbe00700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe39c06ac90 0x7fe39c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.304+0000 7fe3bbe00700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe39c06ac90 0x7fe39c06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80090b0 tx=0x7fe3a8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:03.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.309+0000 7fe3b8c00700 1 -- 172.21.15.99:0/962132231 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe3b0054230 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.513+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe3bc0c9850 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.517+0000 7fe3b8c00700 1 -- 172.21.15.99:0/962132231 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe3b0057850 con 0x7fe3bc0b3cc0 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe39c06ac90 msgr2=0x7fe39c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe39c06ac90 0x7fe39c06d150 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3a80090b0 tx=0x7fe3a8008030 comp rx=0 tx=0).stop 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc0b3cc0 msgr2=0x7fe3bc14e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc14e970 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c520 tx=0x7fe3b000c830 comp rx=0 tx=0).stop 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 shutdown_connections 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe39c06ac90 0x7fe39c06d150 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3bc160770 0x7fe3bc06bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3bc0b3cc0 0x7fe3bc14e970 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 --2- 172.21.15.99:0/962132231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3bc0d3380 0x7fe3bc0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:03.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 >> 172.21.15.99:0/962132231 conn(0x7fe3bc08afd0 msgr2=0x7fe3bc06ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:03.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 shutdown_connections 2024-03-29T09:10:03.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:03.520+0000 7fe3c3400700 1 -- 172.21.15.99:0/962132231 wait complete. 2024-03-29T09:10:03.539 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T09:10:03.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:10:03.620 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:10:03.718 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T09:10:03.718 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:03.719 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:03.760 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:03.760 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:03.843 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.073+0000 7f87ca800700 1 Processor -- start 2024-03-29T09:10:04.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.073+0000 7f87ca800700 1 -- start start 2024-03-29T09:10:04.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.074+0000 7f87ca800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87c4145d70 0x7f87c4161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.074+0000 7f87ca800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 0x7f87c406ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.074+0000 7f87ca800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c40b3990 con 0x7f87c4146150 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.074+0000 7f87ca800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c406b1a0 con 0x7f87c4145d70 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.074+0000 7f87ca800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40b3c70 con 0x7f87c406a7d0 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.074+0000 7f87c2a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f87c4146150 0x7f87c40b2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60886/0 (socket says 172.21.15.99:60886) 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.074+0000 7f87c2a00700 1 -- 172.21.15.99:0/4225570490 learned_addr learned my addr 172.21.15.99:0/4225570490 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c3400700 1 --2- 172.21.15.99:0/4225570490 >> v2:172.21.15.133:3300/0 conn(0x7f87c4145d70 0x7f87c4161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c3e00700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 0x7f87c406ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583361979 0 0) 0x7f87c40b3990 con 0x7f87c4146150 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a80035b0 con 0x7f87c4146150 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 358699302 0 0) 0x7f87a80035b0 con 0x7f87c4146150 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 msgr2=0x7f87c406ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 0x7f87c406ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 >> v2:172.21.15.133:3300/0 conn(0x7f87c4145d70 msgr2=0x7f87c4161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 --2- 172.21.15.99:0/4225570490 >> v2:172.21.15.133:3300/0 conn(0x7f87c4145d70 0x7f87c4161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c4160f40 con 0x7f87c4146150 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f87b0001670 con 0x7f87c4146150 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c3400700 1 --2- 172.21.15.99:0/4225570490 >> v2:172.21.15.133:3300/0 conn(0x7f87c4145d70 0x7f87c4161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 >> v1:172.21.15.99:6789/0 conn(0x7f87c4146150 legacy=0x7f87c40b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a8003a00 0x7f87a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.075+0000 7f87c2000700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87a8004850 0x7f87a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c2000700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 0x7f87a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c406b1a0 con 0x7f87a8003a00 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40b3c70 con 0x7f87a8004850 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a800ba10 con 0x7f87c406a7d0 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3400700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a8003a00 0x7f87a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c2a00700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87a8004850 0x7f87a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3e00700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 0x7f87a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3400700 1 -- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 msgr2=0x7f87a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3400700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87c406a7d0 0x7f87a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3400700 1 -- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87a8004850 msgr2=0x7f87a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3400700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87a8004850 0x7f87a8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3400700 1 -- 172.21.15.99:0/4225570490 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80035b0 con 0x7f87a8003a00 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c3400700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a8003a00 0x7f87a8003e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009d30 tx=0x7f87a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73da5f873c03b2c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4009e80 con 0x7f87a8003a00 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.076+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b400e9a0 con 0x7f87a8003a00 2024-03-29T09:10:04.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87c2000700 1 -- 172.21.15.99:0/4225570490 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b400ed30 con 0x7f87a8003a00 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87ca800700 1 -- 172.21.15.99:0/4225570490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a8003a00 msgr2=0x7f87a8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87ca800700 1 --2- 172.21.15.99:0/4225570490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87a8003a00 0x7f87a8003e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009d30 tx=0x7f87a8005880 comp rx=0 tx=0).stop 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87c3e00700 1 -- 172.21.15.99:0/4225570490 reap_dead start 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87ca800700 1 -- 172.21.15.99:0/4225570490 shutdown_connections 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87ca800700 1 -- 172.21.15.99:0/4225570490 >> 172.21.15.99:0/4225570490 conn(0x7f87c4066df0 msgr2=0x7f87c406d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87ca800700 1 -- 172.21.15.99:0/4225570490 shutdown_connections 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.077+0000 7f87ca800700 1 -- 172.21.15.99:0/4225570490 wait complete. 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.079+0000 7f87ca800700 1 Processor -- start 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.079+0000 7f87ca800700 1 -- start start 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87ca800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87a8003a00 0x7f87c415ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87ca800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c413d860 0x7f87c4144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87ca800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87c413dce0 0x7f87c41451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87ca800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41f7250 con 0x7f87c413dce0 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87ca800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41f6380 con 0x7f87c413d860 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87ca800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41f65d0 con 0x7f87a8003a00 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87c413dce0 0x7f87c41451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87c413dce0 0x7f87c41451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50810/0 (socket says 172.21.15.99:50810) 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87c3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87a8003a00 0x7f87c415ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87c3e00700 1 -- 172.21.15.99:0/1045847222 learned_addr learned my addr 172.21.15.99:0/1045847222 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.080+0000 7f87c2a00700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c413d860 0x7f87c4144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87c3e00700 1 -- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87a8003a00 msgr2=0x7f87c415ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87c3e00700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87a8003a00 0x7f87c415ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87c3e00700 1 -- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c413d860 msgr2=0x7f87c4144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87c3e00700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c413d860 0x7f87c4144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87c3e00700 1 -- 172.21.15.99:0/1045847222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c41f68b0 con 0x7f87c413dce0 2024-03-29T09:10:04.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87c2a00700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c413d860 0x7f87c4144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87c3e00700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87c413dce0 0x7f87c41451c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87c40cfc40 tx=0x7f87bc0111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87bb400700 1 -- 172.21.15.99:0/1045847222 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc00e070 con 0x7f87c413dce0 2024-03-29T09:10:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87bb400700 1 -- 172.21.15.99:0/1045847222 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc003cf0 con 0x7f87c413dce0 2024-03-29T09:10:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c41f6b90 con 0x7f87c413dce0 2024-03-29T09:10:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87bb400700 1 -- 172.21.15.99:0/1045847222 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc011850 con 0x7f87c413dce0 2024-03-29T09:10:04.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.081+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c415b3d0 con 0x7f87c413dce0 2024-03-29T09:10:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.082+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c40b4b50 con 0x7f87c413dce0 2024-03-29T09:10:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.083+0000 7f87bb400700 1 -- 172.21.15.99:0/1045847222 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f87bc0119b0 con 0x7f87c413dce0 2024-03-29T09:10:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.084+0000 7f87bb400700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f87a406ac90 0x7f87a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.084+0000 7f87bb400700 1 -- 172.21.15.99:0/1045847222 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f87bc089a80 con 0x7f87c413dce0 2024-03-29T09:10:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.084+0000 7f87c3400700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f87a406ac90 0x7f87a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.085+0000 7f87c3400700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f87a406ac90 0x7f87a406d150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b401f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.088+0000 7f87bb400700 1 -- 172.21.15.99:0/1045847222 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f87bc054900 con 0x7f87c413dce0 2024-03-29T09:10:04.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.292+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87c40677a0 con 0x7f87c413dce0 2024-03-29T09:10:04.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.294+0000 7f87bb400700 1 -- 172.21.15.99:0/1045847222 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f87bc057f20 con 0x7f87c413dce0 2024-03-29T09:10:04.294 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:04.294 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:04.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f87a406ac90 msgr2=0x7f87a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f87a406ac90 0x7f87a406d150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b401f040 comp rx=0 tx=0).stop 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87c413dce0 msgr2=0x7f87c41451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87c413dce0 0x7f87c41451c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f87c40cfc40 tx=0x7f87bc0111d0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 shutdown_connections 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f87a406ac90 0x7f87a406d150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87c413dce0 0x7f87c41451c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f87a8003a00 0x7f87c415ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 --2- 172.21.15.99:0/1045847222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87c413d860 0x7f87c4144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.296+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 >> 172.21.15.99:0/1045847222 conn(0x7f87c4066df0 msgr2=0x7f87c4071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.297+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 shutdown_connections 2024-03-29T09:10:04.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.297+0000 7f87ca800700 1 -- 172.21.15.99:0/1045847222 wait complete. 2024-03-29T09:10:04.315 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:04.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:04.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.508+0000 7feb7a800700 1 Processor -- start 2024-03-29T09:10:04.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.508+0000 7feb7a800700 1 -- start start 2024-03-29T09:10:04.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.509+0000 7feb7a800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 0x7feb740d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.509+0000 7feb7a800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb740d3810 con 0x7feb74160770 2024-03-29T09:10:04.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.509+0000 7feb7a800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb740d1520 con 0x7feb740d3380 2024-03-29T09:10:04.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.509+0000 7feb7a800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb740b4140 con 0x7feb740b3cc0 2024-03-29T09:10:04.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.509+0000 7feb73400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7feb74160770 0x7feb740d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60898/0 (socket says 172.21.15.99:60898) 2024-03-29T09:10:04.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb73400700 1 -- 172.21.15.99:0/1180462628 learned_addr learned my addr 172.21.15.99:0/1180462628 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:04.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72a00700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 0x7feb740d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932122766 0 0) 0x7feb740d3810 con 0x7feb74160770 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb580035b0 con 0x7feb74160770 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914945901 0 0) 0x7feb740d1520 con 0x7feb740d3380 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb740d3810 con 0x7feb740d3380 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1816934213 0 0) 0x7feb580035b0 con 0x7feb74160770 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 msgr2=0x7feb740d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 0x7feb740d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 >> v1:172.21.15.133:6789/0 conn(0x7feb740d3380 legacy=0x7feb740ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb740d1800 con 0x7feb74160770 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7feb64001670 con 0x7feb74160770 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72a00700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 0x7feb740d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 >> v1:172.21.15.99:6789/0 conn(0x7feb74160770 legacy=0x7feb740d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.510+0000 7feb72000700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb58003bc0 0x7feb58003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb72000700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 0x7feb58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb72000700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb58004d90 0x7feb58005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5800bac0 con 0x7feb58003bc0 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5800bc00 con 0x7feb58004d90 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5800bd40 con 0x7feb740b3cc0 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb73400700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb58003bc0 0x7feb58003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb73400700 1 -- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 msgr2=0x7feb58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb73400700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb740b3cc0 0x7feb58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb73400700 1 -- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb58004d90 msgr2=0x7feb58005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb73400700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb58004d90 0x7feb58005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb73400700 1 -- 172.21.15.99:0/1180462628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600094a0 con 0x7feb58003bc0 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb73400700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb58003bc0 0x7feb58003fc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feb6400c270 tx=0x7feb58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bcca6f7829ecbb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.511+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6400cd70 con 0x7feb58003bc0 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb64005960 con 0x7feb58003bc0 2024-03-29T09:10:04.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb72000700 1 -- 172.21.15.99:0/1180462628 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64005d10 con 0x7feb58003bc0 2024-03-29T09:10:04.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb7a800700 1 -- 172.21.15.99:0/1180462628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb58003bc0 msgr2=0x7feb58003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb7a800700 1 --2- 172.21.15.99:0/1180462628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb58003bc0 0x7feb58003fc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feb6400c270 tx=0x7feb58005880 comp rx=0 tx=0).stop 2024-03-29T09:10:04.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb73e00700 1 -- 172.21.15.99:0/1180462628 reap_dead start 2024-03-29T09:10:04.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb7a800700 1 -- 172.21.15.99:0/1180462628 shutdown_connections 2024-03-29T09:10:04.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb7a800700 1 -- 172.21.15.99:0/1180462628 >> 172.21.15.99:0/1180462628 conn(0x7feb7408afd0 msgr2=0x7feb7409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:04.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb7a800700 1 -- 172.21.15.99:0/1180462628 shutdown_connections 2024-03-29T09:10:04.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.512+0000 7feb7a800700 1 -- 172.21.15.99:0/1180462628 wait complete. 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.514+0000 7feb7a800700 1 Processor -- start 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.515+0000 7feb7a800700 1 -- start start 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb7a800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb58003bc0 0x7feb740b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb7a800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb741f6030 0x7feb741f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb7a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb740ce1f0 0x7feb740a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb7a800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb740b4ed0 con 0x7feb740ce1f0 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb7a800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb741f6880 con 0x7feb741f6030 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb7a800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb741f6b60 con 0x7feb58003bc0 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb740ce1f0 0x7feb740a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb73400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb58003bc0 0x7feb740b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb740ce1f0 0x7feb740a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50840/0 (socket says 172.21.15.99:50840) 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb73e00700 1 -- 172.21.15.99:0/2222765447 learned_addr learned my addr 172.21.15.99:0/2222765447 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.516+0000 7feb72a00700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb741f6030 0x7feb741f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb73e00700 1 -- 172.21.15.99:0/2222765447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb58003bc0 msgr2=0x7feb740b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb73e00700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb58003bc0 0x7feb740b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb73e00700 1 -- 172.21.15.99:0/2222765447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb741f6030 msgr2=0x7feb741f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb73e00700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb741f6030 0x7feb741f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb73e00700 1 -- 172.21.15.99:0/2222765447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb741f6e40 con 0x7feb740ce1f0 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb73e00700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb740ce1f0 0x7feb740a0500 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb7406b4d0 tx=0x7feb6c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb6b400700 1 -- 172.21.15.99:0/2222765447 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00a520 con 0x7feb740ce1f0 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb6b400700 1 -- 172.21.15.99:0/2222765447 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c00a680 con 0x7feb740ce1f0 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb6b400700 1 -- 172.21.15.99:0/2222765447 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00aa10 con 0x7feb740ce1f0 2024-03-29T09:10:04.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb740b5170 con 0x7feb740ce1f0 2024-03-29T09:10:04.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.517+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb741f64b0 con 0x7feb740ce1f0 2024-03-29T09:10:04.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.518+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb740b4440 con 0x7feb740ce1f0 2024-03-29T09:10:04.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.519+0000 7feb6b400700 1 -- 172.21.15.99:0/2222765447 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7feb6c00ab70 con 0x7feb740ce1f0 2024-03-29T09:10:04.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.520+0000 7feb6b400700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7feb5406ac90 0x7feb5406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.520+0000 7feb6b400700 1 -- 172.21.15.99:0/2222765447 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7feb6c0887d0 con 0x7feb740ce1f0 2024-03-29T09:10:04.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.520+0000 7feb73400700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7feb5406ac90 0x7feb5406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.521+0000 7feb73400700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7feb5406ac90 0x7feb5406d150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb64007040 tx=0x7feb64007f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.524+0000 7feb6b400700 1 -- 172.21.15.99:0/2222765447 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7feb6c0535c0 con 0x7feb740ce1f0 2024-03-29T09:10:04.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.727+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb741f6fd0 con 0x7feb740ce1f0 2024-03-29T09:10:04.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.728+0000 7feb6b400700 1 -- 172.21.15.99:0/2222765447 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb6c056be0 con 0x7feb740ce1f0 2024-03-29T09:10:04.728 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:04.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7feb5406ac90 msgr2=0x7feb5406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7feb5406ac90 0x7feb5406d150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb64007040 tx=0x7feb64007f90 comp rx=0 tx=0).stop 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb740ce1f0 msgr2=0x7feb740a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb740ce1f0 0x7feb740a0500 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feb7406b4d0 tx=0x7feb6c00ce70 comp rx=0 tx=0).stop 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 shutdown_connections 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7feb5406ac90 0x7feb5406d150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb740ce1f0 0x7feb740a0500 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb58003bc0 0x7feb740b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 --2- 172.21.15.99:0/2222765447 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb741f6030 0x7feb741f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.730+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 >> 172.21.15.99:0/2222765447 conn(0x7feb7408afd0 msgr2=0x7feb7406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:04.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.731+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 shutdown_connections 2024-03-29T09:10:04.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.731+0000 7feb7a800700 1 -- 172.21.15.99:0/2222765447 wait complete. 2024-03-29T09:10:04.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:04.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.941+0000 7f61fb200700 1 Processor -- start 2024-03-29T09:10:04.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.942+0000 7f61fb200700 1 -- start start 2024-03-29T09:10:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61fb200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40b3cc0 0x7f61f40d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61fb200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61f40d3380 0x7f61f40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61fb200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40d3810 con 0x7f61f4160770 2024-03-29T09:10:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61fb200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f40b4140 con 0x7f61f40d3380 2024-03-29T09:10:04.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61fb200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40d1520 con 0x7f61f40b3cc0 2024-03-29T09:10:04.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61f3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40b3cc0 0x7f61f40d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61f3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40b3cc0 0x7f61f40d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:40704/0 (socket says 172.21.15.99:40704) 2024-03-29T09:10:04.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61f3400700 1 -- 172.21.15.99:0/880767370 learned_addr learned my addr 172.21.15.99:0/880767370 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:04.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.943+0000 7f61f9200700 1 --2- 172.21.15.99:0/880767370 >> v2:172.21.15.133:3300/0 conn(0x7f61f40d3380 0x7f61f40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f2a00700 1 -- 172.21.15.99:0/880767370 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217124626 0 0) 0x7f61f40d3810 con 0x7f61f4160770 2024-03-29T09:10:04.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f2a00700 1 -- 172.21.15.99:0/880767370 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d80035b0 con 0x7f61f4160770 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f3400700 1 -- 172.21.15.99:0/880767370 >> v2:172.21.15.133:3300/0 conn(0x7f61f40d3380 msgr2=0x7f61f40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f3400700 1 --2- 172.21.15.99:0/880767370 >> v2:172.21.15.133:3300/0 conn(0x7f61f40d3380 0x7f61f40ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f3400700 1 -- 172.21.15.99:0/880767370 >> v1:172.21.15.133:6789/0 conn(0x7f61f4160770 legacy=0x7f61f40d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f3400700 1 -- 172.21.15.99:0/880767370 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40d1800 con 0x7f61f40b3cc0 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f3400700 1 --2- 172.21.15.99:0/880767370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40b3cc0 0x7f61f40d2e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61e00098f0 tx=0x7f61e0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507fe0b434f46933 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f2a00700 1 -- 172.21.15.99:0/880767370 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0011830 con 0x7f61f40b3cc0 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.944+0000 7f61f2a00700 1 -- 172.21.15.99:0/880767370 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e0011990 con 0x7f61f40b3cc0 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61f2a00700 1 -- 172.21.15.99:0/880767370 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e0011d20 con 0x7f61f40b3cc0 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 -- 172.21.15.99:0/880767370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40b3cc0 msgr2=0x7f61f40d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 --2- 172.21.15.99:0/880767370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40b3cc0 0x7f61f40d2e40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61e00098f0 tx=0x7f61e0009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:04.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 -- 172.21.15.99:0/880767370 shutdown_connections 2024-03-29T09:10:04.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 --2- 172.21.15.99:0/880767370 >> v2:172.21.15.133:3300/0 conn(0x7f61f40d3380 0x7f61f40ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 --2- 172.21.15.99:0/880767370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40b3cc0 0x7f61f40d2e40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 -- 172.21.15.99:0/880767370 >> 172.21.15.99:0/880767370 conn(0x7f61f408afd0 msgr2=0x7f61f409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:04.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 -- 172.21.15.99:0/880767370 shutdown_connections 2024-03-29T09:10:04.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.945+0000 7f61fb200700 1 -- 172.21.15.99:0/880767370 wait complete. 2024-03-29T09:10:04.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.947+0000 7f61fb200700 1 Processor -- start 2024-03-29T09:10:04.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.947+0000 7f61fb200700 1 -- start start 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61fb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f40b3cc0 0x7f61f4153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61fb200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40d3380 0x7f61f414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61fb200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f4160770 0x7f61f40cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61fb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f414f8a0 con 0x7f61f40b3cc0 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61fb200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f414fb80 con 0x7f61f4160770 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61fb200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f414fe60 con 0x7f61f40d3380 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f40b3cc0 0x7f61f4153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f40b3cc0 0x7f61f4153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50854/0 (socket says 172.21.15.99:50854) 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61f3e00700 1 -- 172.21.15.99:0/3981628193 learned_addr learned my addr 172.21.15.99:0/3981628193 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61f3400700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40d3380 0x7f61f414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61f9200700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f4160770 0x7f61f40cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61f3e00700 1 -- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40d3380 msgr2=0x7f61f414ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.948+0000 7f61f3e00700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40d3380 0x7f61f414ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61f3e00700 1 -- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f4160770 msgr2=0x7f61f40cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61f3e00700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f4160770 0x7f61f40cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61f3e00700 1 -- 172.21.15.99:0/3981628193 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0009550 con 0x7f61f40b3cc0 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61f3e00700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f40b3cc0 0x7f61f4153000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f61e400c500 tx=0x7f61e400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61ebe00700 1 -- 172.21.15.99:0/3981628193 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40052d0 con 0x7f61f40b3cc0 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61ebe00700 1 -- 172.21.15.99:0/3981628193 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e4005450 con 0x7f61f40b3cc0 2024-03-29T09:10:04.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61ebe00700 1 -- 172.21.15.99:0/3981628193 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e40038c0 con 0x7f61f40b3cc0 2024-03-29T09:10:04.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f40b5df0 con 0x7f61f40b3cc0 2024-03-29T09:10:04.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.949+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f4150400 con 0x7f61f40b3cc0 2024-03-29T09:10:04.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.950+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f40b6310 con 0x7f61f40b3cc0 2024-03-29T09:10:04.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.951+0000 7f61ebe00700 1 -- 172.21.15.99:0/3981628193 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f61e4003a20 con 0x7f61f40b3cc0 2024-03-29T09:10:04.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.952+0000 7f61ebe00700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61d406ac90 0x7f61d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:04.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.952+0000 7f61ebe00700 1 -- 172.21.15.99:0/3981628193 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f61e4088a80 con 0x7f61f40b3cc0 2024-03-29T09:10:04.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.952+0000 7f61f3400700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61d406ac90 0x7f61d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:04.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.953+0000 7f61f3400700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61d406ac90 0x7f61d406d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009c40 tx=0x7f61e0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:04.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:04.956+0000 7f61ebe00700 1 -- 172.21.15.99:0/3981628193 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f61f40b6310 con 0x7f61f40b3cc0 2024-03-29T09:10:05.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.166+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61f40b6310 con 0x7f61f40b3cc0 2024-03-29T09:10:05.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.166+0000 7f61ebe00700 1 -- 172.21.15.99:0/3981628193 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61f40b6310 con 0x7f61f40b3cc0 2024-03-29T09:10:05.166 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61d406ac90 msgr2=0x7f61d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61d406ac90 0x7f61d406d150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009c40 tx=0x7f61e0008030 comp rx=0 tx=0).stop 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f40b3cc0 msgr2=0x7f61f4153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f40b3cc0 0x7f61f4153000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f61e400c500 tx=0x7f61e400c810 comp rx=0 tx=0).stop 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 shutdown_connections 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f61d406ac90 0x7f61d406d150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f4160770 0x7f61f40cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f61f40b3cc0 0x7f61f4153000 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 --2- 172.21.15.99:0/3981628193 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f40d3380 0x7f61f414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 >> 172.21.15.99:0/3981628193 conn(0x7f61f408afd0 msgr2=0x7f61f406c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:05.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 shutdown_connections 2024-03-29T09:10:05.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.169+0000 7f61fb200700 1 -- 172.21.15.99:0/3981628193 wait complete. 2024-03-29T09:10:05.187 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:05.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.380+0000 7f002b800700 1 Processor -- start 2024-03-29T09:10:05.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.380+0000 7f002b800700 1 -- start start 2024-03-29T09:10:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f002b800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0024160770 0x7f00240b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f002b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f00240ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f002b800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00240d1520 con 0x7f00240d0d60 2024-03-29T09:10:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f002b800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00240d1800 con 0x7f0024160770 2024-03-29T09:10:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f002b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00240d1ae0 con 0x7f00240d3550 2024-03-29T09:10:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f0023e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f00240d0d60 0x7f00240d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60912/0 (socket says 172.21.15.99:60912) 2024-03-29T09:10:05.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f0023e00700 1 -- 172.21.15.99:0/3757750701 learned_addr learned my addr 172.21.15.99:0/3757750701 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f0028e00700 1 --2- 172.21.15.99:0/3757750701 >> v2:172.21.15.133:3300/0 conn(0x7f0024160770 0x7f00240b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.381+0000 7f0029800700 1 --2- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f00240ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0023400700 1 -- 172.21.15.99:0/3757750701 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135642010 0 0) 0x7f00240d1520 con 0x7f00240d0d60 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0023400700 1 -- 172.21.15.99:0/3757750701 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f000c0035b0 con 0x7f00240d0d60 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0028e00700 1 -- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 msgr2=0x7f00240ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0028e00700 1 --2- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f00240ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0023400700 1 -- 172.21.15.99:0/3757750701 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2574256792 0 0) 0x7f000c0035b0 con 0x7f00240d0d60 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0028e00700 1 -- 172.21.15.99:0/3757750701 >> v1:172.21.15.99:6789/0 conn(0x7f00240d0d60 legacy=0x7f00240d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0028e00700 1 -- 172.21.15.99:0/3757750701 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00240d1dc0 con 0x7f0024160770 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.382+0000 7f0028e00700 1 --2- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024160770 0x7f00240b3cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0018009840 tx=0x7f0018009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca3d7d4fece98910 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f0023400700 1 -- 172.21.15.99:0/3757750701 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0010001670 con 0x7f0024160770 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f0023400700 1 -- 172.21.15.99:0/3757750701 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018004030 con 0x7f0024160770 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f0023400700 1 -- 172.21.15.99:0/3757750701 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00180043c0 con 0x7f0024160770 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 -- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024160770 msgr2=0x7f00240b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 --2- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024160770 0x7f00240b3cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0018009840 tx=0x7f0018009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 -- 172.21.15.99:0/3757750701 shutdown_connections 2024-03-29T09:10:05.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 --2- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f00240ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 --2- 172.21.15.99:0/3757750701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0024160770 0x7f00240b3cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 -- 172.21.15.99:0/3757750701 >> 172.21.15.99:0/3757750701 conn(0x7f002408afd0 msgr2=0x7f002409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:05.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 -- 172.21.15.99:0/3757750701 shutdown_connections 2024-03-29T09:10:05.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.383+0000 7f002b800700 1 -- 172.21.15.99:0/3757750701 wait complete. 2024-03-29T09:10:05.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.385+0000 7f002b800700 1 Processor -- start 2024-03-29T09:10:05.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.385+0000 7f002b800700 1 -- start start 2024-03-29T09:10:05.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f002b800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00240d0d60 0x7f00240b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f002b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f002415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f002b800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0024160770 0x7f00240a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f002b800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00240b7e10 con 0x7f0024160770 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f002b800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002415d4f0 con 0x7f00240d0d60 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f002b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002415d710 con 0x7f00240d3550 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f0029800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0024160770 0x7f00240a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f0029800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0024160770 0x7f00240a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50862/0 (socket says 172.21.15.99:50862) 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.386+0000 7f0029800700 1 -- 172.21.15.99:0/1768747289 learned_addr learned my addr 172.21.15.99:0/1768747289 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0028e00700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00240d0d60 0x7f00240b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0023e00700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f002415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0029800700 1 -- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 msgr2=0x7f002415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0029800700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f002415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0029800700 1 -- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00240d0d60 msgr2=0x7f00240b78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0029800700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00240d0d60 0x7f00240b78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0029800700 1 -- 172.21.15.99:0/1768747289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180094a0 con 0x7f0024160770 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0029800700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0024160770 0x7f00240a0870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f001c009e30 tx=0x7f001c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0020c00700 1 -- 172.21.15.99:0/1768747289 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c009ca0 con 0x7f0024160770 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0020c00700 1 -- 172.21.15.99:0/1768747289 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c00f040 con 0x7f0024160770 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f0020c00700 1 -- 172.21.15.99:0/1768747289 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c014480 con 0x7f0024160770 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.387+0000 7f002b800700 1 -- 172.21.15.99:0/1768747289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002415d9f0 con 0x7f0024160770 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.388+0000 7f002b800700 1 -- 172.21.15.99:0/1768747289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002415dfb0 con 0x7f0024160770 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.389+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0024068150 con 0x7f0024160770 2024-03-29T09:10:05.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.390+0000 7f0020c00700 1 -- 172.21.15.99:0/1768747289 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f001c0145e0 con 0x7f0024160770 2024-03-29T09:10:05.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.390+0000 7f0020c00700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f000406ac90 0x7f000406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.390+0000 7f0020c00700 1 -- 172.21.15.99:0/1768747289 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f001c08a150 con 0x7f0024160770 2024-03-29T09:10:05.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.391+0000 7f0028e00700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f000406ac90 0x7f000406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.391+0000 7f0028e00700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f000406ac90 0x7f000406d150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f00180090b0 tx=0x7f0018008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:05.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.395+0000 7f0020c00700 1 -- 172.21.15.99:0/1768747289 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f001c055070 con 0x7f0024160770 2024-03-29T09:10:05.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.604+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00240678b0 con 0x7f0024160770 2024-03-29T09:10:05.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.604+0000 7f0020c00700 1 -- 172.21.15.99:0/1768747289 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f001c058690 con 0x7f0024160770 2024-03-29T09:10:05.604 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.606+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f000406ac90 msgr2=0x7f000406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.606+0000 7f0016000700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f000406ac90 0x7f000406d150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f00180090b0 tx=0x7f0018008030 comp rx=0 tx=0).stop 2024-03-29T09:10:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.606+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0024160770 msgr2=0x7f00240a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.606+0000 7f0016000700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0024160770 0x7f00240a0870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f001c009e30 tx=0x7f001c00c3a0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.606+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 shutdown_connections 2024-03-29T09:10:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.606+0000 7f0016000700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f000406ac90 0x7f000406d150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.607+0000 7f0016000700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0024160770 0x7f00240a0870 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.607+0000 7f0016000700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00240d0d60 0x7f00240b78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.607+0000 7f0016000700 1 --2- 172.21.15.99:0/1768747289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00240d3550 0x7f002415cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.607+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 >> 172.21.15.99:0/1768747289 conn(0x7f002408afd0 msgr2=0x7f00240b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.607+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 shutdown_connections 2024-03-29T09:10:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.607+0000 7f0016000700 1 -- 172.21.15.99:0/1768747289 wait complete. 2024-03-29T09:10:05.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:05.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.817+0000 7fc611a00700 1 Processor -- start 2024-03-29T09:10:05.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.818+0000 7fc611a00700 1 -- start start 2024-03-29T09:10:05.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc611a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 0x7fc60c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc611a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c0d1520 con 0x7fc60c0d0d60 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc611a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c0d1800 con 0x7fc60c0d3550 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc611a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0d1ae0 con 0x7fc60c160770 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc60a000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc60c0d0d60 0x7fc60c0d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:60928/0 (socket says 172.21.15.99:60928) 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc60a000700 1 -- 172.21.15.99:0/1251620250 learned_addr learned my addr 172.21.15.99:0/1251620250 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc60aa00700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 0x7fc60c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051759707 0 0) 0x7fc60c0d1520 con 0x7fc60c0d0d60 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.819+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f00035b0 con 0x7fc60c0d0d60 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184524812 0 0) 0x7fc60c0d1800 con 0x7fc60c0d3550 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc60c0d1520 con 0x7fc60c0d3550 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2476627586 0 0) 0x7fc5f00035b0 con 0x7fc60c0d0d60 2024-03-29T09:10:05.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 msgr2=0x7fc60c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 0x7fc60c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 >> v1:172.21.15.133:6789/0 conn(0x7fc60c0d3550 legacy=0x7fc60c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0d1dc0 con 0x7fc60c0d0d60 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fc5f80015f0 con 0x7fc60c0d0d60 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 >> v1:172.21.15.99:6789/0 conn(0x7fc60c0d0d60 legacy=0x7fc60c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 0x7fc5f0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5f0004170 0x7fc5f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004d90 0x7fc5f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0d1ae0 con 0x7fc5f0004170 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f000bbd0 con 0x7fc5f0004d90 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f000bd90 con 0x7fc60c160770 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc60a000700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5f0004170 0x7fc5f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc60aa00700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 0x7fc5f0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.820+0000 7fc60b400700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004d90 0x7fc5f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc60a000700 1 -- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 msgr2=0x7fc5f0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc60a000700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c160770 0x7fc5f0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc60a000700 1 -- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004d90 msgr2=0x7fc5f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc60a000700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004d90 0x7fc5f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc60a000700 1 -- 172.21.15.99:0/1251620250 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00035b0 con 0x7fc5f0004170 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc60a000700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5f0004170 0x7fc5f0004850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007fa0 tx=0x7fc5f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01b969c660f151b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8007a60 con 0x7fc5f0004170 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800cc20 con 0x7fc5f0004170 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.821+0000 7fc609600700 1 -- 172.21.15.99:0/1251620250 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8005960 con 0x7fc5f0004170 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.822+0000 7fc611a00700 1 -- 172.21.15.99:0/1251620250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5f0004170 msgr2=0x7fc5f0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.822+0000 7fc611a00700 1 --2- 172.21.15.99:0/1251620250 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5f0004170 0x7fc5f0004850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8007fa0 tx=0x7fc5f0005880 comp rx=0 tx=0).stop 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.822+0000 7fc60b400700 1 -- 172.21.15.99:0/1251620250 reap_dead start 2024-03-29T09:10:05.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.822+0000 7fc611a00700 1 -- 172.21.15.99:0/1251620250 shutdown_connections 2024-03-29T09:10:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.822+0000 7fc611a00700 1 -- 172.21.15.99:0/1251620250 >> 172.21.15.99:0/1251620250 conn(0x7fc60c08afd0 msgr2=0x7fc60c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.822+0000 7fc611a00700 1 -- 172.21.15.99:0/1251620250 shutdown_connections 2024-03-29T09:10:05.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.822+0000 7fc611a00700 1 -- 172.21.15.99:0/1251620250 wait complete. 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.824+0000 7fc611a00700 1 Processor -- start 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.824+0000 7fc611a00700 1 -- start start 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.825+0000 7fc611a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004170 0x7fc60c1f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.825+0000 7fc611a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c14aca0 0x7fc60c14a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.825+0000 7fc611a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc60c14d6f0 0x7fc60c06c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.825+0000 7fc611a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0b6110 con 0x7fc60c14d6f0 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.825+0000 7fc611a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0b7c20 con 0x7fc5f0004170 2024-03-29T09:10:05.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.825+0000 7fc611a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0b7db0 con 0x7fc60c14aca0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc60c14d6f0 0x7fc60c06c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc60c14d6f0 0x7fc60c06c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50886/0 (socket says 172.21.15.99:50886) 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 -- 172.21.15.99:0/3101414891 learned_addr learned my addr 172.21.15.99:0/3101414891 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60aa00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004170 0x7fc60c1f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60a000700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c14aca0 0x7fc60c14a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 -- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c14aca0 msgr2=0x7fc60c14a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c14aca0 0x7fc60c14a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 -- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004170 msgr2=0x7fc60c1f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004170 0x7fc60c1f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 -- 172.21.15.99:0/3101414891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c14a9b0 con 0x7fc60c14d6f0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60a000700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c14aca0 0x7fc60c14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.826+0000 7fc60b400700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc60c14d6f0 0x7fc60c06c8c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0b2a30 tx=0x7fc60400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.827+0000 7fc602a00700 1 -- 172.21.15.99:0/3101414891 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60400dc70 con 0x7fc60c14d6f0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.827+0000 7fc602a00700 1 -- 172.21.15.99:0/3101414891 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60400ddf0 con 0x7fc60c14d6f0 2024-03-29T09:10:05.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.827+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c1f2e10 con 0x7fc60c14d6f0 2024-03-29T09:10:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.827+0000 7fc602a00700 1 -- 172.21.15.99:0/3101414891 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604003c40 con 0x7fc60c14d6f0 2024-03-29T09:10:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.827+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c1f33d0 con 0x7fc60c14d6f0 2024-03-29T09:10:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.828+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c0d2340 con 0x7fc60c14d6f0 2024-03-29T09:10:05.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.829+0000 7fc602a00700 1 -- 172.21.15.99:0/3101414891 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc6040050d0 con 0x7fc60c14d6f0 2024-03-29T09:10:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.829+0000 7fc602a00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc5ec06ac90 0x7fc5ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.830+0000 7fc602a00700 1 -- 172.21.15.99:0/3101414891 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc604089b00 con 0x7fc60c14d6f0 2024-03-29T09:10:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.830+0000 7fc60aa00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc5ec06ac90 0x7fc5ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:05.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.830+0000 7fc60aa00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc5ec06ac90 0x7fc5ec06d150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009ac0 tx=0x7fc5fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:05.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:05.834+0000 7fc602a00700 1 -- 172.21.15.99:0/3101414891 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc60408be60 con 0x7fc60c14d6f0 2024-03-29T09:10:06.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.041+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc60c0678b0 con 0x7fc60c14d6f0 2024-03-29T09:10:06.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.042+0000 7fc602a00700 1 -- 172.21.15.99:0/3101414891 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc6040548f0 con 0x7fc60c14d6f0 2024-03-29T09:10:06.042 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc5ec06ac90 msgr2=0x7fc5ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc5ec06ac90 0x7fc5ec06d150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009ac0 tx=0x7fc5fc008040 comp rx=0 tx=0).stop 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc60c14d6f0 msgr2=0x7fc60c06c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc60c14d6f0 0x7fc60c06c8c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0b2a30 tx=0x7fc60400d710 comp rx=0 tx=0).stop 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 shutdown_connections 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc5ec06ac90 0x7fc5ec06d150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc60c14d6f0 0x7fc60c06c8c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5f0004170 0x7fc60c1f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 --2- 172.21.15.99:0/3101414891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc60c14aca0 0x7fc60c14a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.044+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 >> 172.21.15.99:0/3101414891 conn(0x7fc60c08afd0 msgr2=0x7fc60c0cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.045+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 shutdown_connections 2024-03-29T09:10:06.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.045+0000 7fc611a00700 1 -- 172.21.15.99:0/3101414891 wait complete. 2024-03-29T09:10:06.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:06.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.253+0000 7f7cbcc00700 1 Processor -- start 2024-03-29T09:10:06.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.253+0000 7f7cbcc00700 1 -- start start 2024-03-29T09:10:06.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.254+0000 7f7cbcc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7cb81608e0 0x7f7cb80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.254+0000 7f7cbcc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7cb80d1b20 0x7f7cb80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.254+0000 7f7cbcc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb80cddc0 con 0x7f7cb8160cc0 2024-03-29T09:10:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.254+0000 7f7cbcc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb80ce0a0 con 0x7f7cb80d1b20 2024-03-29T09:10:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.254+0000 7f7cbcc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb80ce380 con 0x7f7cb81608e0 2024-03-29T09:10:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7cb80d1b20 0x7f7cb80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7cb80d1b20 0x7f7cb80ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50900/0 (socket says 172.21.15.99:50900) 2024-03-29T09:10:06.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 -- 172.21.15.99:0/3938201051 learned_addr learned my addr 172.21.15.99:0/3938201051 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb5e00700 1 --2- 172.21.15.99:0/3938201051 >> v2:172.21.15.133:3300/0 conn(0x7f7cb81608e0 0x7f7cb80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 -- 172.21.15.99:0/3938201051 >> v2:172.21.15.133:3300/0 conn(0x7f7cb81608e0 msgr2=0x7f7cb80d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:06.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 --2- 172.21.15.99:0/3938201051 >> v2:172.21.15.133:3300/0 conn(0x7f7cb81608e0 0x7f7cb80d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 -- 172.21.15.99:0/3938201051 >> v1:172.21.15.133:6789/0 conn(0x7f7cb8160cc0 legacy=0x7f7cb80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 -- 172.21.15.99:0/3938201051 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb80ce660 con 0x7f7cb80d1b20 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.255+0000 7f7cb6800700 1 --2- 172.21.15.99:0/3938201051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb80d1b20 0x7f7cb80ceb80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7cb808d790 tx=0x7f7cb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d5c932e3c1a513 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cb4a00700 1 -- 172.21.15.99:0/3938201051 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb000e070 con 0x7f7cb80d1b20 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cb4a00700 1 -- 172.21.15.99:0/3938201051 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb000b9c0 con 0x7f7cb80d1b20 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cb4a00700 1 -- 172.21.15.99:0/3938201051 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb000bd70 con 0x7f7cb80d1b20 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3938201051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb80d1b20 msgr2=0x7f7cb80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3938201051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb80d1b20 0x7f7cb80ceb80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7cb808d790 tx=0x7f7cb000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3938201051 shutdown_connections 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3938201051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb80d1b20 0x7f7cb80ceb80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3938201051 >> v2:172.21.15.133:3300/0 conn(0x7f7cb81608e0 0x7f7cb80d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3938201051 >> 172.21.15.99:0/3938201051 conn(0x7f7cb809c490 msgr2=0x7f7cb80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3938201051 shutdown_connections 2024-03-29T09:10:06.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.256+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3938201051 wait complete. 2024-03-29T09:10:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.258+0000 7f7cbcc00700 1 Processor -- start 2024-03-29T09:10:06.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.258+0000 7f7cbcc00700 1 -- start start 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cbcc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cb80d1b20 0x7f7cb8151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cbcc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb81608e0 0x7f7cb80a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cbcc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb8160cc0 0x7f7cb80d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cbcc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb809cf00 con 0x7f7cb8160cc0 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cbcc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81f3290 con 0x7f7cb81608e0 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cbcc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb809d150 con 0x7f7cb80d1b20 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cb6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb8160cc0 0x7f7cb80d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cb6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb8160cc0 0x7f7cb80d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50916/0 (socket says 172.21.15.99:50916) 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cb6800700 1 -- 172.21.15.99:0/3646459323 learned_addr learned my addr 172.21.15.99:0/3646459323 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.259+0000 7f7cb5e00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cb80d1b20 0x7f7cb8151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cb5400700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb81608e0 0x7f7cb80a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cb6800700 1 -- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cb80d1b20 msgr2=0x7f7cb8151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cb6800700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cb80d1b20 0x7f7cb8151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cb6800700 1 -- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb81608e0 msgr2=0x7f7cb80a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cb6800700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb81608e0 0x7f7cb80a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cb6800700 1 -- 172.21.15.99:0/3646459323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0009b30 con 0x7f7cb8160cc0 2024-03-29T09:10:06.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cb6800700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb8160cc0 0x7f7cb80d0d00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7cb808c9a0 tx=0x7f7cb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cae000700 1 -- 172.21.15.99:0/3646459323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0004880 con 0x7f7cb8160cc0 2024-03-29T09:10:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cae000700 1 -- 172.21.15.99:0/3646459323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb00099b0 con 0x7f7cb8160cc0 2024-03-29T09:10:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.260+0000 7f7cae000700 1 -- 172.21.15.99:0/3646459323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb001c680 con 0x7f7cb8160cc0 2024-03-29T09:10:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.261+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb809d430 con 0x7f7cb8160cc0 2024-03-29T09:10:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.261+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb809d9f0 con 0x7f7cb8160cc0 2024-03-29T09:10:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.262+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb80d2300 con 0x7f7cb8160cc0 2024-03-29T09:10:06.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.263+0000 7f7cae000700 1 -- 172.21.15.99:0/3646459323 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7cb001c7e0 con 0x7f7cb8160cc0 2024-03-29T09:10:06.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.264+0000 7f7cae000700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c9806ac90 0x7f7c9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.264+0000 7f7cae000700 1 -- 172.21.15.99:0/3646459323 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7cb008ae80 con 0x7f7cb8160cc0 2024-03-29T09:10:06.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.264+0000 7f7cb5e00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c9806ac90 0x7f7c9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.264+0000 7f7cb5e00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c9806ac90 0x7f7c9806d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80099d0 tx=0x7f7ca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:06.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.268+0000 7f7cae000700 1 -- 172.21.15.99:0/3646459323 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7cb0055d10 con 0x7f7cb8160cc0 2024-03-29T09:10:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.491+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7cb809d290 con 0x7f7cb8160cc0 2024-03-29T09:10:06.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.491+0000 7f7cae000700 1 -- 172.21.15.99:0/3646459323 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7cb809d290 con 0x7f7cb8160cc0 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c9806ac90 msgr2=0x7f7c9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c9806ac90 0x7f7c9806d150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80099d0 tx=0x7f7ca8008040 comp rx=0 tx=0).stop 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb8160cc0 msgr2=0x7f7cb80d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb8160cc0 0x7f7cb80d0d00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7cb808c9a0 tx=0x7f7cb0004ad0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 shutdown_connections 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c9806ac90 0x7f7c9806d150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7cb8160cc0 0x7f7cb80d0d00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7cb80d1b20 0x7f7cb8151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 --2- 172.21.15.99:0/3646459323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb81608e0 0x7f7cb80a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.493+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 >> 172.21.15.99:0/3646459323 conn(0x7f7cb809c490 msgr2=0x7f7cb80cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:06.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.494+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 shutdown_connections 2024-03-29T09:10:06.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.494+0000 7f7cbcc00700 1 -- 172.21.15.99:0/3646459323 wait complete. 2024-03-29T09:10:06.495 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:06.512 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:06.512 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:06.554 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:06.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.782+0000 7fa06f200700 1 Processor -- start 2024-03-29T09:10:06.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.782+0000 7fa06f200700 1 -- start start 2024-03-29T09:10:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06f200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa0700d1ba0 0x7fa0700cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06f200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0700d0110 con 0x7fa0700cfd00 2024-03-29T09:10:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06f200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0700d38d0 con 0x7fa07015d140 2024-03-29T09:10:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06f200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0700d0b00 con 0x7fa0700d1ba0 2024-03-29T09:10:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06d400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa0700d1ba0 0x7fa0700cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06d400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa0700d1ba0 0x7fa0700cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50924/0 (socket says 172.21.15.99:50924) 2024-03-29T09:10:06.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06d400700 1 -- 172.21.15.99:0/429196474 learned_addr learned my addr 172.21.15.99:0/429196474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06d400700 1 -- 172.21.15.99:0/429196474 >> v1:172.21.15.133:6789/0 conn(0x7fa07015d140 legacy=0x7fa0700ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06d400700 1 -- 172.21.15.99:0/429196474 >> v1:172.21.15.99:6789/0 conn(0x7fa0700cfd00 legacy=0x7fa0700b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.784+0000 7fa06d400700 1 -- 172.21.15.99:0/429196474 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0700d0de0 con 0x7fa0700d1ba0 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06d400700 1 --2- 172.21.15.99:0/429196474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0700d1ba0 0x7fa0700cf7c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0098f0 tx=0x7fa05c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21996085b2badcda server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06ca00700 1 -- 172.21.15.99:0/429196474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00b8a0 con 0x7fa0700d1ba0 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06ca00700 1 -- 172.21.15.99:0/429196474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c00ba00 con 0x7fa0700d1ba0 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06ca00700 1 -- 172.21.15.99:0/429196474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c00bd90 con 0x7fa0700d1ba0 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06f200700 1 -- 172.21.15.99:0/429196474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0700d1ba0 msgr2=0x7fa0700cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06f200700 1 --2- 172.21.15.99:0/429196474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0700d1ba0 0x7fa0700cf7c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0098f0 tx=0x7fa05c009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06f200700 1 -- 172.21.15.99:0/429196474 shutdown_connections 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.785+0000 7fa06f200700 1 --2- 172.21.15.99:0/429196474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa0700d1ba0 0x7fa0700cf7c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.786+0000 7fa06f200700 1 -- 172.21.15.99:0/429196474 >> 172.21.15.99:0/429196474 conn(0x7fa07009c2e0 msgr2=0x7fa0700d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:06.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.786+0000 7fa06f200700 1 -- 172.21.15.99:0/429196474 shutdown_connections 2024-03-29T09:10:06.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.786+0000 7fa06f200700 1 -- 172.21.15.99:0/429196474 wait complete. 2024-03-29T09:10:06.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.787+0000 7fa06f200700 1 Processor -- start 2024-03-29T09:10:06.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.787+0000 7fa06f200700 1 -- start start 2024-03-29T09:10:06.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.788+0000 7fa06f200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0700cfd00 0x7fa0700a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.788+0000 7fa06f200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0700d1ba0 0x7fa0700b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.788+0000 7fa06f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa07015d140 0x7fa0701f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.788+0000 7fa06f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07009da40 con 0x7fa07015d140 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.788+0000 7fa06f200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07009dd20 con 0x7fa0700d1ba0 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.788+0000 7fa06f200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07009e000 con 0x7fa0700cfd00 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa07015d140 0x7fa0701f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa07015d140 0x7fa0701f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50938/0 (socket says 172.21.15.99:50938) 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 -- 172.21.15.99:0/3624227340 learned_addr learned my addr 172.21.15.99:0/3624227340 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06d400700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0700d1ba0 0x7fa0700b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 -- 172.21.15.99:0/3624227340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0700cfd00 msgr2=0x7fa0700a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0700cfd00 0x7fa0700a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 -- 172.21.15.99:0/3624227340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0700d1ba0 msgr2=0x7fa0700b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0700d1ba0 0x7fa0700b5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 -- 172.21.15.99:0/3624227340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c009550 con 0x7fa07015d140 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06e800700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa07015d140 0x7fa0701f6200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa06800cc50 tx=0x7fa06800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa066000700 1 -- 172.21.15.99:0/3624227340 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800ad80 con 0x7fa07015d140 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa066000700 1 -- 172.21.15.99:0/3624227340 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06800ce50 con 0x7fa07015d140 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.789+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07009e2e0 con 0x7fa07015d140 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.790+0000 7fa066000700 1 -- 172.21.15.99:0/3624227340 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068003c40 con 0x7fa07015d140 2024-03-29T09:10:06.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.790+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07009d660 con 0x7fa07015d140 2024-03-29T09:10:06.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.791+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07009e800 con 0x7fa07015d140 2024-03-29T09:10:06.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.792+0000 7fa066000700 1 -- 172.21.15.99:0/3624227340 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa0680050d0 con 0x7fa07015d140 2024-03-29T09:10:06.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.793+0000 7fa066000700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa05006ac90 0x7fa05006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.793+0000 7fa066000700 1 -- 172.21.15.99:0/3624227340 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa068089760 con 0x7fa07015d140 2024-03-29T09:10:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.793+0000 7fa06de00700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa05006ac90 0x7fa05006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:06.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.794+0000 7fa06de00700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa05006ac90 0x7fa05006d150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa060000f60 tx=0x7fa060007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:06.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:06.797+0000 7fa066000700 1 -- 172.21.15.99:0/3624227340 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa07009e800 con 0x7fa07015d140 2024-03-29T09:10:07.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.002+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa07009e800 con 0x7fa07015d140 2024-03-29T09:10:07.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.003+0000 7fa066000700 1 -- 172.21.15.99:0/3624227340 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7fa07009e800 con 0x7fa07015d140 2024-03-29T09:10:07.004 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:07.004 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:07.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa05006ac90 msgr2=0x7fa05006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa05006ac90 0x7fa05006d150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa060000f60 tx=0x7fa060007a40 comp rx=0 tx=0).stop 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa07015d140 msgr2=0x7fa0701f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa07015d140 0x7fa0701f6200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa06800cc50 tx=0x7fa06800a340 comp rx=0 tx=0).stop 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 shutdown_connections 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa05006ac90 0x7fa05006d150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa07015d140 0x7fa0701f6200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0700cfd00 0x7fa0700a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 --2- 172.21.15.99:0/3624227340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0700d1ba0 0x7fa0700b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 >> 172.21.15.99:0/3624227340 conn(0x7fa07009c2e0 msgr2=0x7fa07015f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 shutdown_connections 2024-03-29T09:10:07.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.006+0000 7fa06f200700 1 -- 172.21.15.99:0/3624227340 wait complete. 2024-03-29T09:10:07.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:07.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:07.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.218+0000 7f9f45600700 1 Processor -- start 2024-03-29T09:10:07.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.218+0000 7f9f45600700 1 -- start start 2024-03-29T09:10:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.219+0000 7f9f45600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f4015d800 0x7f9f400d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.219+0000 7f9f45600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f400d1e90 0x7f9f400d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.219+0000 7f9f45600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f400d38d0 con 0x7f9f4015d420 2024-03-29T09:10:07.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.219+0000 7f9f45600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400d12b0 con 0x7f9f400d1e90 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.219+0000 7f9f45600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400afdd0 con 0x7f9f4015d800 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.220+0000 7f9f3e000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f4015d800 0x7f9f400d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.220+0000 7f9f3f400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f400d1e90 0x7f9f400d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.220+0000 7f9f3ea00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9f4015d420 0x7f9f400d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:37500/0 (socket says 172.21.15.99:37500) 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.220+0000 7f9f3ea00700 1 -- 172.21.15.99:0/2073267800 learned_addr learned my addr 172.21.15.99:0/2073267800 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3e000700 1 -- 172.21.15.99:0/2073267800 >> v2:172.21.15.133:3300/0 conn(0x7f9f400d1e90 msgr2=0x7f9f400d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.220+0000 7f9f3d600700 1 -- 172.21.15.99:0/2073267800 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372338682 0 0) 0x7f9f400d38d0 con 0x7f9f4015d420 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3e000700 1 --2- 172.21.15.99:0/2073267800 >> v2:172.21.15.133:3300/0 conn(0x7f9f400d1e90 0x7f9f400d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3e000700 1 -- 172.21.15.99:0/2073267800 >> v1:172.21.15.133:6789/0 conn(0x7f9f4015d420 legacy=0x7f9f400d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3e000700 1 -- 172.21.15.99:0/2073267800 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400d3b00 con 0x7f9f4015d800 2024-03-29T09:10:07.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3f400700 1 --2- 172.21.15.99:0/2073267800 >> v2:172.21.15.133:3300/0 conn(0x7f9f400d1e90 0x7f9f400d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3e000700 1 --2- 172.21.15.99:0/2073267800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f4015d800 0x7f9f400d2e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009840 tx=0x7f9f2c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e979efe163037d4f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3d600700 1 -- 172.21.15.99:0/2073267800 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c011830 con 0x7f9f4015d800 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3d600700 1 -- 172.21.15.99:0/2073267800 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c011990 con 0x7f9f4015d800 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.221+0000 7f9f3d600700 1 -- 172.21.15.99:0/2073267800 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c011d20 con 0x7f9f4015d800 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 -- 172.21.15.99:0/2073267800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f4015d800 msgr2=0x7f9f400d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 --2- 172.21.15.99:0/2073267800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f4015d800 0x7f9f400d2e50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009840 tx=0x7f9f2c009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 -- 172.21.15.99:0/2073267800 shutdown_connections 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 --2- 172.21.15.99:0/2073267800 >> v2:172.21.15.133:3300/0 conn(0x7f9f400d1e90 0x7f9f400d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 --2- 172.21.15.99:0/2073267800 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f4015d800 0x7f9f400d2e50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 -- 172.21.15.99:0/2073267800 >> 172.21.15.99:0/2073267800 conn(0x7f9f4008afd0 msgr2=0x7f9f4009c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 -- 172.21.15.99:0/2073267800 shutdown_connections 2024-03-29T09:10:07.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.222+0000 7f9f45600700 1 -- 172.21.15.99:0/2073267800 wait complete. 2024-03-29T09:10:07.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.224+0000 7f9f45600700 1 Processor -- start 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.224+0000 7f9f45600700 1 -- start start 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f45600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f400d1e90 0x7f9f4014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f45600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f4015d420 0x7f9f400cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f45600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f400cbed0 0x7f9f400b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f45600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4015fe80 con 0x7f9f400d1e90 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f45600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4015f2d0 con 0x7f9f4015d420 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f45600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f401602a0 con 0x7f9f400cbed0 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f400d1e90 0x7f9f4014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f400d1e90 0x7f9f4014fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50948/0 (socket says 172.21.15.99:50948) 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 -- 172.21.15.99:0/3134326455 learned_addr learned my addr 172.21.15.99:0/3134326455 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3e000700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f4015d420 0x7f9f400cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3f400700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f400cbed0 0x7f9f400b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 -- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f400cbed0 msgr2=0x7f9f400b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f400cbed0 0x7f9f400b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 -- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f4015d420 msgr2=0x7f9f400cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f4015d420 0x7f9f400cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3ea00700 1 -- 172.21.15.99:0/3134326455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38009a70 con 0x7f9f400d1e90 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.225+0000 7f9f3f400700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f400cbed0 0x7f9f400b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.226+0000 7f9f3ea00700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f400d1e90 0x7f9f4014fd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9f300011e0 tx=0x7f9f3000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.226+0000 7f9f36a00700 1 -- 172.21.15.99:0/3134326455 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30004c80 con 0x7f9f400d1e90 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.226+0000 7f9f36a00700 1 -- 172.21.15.99:0/3134326455 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30004e00 con 0x7f9f400d1e90 2024-03-29T09:10:07.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.226+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c0094a0 con 0x7f9f400d1e90 2024-03-29T09:10:07.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.226+0000 7f9f36a00700 1 -- 172.21.15.99:0/3134326455 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30007310 con 0x7f9f400d1e90 2024-03-29T09:10:07.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.226+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4015f4e0 con 0x7f9f400d1e90 2024-03-29T09:10:07.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.227+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f400678b0 con 0x7f9f400d1e90 2024-03-29T09:10:07.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.228+0000 7f9f36a00700 1 -- 172.21.15.99:0/3134326455 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f30007470 con 0x7f9f400d1e90 2024-03-29T09:10:07.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.229+0000 7f9f36a00700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f2006ac90 0x7f9f2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.229+0000 7f9f36a00700 1 -- 172.21.15.99:0/3134326455 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f30089330 con 0x7f9f400d1e90 2024-03-29T09:10:07.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.229+0000 7f9f3e000700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f2006ac90 0x7f9f2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.229+0000 7f9f3e000700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f2006ac90 0x7f9f2006d150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0090b0 tx=0x7f9f2c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:07.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.233+0000 7f9f36a00700 1 -- 172.21.15.99:0/3134326455 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f300541c0 con 0x7f9f400d1e90 2024-03-29T09:10:07.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.435+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f400b23d0 con 0x7f9f400d1e90 2024-03-29T09:10:07.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.436+0000 7f9f36a00700 1 -- 172.21.15.99:0/3134326455 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f300577e0 con 0x7f9f400d1e90 2024-03-29T09:10:07.436 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.438+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f2006ac90 msgr2=0x7f9f2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.438+0000 7f9f45600700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f2006ac90 0x7f9f2006d150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0090b0 tx=0x7f9f2c008030 comp rx=0 tx=0).stop 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.438+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f400d1e90 msgr2=0x7f9f4014fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.438+0000 7f9f45600700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f400d1e90 0x7f9f4014fd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9f300011e0 tx=0x7f9f3000ca30 comp rx=0 tx=0).stop 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 shutdown_connections 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9f2006ac90 0x7f9f2006d150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f400cbed0 0x7f9f400b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f400d1e90 0x7f9f4014fd20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 --2- 172.21.15.99:0/3134326455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f4015d420 0x7f9f400cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 >> 172.21.15.99:0/3134326455 conn(0x7f9f4008afd0 msgr2=0x7f9f400cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:07.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 shutdown_connections 2024-03-29T09:10:07.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.439+0000 7f9f45600700 1 -- 172.21.15.99:0/3134326455 wait complete. 2024-03-29T09:10:07.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:07.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.649+0000 7fc2b1200700 1 Processor -- start 2024-03-29T09:10:07.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.650+0000 7fc2b1200700 1 -- start start 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2b1200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac160770 0x7fc2ac0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2b1200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2ac0d3550 0x7fc2ac0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2b1200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ac0d1520 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2b1200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ac0d1800 con 0x7fc2ac0d3550 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2b1200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac0d1ae0 con 0x7fc2ac160770 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2aa800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2ac0d3550 0x7fc2ac0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2aa800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2ac0d3550 0x7fc2ac0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50952/0 (socket says 172.21.15.99:50952) 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2aa800700 1 -- 172.21.15.99:0/1576775422 learned_addr learned my addr 172.21.15.99:0/1576775422 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2a9e00700 1 --2- 172.21.15.99:0/1576775422 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac160770 0x7fc2ac0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2aa800700 1 -- 172.21.15.99:0/1576775422 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac160770 msgr2=0x7fc2ac0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2aa800700 1 --2- 172.21.15.99:0/1576775422 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac160770 0x7fc2ac0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2aa800700 1 -- 172.21.15.99:0/1576775422 >> v1:172.21.15.133:6789/0 conn(0x7fc2ac0d0d60 legacy=0x7fc2ac0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.651+0000 7fc2aa800700 1 -- 172.21.15.99:0/1576775422 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac0d1dc0 con 0x7fc2ac0d3550 2024-03-29T09:10:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2aa800700 1 --2- 172.21.15.99:0/1576775422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d3550 0x7fc2ac0ce080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009e50 tx=0x7fc2a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438af22720154027 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2a8a00700 1 -- 172.21.15.99:0/1576775422 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400e070 con 0x7fc2ac0d3550 2024-03-29T09:10:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2a8a00700 1 -- 172.21.15.99:0/1576775422 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a400b9c0 con 0x7fc2ac0d3550 2024-03-29T09:10:07.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2a8a00700 1 -- 172.21.15.99:0/1576775422 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400bd70 con 0x7fc2ac0d3550 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2b1200700 1 -- 172.21.15.99:0/1576775422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d3550 msgr2=0x7fc2ac0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2b1200700 1 --2- 172.21.15.99:0/1576775422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d3550 0x7fc2ac0ce080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009e50 tx=0x7fc2a400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2b1200700 1 -- 172.21.15.99:0/1576775422 shutdown_connections 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2b1200700 1 --2- 172.21.15.99:0/1576775422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d3550 0x7fc2ac0ce080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2b1200700 1 --2- 172.21.15.99:0/1576775422 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac160770 0x7fc2ac0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.652+0000 7fc2b1200700 1 -- 172.21.15.99:0/1576775422 >> 172.21.15.99:0/1576775422 conn(0x7fc2ac08afd0 msgr2=0x7fc2ac09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.653+0000 7fc2b1200700 1 -- 172.21.15.99:0/1576775422 shutdown_connections 2024-03-29T09:10:07.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.653+0000 7fc2b1200700 1 -- 172.21.15.99:0/1576775422 wait complete. 2024-03-29T09:10:07.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.655+0000 7fc2b1200700 1 Processor -- start 2024-03-29T09:10:07.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.655+0000 7fc2b1200700 1 -- start start 2024-03-29T09:10:07.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.656+0000 7fc2b1200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d0d60 0x7fc2ac1f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.656+0000 7fc2b1200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac0d3550 0x7fc2ac0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.656+0000 7fc2b1200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2ac160770 0x7fc2ac14e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.656+0000 7fc2b1200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac14c1d0 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.656+0000 7fc2b1200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac0cb710 con 0x7fc2ac160770 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.656+0000 7fc2b1200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac0cb9f0 con 0x7fc2ac0d3550 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.656+0000 7fc2a9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d0d60 0x7fc2ac1f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d0d60 0x7fc2ac1f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:50956/0 (socket says 172.21.15.99:50956) 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 -- 172.21.15.99:0/3130732530 learned_addr learned my addr 172.21.15.99:0/3130732530 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 -- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac0d3550 msgr2=0x7fc2ac0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac0d3550 0x7fc2ac0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 -- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2ac160770 msgr2=0x7fc2ac14e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2aa800700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2ac160770 0x7fc2ac14e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2ac160770 0x7fc2ac14e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 -- 172.21.15.99:0/3130732530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a4009b30 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2aa800700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2ac160770 0x7fc2ac14e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a9e00700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d0d60 0x7fc2ac1f1fd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00d800 tx=0x7fc29c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a2000700 1 -- 172.21.15.99:0/3130732530 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0125f0 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a2000700 1 -- 172.21.15.99:0/3130732530 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c012750 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac14c3e0 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.657+0000 7fc2a2000700 1 -- 172.21.15.99:0/3130732530 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c012b00 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.658+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac14f000 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.658+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac0cbc10 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.659+0000 7fc2a2000700 1 -- 172.21.15.99:0/3130732530 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc29c012c60 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.661+0000 7fc2a2000700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc28c06ac90 0x7fc28c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:07.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.661+0000 7fc2a2000700 1 -- 172.21.15.99:0/3130732530 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc29c089ab0 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.662+0000 7fc2a9400700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc28c06ac90 0x7fc28c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:07.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.662+0000 7fc2a9400700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc28c06ac90 0x7fc28c06d150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc298000f60 tx=0x7fc298007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:07.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.666+0000 7fc2a2000700 1 -- 172.21.15.99:0/3130732530 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc29c0548b0 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.866+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2ac0d20a0 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.867+0000 7fc2a2000700 1 -- 172.21.15.99:0/3130732530 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc29c057ed0 con 0x7fc2ac0d0d60 2024-03-29T09:10:07.867 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc28c06ac90 msgr2=0x7fc28c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc28c06ac90 0x7fc28c06d150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc298000f60 tx=0x7fc298007a40 comp rx=0 tx=0).stop 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d0d60 msgr2=0x7fc2ac1f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d0d60 0x7fc2ac1f1fd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00d800 tx=0x7fc29c00dbc0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 shutdown_connections 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc28c06ac90 0x7fc28c06d150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2ac160770 0x7fc2ac14e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2ac0d0d60 0x7fc2ac1f1fd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 --2- 172.21.15.99:0/3130732530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2ac0d3550 0x7fc2ac0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.869+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 >> 172.21.15.99:0/3130732530 conn(0x7fc2ac08afd0 msgr2=0x7fc2ac06cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.870+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 shutdown_connections 2024-03-29T09:10:07.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:07.870+0000 7fc2b1200700 1 -- 172.21.15.99:0/3130732530 wait complete. 2024-03-29T09:10:07.888 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:08.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.079+0000 7f9952e00700 1 Processor -- start 2024-03-29T09:10:08.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.079+0000 7f9952e00700 1 -- start start 2024-03-29T09:10:08.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.080+0000 7f9952e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c160cd0 0x7f994c0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.080+0000 7f9952e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c0d1f00 con 0x7f994c0b47c0 2024-03-29T09:10:08.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.080+0000 7f9952e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f994c0cf3e0 con 0x7f994c0b3ab0 2024-03-29T09:10:08.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.080+0000 7f9952e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0d2390 con 0x7f994c160cd0 2024-03-29T09:10:08.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.080+0000 7f994b400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f994c0b47c0 0x7f994c0cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34868/0 (socket says 172.21.15.99:34868) 2024-03-29T09:10:08.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994b400700 1 -- 172.21.15.99:0/1908545187 learned_addr learned my addr 172.21.15.99:0/1908545187 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994be00700 1 --2- 172.21.15.99:0/1908545187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c160cd0 0x7f994c0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679688465 0 0) 0x7f994c0d1f00 con 0x7f994c0b47c0 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99300035b0 con 0x7f994c0b47c0 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121583616 0 0) 0x7f994c0cf3e0 con 0x7f994c0b3ab0 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f994c0d1f00 con 0x7f994c0b3ab0 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994be00700 1 -- 172.21.15.99:0/1908545187 >> v1:172.21.15.133:6789/0 conn(0x7f994c0b3ab0 legacy=0x7f994c0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994be00700 1 -- 172.21.15.99:0/1908545187 >> v1:172.21.15.99:6789/0 conn(0x7f994c0b47c0 legacy=0x7f994c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127475305 0 0) 0x7f99300035b0 con 0x7f994c0b47c0 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994be00700 1 -- 172.21.15.99:0/1908545187 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c0d15d0 con 0x7f994c160cd0 2024-03-29T09:10:08.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.081+0000 7f994be00700 1 --2- 172.21.15.99:0/1908545187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c160cd0 0x7f994c0b1f90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f993c009870 tx=0x7f993c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f417f8a2d6e2ad server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938001670 con 0x7f994c160cd0 2024-03-29T09:10:08.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c004030 con 0x7f994c160cd0 2024-03-29T09:10:08.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f994aa00700 1 -- 172.21.15.99:0/1908545187 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c0043c0 con 0x7f994c160cd0 2024-03-29T09:10:08.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f9952e00700 1 -- 172.21.15.99:0/1908545187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c160cd0 msgr2=0x7f994c0b1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f9952e00700 1 --2- 172.21.15.99:0/1908545187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c160cd0 0x7f994c0b1f90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f993c009870 tx=0x7f993c009b80 comp rx=0 tx=0).stop 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f9952e00700 1 -- 172.21.15.99:0/1908545187 shutdown_connections 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f9952e00700 1 --2- 172.21.15.99:0/1908545187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c160cd0 0x7f994c0b1f90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f9952e00700 1 -- 172.21.15.99:0/1908545187 >> 172.21.15.99:0/1908545187 conn(0x7f994c09c4d0 msgr2=0x7f994c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f9952e00700 1 -- 172.21.15.99:0/1908545187 shutdown_connections 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.082+0000 7f9952e00700 1 -- 172.21.15.99:0/1908545187 wait complete. 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.084+0000 7f9952e00700 1 Processor -- start 2024-03-29T09:10:08.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.084+0000 7f9952e00700 1 -- start start 2024-03-29T09:10:08.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.085+0000 7f9952e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f994c0b3ab0 0x7f994c1f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.085+0000 7f9952e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c0b47c0 0x7f994c1f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.085+0000 7f9952e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c160cd0 0x7f994c15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.085+0000 7f9952e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1f4060 con 0x7f994c160cd0 2024-03-29T09:10:08.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.085+0000 7f9952e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1f7580 con 0x7f994c0b3ab0 2024-03-29T09:10:08.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.085+0000 7f9952e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994c1f33e0 con 0x7f994c0b47c0 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f994be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f994c0b3ab0 0x7f994c1f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f994b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c0b47c0 0x7f994c1f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f994be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f994c0b3ab0 0x7f994c1f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:49858/0 (socket says 172.21.15.99:49858) 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f994be00700 1 -- 172.21.15.99:0/3463537024 learned_addr learned my addr 172.21.15.99:0/3463537024 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f9950e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c160cd0 0x7f994c15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f9950e00700 1 -- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c0b47c0 msgr2=0x7f994c1f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f9950e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c0b47c0 0x7f994c1f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f9950e00700 1 -- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f994c0b3ab0 msgr2=0x7f994c1f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f9950e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f994c0b3ab0 0x7f994c1f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f9950e00700 1 -- 172.21.15.99:0/3463537024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0094a0 con 0x7f994c160cd0 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f994be00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f994c0b3ab0 0x7f994c1f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f994b400700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c0b47c0 0x7f994c1f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:08.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.086+0000 7f9950e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c160cd0 0x7f994c15e9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f994400ca70 tx=0x7f994400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.088+0000 7f9943e00700 1 -- 172.21.15.99:0/3463537024 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9944003c00 con 0x7f994c160cd0 2024-03-29T09:10:08.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.088+0000 7f9943e00700 1 -- 172.21.15.99:0/3463537024 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9944003d60 con 0x7f994c160cd0 2024-03-29T09:10:08.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.088+0000 7f9943e00700 1 -- 172.21.15.99:0/3463537024 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400aa20 con 0x7f994c160cd0 2024-03-29T09:10:08.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.088+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c1f36c0 con 0x7f994c160cd0 2024-03-29T09:10:08.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.088+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c1f3c80 con 0x7f994c160cd0 2024-03-29T09:10:08.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.090+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c0678b0 con 0x7f994c160cd0 2024-03-29T09:10:08.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.090+0000 7f9943e00700 1 -- 172.21.15.99:0/3463537024 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f994400ab80 con 0x7f994c160cd0 2024-03-29T09:10:08.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.091+0000 7f9943e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f992c06ac90 0x7f992c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.091+0000 7f9943e00700 1 -- 172.21.15.99:0/3463537024 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f99440893b0 con 0x7f994c160cd0 2024-03-29T09:10:08.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.091+0000 7f994be00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f992c06ac90 0x7f992c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.092+0000 7f994be00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f992c06ac90 0x7f992c06d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f993c009bc0 tx=0x7f993c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.096+0000 7f9943e00700 1 -- 172.21.15.99:0/3463537024 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9944054380 con 0x7f994c160cd0 2024-03-29T09:10:08.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.297+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f994c0ceda0 con 0x7f994c160cd0 2024-03-29T09:10:08.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.297+0000 7f9943e00700 1 -- 172.21.15.99:0/3463537024 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99440579a0 con 0x7f994c160cd0 2024-03-29T09:10:08.297 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f992c06ac90 msgr2=0x7f992c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f992c06ac90 0x7f992c06d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f993c009bc0 tx=0x7f993c008030 comp rx=0 tx=0).stop 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c160cd0 msgr2=0x7f994c15e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c160cd0 0x7f994c15e9c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f994400ca70 tx=0x7f994400cd80 comp rx=0 tx=0).stop 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 shutdown_connections 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f992c06ac90 0x7f992c06d150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f994c160cd0 0x7f994c15e9c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f994c0b3ab0 0x7f994c1f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 --2- 172.21.15.99:0/3463537024 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f994c0b47c0 0x7f994c1f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 >> 172.21.15.99:0/3463537024 conn(0x7f994c09c4d0 msgr2=0x7f994c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:08.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 shutdown_connections 2024-03-29T09:10:08.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.300+0000 7f9952e00700 1 -- 172.21.15.99:0/3463537024 wait complete. 2024-03-29T09:10:08.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:08.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.511+0000 7f5663800700 1 Processor -- start 2024-03-29T09:10:08.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.511+0000 7f5663800700 1 -- start start 2024-03-29T09:10:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f5663800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f565c160770 0x7f565c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f5663800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f565c0d0d60 0x7f565c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f5663800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565c0d3810 con 0x7f565c0d3380 2024-03-29T09:10:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f5663800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c0d1520 con 0x7f565c0d0d60 2024-03-29T09:10:08.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f5663800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c0d1800 con 0x7f565c160770 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f565be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f565c0d0d60 0x7f565c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f565be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f565c0d0d60 0x7f565c0d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36410/0 (socket says 172.21.15.99:36410) 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.512+0000 7f565be00700 1 -- 172.21.15.99:0/3892677983 learned_addr learned my addr 172.21.15.99:0/3892677983 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f5660e00700 1 --2- 172.21.15.99:0/3892677983 >> v2:172.21.15.133:3300/0 conn(0x7f565c160770 0x7f565c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565be00700 1 -- 172.21.15.99:0/3892677983 >> v2:172.21.15.133:3300/0 conn(0x7f565c160770 msgr2=0x7f565c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565be00700 1 --2- 172.21.15.99:0/3892677983 >> v2:172.21.15.133:3300/0 conn(0x7f565c160770 0x7f565c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565be00700 1 -- 172.21.15.99:0/3892677983 >> v1:172.21.15.133:6789/0 conn(0x7f565c0d3380 legacy=0x7f565c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565be00700 1 -- 172.21.15.99:0/3892677983 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c0d1ae0 con 0x7f565c0d0d60 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565be00700 1 --2- 172.21.15.99:0/3892677983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c0d0d60 0x7f565c0d2e40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f565c0b5d20 tx=0x7f5648009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11154f2b57ac7840 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565b400700 1 -- 172.21.15.99:0/3892677983 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800b870 con 0x7f565c0d0d60 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565b400700 1 -- 172.21.15.99:0/3892677983 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564800b9d0 con 0x7f565c0d0d60 2024-03-29T09:10:08.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.513+0000 7f565b400700 1 -- 172.21.15.99:0/3892677983 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800bd60 con 0x7f565c0d0d60 2024-03-29T09:10:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 -- 172.21.15.99:0/3892677983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c0d0d60 msgr2=0x7f565c0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 --2- 172.21.15.99:0/3892677983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c0d0d60 0x7f565c0d2e40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f565c0b5d20 tx=0x7f5648009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 -- 172.21.15.99:0/3892677983 shutdown_connections 2024-03-29T09:10:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 --2- 172.21.15.99:0/3892677983 >> v2:172.21.15.133:3300/0 conn(0x7f565c160770 0x7f565c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 --2- 172.21.15.99:0/3892677983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c0d0d60 0x7f565c0d2e40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 -- 172.21.15.99:0/3892677983 >> 172.21.15.99:0/3892677983 conn(0x7f565c08afd0 msgr2=0x7f565c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 -- 172.21.15.99:0/3892677983 shutdown_connections 2024-03-29T09:10:08.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.514+0000 7f5663800700 1 -- 172.21.15.99:0/3892677983 wait complete. 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.516+0000 7f5663800700 1 Processor -- start 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.516+0000 7f5663800700 1 -- start start 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5663800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f565c0d0d60 0x7f565c0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5663800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c0d3380 0x7f565c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5663800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c160770 0x7f565c15a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5663800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0b6420 con 0x7f565c160770 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5663800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0cc780 con 0x7f565c0d3380 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5663800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0cca60 con 0x7f565c0d0d60 2024-03-29T09:10:08.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5661800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c160770 0x7f565c15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5661800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c160770 0x7f565c15a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36416/0 (socket says 172.21.15.99:36416) 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5661800700 1 -- 172.21.15.99:0/551881437 learned_addr learned my addr 172.21.15.99:0/551881437 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f565be00700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c0d3380 0x7f565c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.517+0000 7f5660e00700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f565c0d0d60 0x7f565c0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5661800700 1 -- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f565c0d0d60 msgr2=0x7f565c0cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5661800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f565c0d0d60 0x7f565c0cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5661800700 1 -- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c0d3380 msgr2=0x7f565c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5661800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c0d3380 0x7f565c159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5661800700 1 -- 172.21.15.99:0/551881437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648009550 con 0x7f565c160770 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5661800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c160770 0x7f565c15a410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f565400cc50 tx=0x7f565400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5658c00700 1 -- 172.21.15.99:0/551881437 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654003c00 con 0x7f565c160770 2024-03-29T09:10:08.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5658c00700 1 -- 172.21.15.99:0/551881437 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654003d60 con 0x7f565c160770 2024-03-29T09:10:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c0b6630 con 0x7f565c160770 2024-03-29T09:10:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5658c00700 1 -- 172.21.15.99:0/551881437 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565400aab0 con 0x7f565c160770 2024-03-29T09:10:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.518+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c159a50 con 0x7f565c160770 2024-03-29T09:10:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.519+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c0b6b50 con 0x7f565c160770 2024-03-29T09:10:08.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.520+0000 7f5658c00700 1 -- 172.21.15.99:0/551881437 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f565400ac10 con 0x7f565c160770 2024-03-29T09:10:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.521+0000 7f5658c00700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f563c06acc0 0x7f563c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.521+0000 7f5658c00700 1 -- 172.21.15.99:0/551881437 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f56540896e0 con 0x7f565c160770 2024-03-29T09:10:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.521+0000 7f5660e00700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f563c06acc0 0x7f563c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.522+0000 7f5660e00700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f563c06acc0 0x7f563c06d180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5650007a30 tx=0x7f5650008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.525+0000 7f5658c00700 1 -- 172.21.15.99:0/551881437 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f565c0b6b50 con 0x7f565c160770 2024-03-29T09:10:08.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.727+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f565c0b6b50 con 0x7f565c160770 2024-03-29T09:10:08.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.728+0000 7f5658c00700 1 -- 172.21.15.99:0/551881437 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f565c0b6b50 con 0x7f565c160770 2024-03-29T09:10:08.728 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:08.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f563c06acc0 msgr2=0x7f563c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f563c06acc0 0x7f563c06d180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5650007a30 tx=0x7f5650008040 comp rx=0 tx=0).stop 2024-03-29T09:10:08.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c160770 msgr2=0x7f565c15a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c160770 0x7f565c15a410 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f565400cc50 tx=0x7f565400cf60 comp rx=0 tx=0).stop 2024-03-29T09:10:08.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 shutdown_connections 2024-03-29T09:10:08.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f563c06acc0 0x7f563c06d180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f565c160770 0x7f565c15a410 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f565c0d0d60 0x7f565c0cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 --2- 172.21.15.99:0/551881437 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f565c0d3380 0x7f565c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 >> 172.21.15.99:0/551881437 conn(0x7f565c08afd0 msgr2=0x7f565c15f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:08.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 shutdown_connections 2024-03-29T09:10:08.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.730+0000 7f5663800700 1 -- 172.21.15.99:0/551881437 wait complete. 2024-03-29T09:10:08.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:08.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.941+0000 7f95abc00700 1 Processor -- start 2024-03-29T09:10:08.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.942+0000 7f95abc00700 1 -- start start 2024-03-29T09:10:08.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95abc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95a40cecc0 0x7f95a4168860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95abc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a40d1be0 con 0x7f95a40cfd70 2024-03-29T09:10:08.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95abc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a40b4980 con 0x7f95a40d0150 2024-03-29T09:10:08.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95abc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a41623c0 con 0x7f95a40cecc0 2024-03-29T09:10:08.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95a9200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f95a40cfd70 0x7f95a41615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34878/0 (socket says 172.21.15.99:34878) 2024-03-29T09:10:08.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95a9200700 1 -- 172.21.15.99:0/3686670318 learned_addr learned my addr 172.21.15.99:0/3686670318 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95a9c00700 1 --2- 172.21.15.99:0/3686670318 >> v2:172.21.15.133:3300/0 conn(0x7f95a40cecc0 0x7f95a4168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620999079 0 0) 0x7f95a40d1be0 con 0x7f95a40cfd70 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.943+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f958c0035b0 con 0x7f95a40cfd70 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295388225 0 0) 0x7f95a40b4980 con 0x7f95a40d0150 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a40d1be0 con 0x7f95a40d0150 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220426777 0 0) 0x7f958c0035b0 con 0x7f95a40cfd70 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 >> v2:172.21.15.133:3300/0 conn(0x7f95a40cecc0 msgr2=0x7f95a4168860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:08.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 --2- 172.21.15.99:0/3686670318 >> v2:172.21.15.133:3300/0 conn(0x7f95a40cecc0 0x7f95a4168860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 >> v1:172.21.15.133:6789/0 conn(0x7f95a40d0150 legacy=0x7f95a4161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a40b46a0 con 0x7f95a40cfd70 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f9598001670 con 0x7f95a40cfd70 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 >> v1:172.21.15.99:6789/0 conn(0x7f95a40cfd70 legacy=0x7f95a41615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c003b90 0x7f958c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f958c004850 0x7f958c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f958c005210 0x7f958c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a41623c0 con 0x7f958c004850 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c00c090 con 0x7f958c003b90 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c00c250 con 0x7f958c005210 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3e00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f958c004850 0x7f958c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a9200700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c003b90 0x7f958c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a9c00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f958c005210 0x7f958c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3e00700 1 -- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f958c005210 msgr2=0x7f958c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3e00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f958c005210 0x7f958c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3e00700 1 -- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c003b90 msgr2=0x7f958c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3e00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c003b90 0x7f958c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a3e00700 1 -- 172.21.15.99:0/3686670318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c0035b0 con 0x7f958c004850 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.944+0000 7f95a9c00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f958c005210 0x7f958c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95a9c00700 1 -- 172.21.15.99:0/3686670318 reap_dead start 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95a3e00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f958c004850 0x7f958c004cd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f95a40d0600 tx=0x7f958c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc2804521dd04c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590001430 con 0x7f958c004850 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95900052d0 con 0x7f958c004850 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95a3400700 1 -- 172.21.15.99:0/3686670318 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590001d40 con 0x7f958c004850 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95abc00700 1 -- 172.21.15.99:0/3686670318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f958c004850 msgr2=0x7f958c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95abc00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f958c004850 0x7f958c004cd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f95a40d0600 tx=0x7f958c005880 comp rx=0 tx=0).stop 2024-03-29T09:10:08.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95abc00700 1 -- 172.21.15.99:0/3686670318 shutdown_connections 2024-03-29T09:10:08.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95abc00700 1 --2- 172.21.15.99:0/3686670318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f958c004850 0x7f958c004cd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.945+0000 7f95abc00700 1 -- 172.21.15.99:0/3686670318 >> 172.21.15.99:0/3686670318 conn(0x7f95a409c4a0 msgr2=0x7f95a40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:08.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.946+0000 7f95abc00700 1 -- 172.21.15.99:0/3686670318 shutdown_connections 2024-03-29T09:10:08.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.946+0000 7f95abc00700 1 -- 172.21.15.99:0/3686670318 wait complete. 2024-03-29T09:10:08.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.947+0000 7f95abc00700 1 Processor -- start 2024-03-29T09:10:08.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.948+0000 7f95abc00700 1 -- start start 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.948+0000 7f95abc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c004850 0x7f95a40cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.948+0000 7f95abc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a40b3a60 0x7f95a41f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.948+0000 7f95abc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a40b7d80 0x7f95a40b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.948+0000 7f95abc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40c8940 con 0x7f95a40b7d80 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.948+0000 7f95abc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40b8740 con 0x7f958c004850 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.948+0000 7f95abc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a40cb7b0 con 0x7f95a40b3a60 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a40b7d80 0x7f95a40b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a40b7d80 0x7f95a40b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36434/0 (socket says 172.21.15.99:36434) 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 -- 172.21.15.99:0/268812934 learned_addr learned my addr 172.21.15.99:0/268812934 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a3e00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a40b3a60 0x7f95a41f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9200700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c004850 0x7f95a40cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 -- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a40b3a60 msgr2=0x7f95a41f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a40b3a60 0x7f95a41f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 -- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c004850 msgr2=0x7f95a40cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c004850 0x7f95a40cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 -- 172.21.15.99:0/268812934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a41f5b00 con 0x7f95a40b7d80 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.949+0000 7f95a9c00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a40b7d80 0x7f95a40b8200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f958c0047d0 tx=0x7f959c0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.950+0000 7f95a0c00700 1 -- 172.21.15.99:0/268812934 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c01b070 con 0x7f95a40b7d80 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.950+0000 7f95a0c00700 1 -- 172.21.15.99:0/268812934 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00f040 con 0x7f95a40b7d80 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.950+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a40cba30 con 0x7f95a40b7d80 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.950+0000 7f95a0c00700 1 -- 172.21.15.99:0/268812934 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00e050 con 0x7f95a40b7d80 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.950+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a40cbff0 con 0x7f95a40b7d80 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.951+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a40677a0 con 0x7f95a40b7d80 2024-03-29T09:10:08.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.952+0000 7f95a0c00700 1 -- 172.21.15.99:0/268812934 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f959c010d60 con 0x7f95a40b7d80 2024-03-29T09:10:08.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.952+0000 7f95a0c00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958406ac90 0x7f958406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:08.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.953+0000 7f95a0c00700 1 -- 172.21.15.99:0/268812934 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f959c089890 con 0x7f95a40b7d80 2024-03-29T09:10:08.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.953+0000 7f95a9200700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958406ac90 0x7f958406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:08.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.953+0000 7f95a9200700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958406ac90 0x7f958406d150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f95a408d5e0 tx=0x7f9598001df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:08.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:08.957+0000 7f95a0c00700 1 -- 172.21.15.99:0/268812934 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f959c054680 con 0x7f95a40b7d80 2024-03-29T09:10:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.179+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95a4067d80 con 0x7f95a40b7d80 2024-03-29T09:10:09.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.179+0000 7f95a0c00700 1 -- 172.21.15.99:0/268812934 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f959c057ca0 con 0x7f95a40b7d80 2024-03-29T09:10:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958406ac90 msgr2=0x7f958406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958406ac90 0x7f958406d150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f95a408d5e0 tx=0x7f9598001df0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a40b7d80 msgr2=0x7f95a40b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a40b7d80 0x7f95a40b8200 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f958c0047d0 tx=0x7f959c0104a0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 shutdown_connections 2024-03-29T09:10:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958406ac90 0x7f958406d150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f95a40b7d80 0x7f95a40b8200 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958c004850 0x7f95a40cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 --2- 172.21.15.99:0/268812934 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a40b3a60 0x7f95a41f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 >> 172.21.15.99:0/268812934 conn(0x7f95a409c4a0 msgr2=0x7f95a415d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 shutdown_connections 2024-03-29T09:10:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.182+0000 7f95abc00700 1 -- 172.21.15.99:0/268812934 wait complete. 2024-03-29T09:10:09.183 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:09.200 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:10:09.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:09.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T09:10:09.266 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:09.266 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:10:09.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:09.267 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:10:09.308 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:10:09.308 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:09.355 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:10:09.355 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:09.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:09.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.629+0000 7f122ac00700 1 Processor -- start 2024-03-29T09:10:09.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.629+0000 7f122ac00700 1 -- start start 2024-03-29T09:10:09.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f122ac00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1224143940 0x7f1224070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:09.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f122ac00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f12240ce2c0 0x7f12240ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:09.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f122ac00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12240cec50 con 0x7f1224070ca0 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f122ac00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1224143d20 con 0x7f1224143940 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f122ac00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122415eaf0 con 0x7f12240ce2c0 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1223e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1224143940 0x7f1224070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1223e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1224143940 0x7f1224070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36448/0 (socket says 172.21.15.99:36448) 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1223e00700 1 -- 172.21.15.99:0/2390785981 learned_addr learned my addr 172.21.15.99:0/2390785981 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1228c00700 1 --2- 172.21.15.99:0/2390785981 >> v2:172.21.15.133:3300/0 conn(0x7f12240ce2c0 0x7f12240ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1223e00700 1 -- 172.21.15.99:0/2390785981 >> v2:172.21.15.133:3300/0 conn(0x7f12240ce2c0 msgr2=0x7f12240ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1223e00700 1 --2- 172.21.15.99:0/2390785981 >> v2:172.21.15.133:3300/0 conn(0x7f12240ce2c0 0x7f12240ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1223e00700 1 -- 172.21.15.99:0/2390785981 >> v1:172.21.15.133:6789/0 conn(0x7f1224070ca0 legacy=0x7f122415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.631+0000 7f1223e00700 1 -- 172.21.15.99:0/2390785981 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240cd820 con 0x7f1224143940 2024-03-29T09:10:09.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.632+0000 7f1223e00700 1 --2- 172.21.15.99:0/2390785981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224143940 0x7f1224070760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f12240b4390 tx=0x7f121400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01e78c0209c7d0e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:09.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.632+0000 7f1222a00700 1 -- 172.21.15.99:0/2390785981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121400dd30 con 0x7f1224143940 2024-03-29T09:10:09.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.632+0000 7f1222a00700 1 -- 172.21.15.99:0/2390785981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121400de90 con 0x7f1224143940 2024-03-29T09:10:09.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.632+0000 7f1222a00700 1 -- 172.21.15.99:0/2390785981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214010690 con 0x7f1224143940 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.632+0000 7f122ac00700 1 -- 172.21.15.99:0/2390785981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224143940 msgr2=0x7f1224070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.632+0000 7f122ac00700 1 --2- 172.21.15.99:0/2390785981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224143940 0x7f1224070760 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f12240b4390 tx=0x7f121400be70 comp rx=0 tx=0).stop 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.633+0000 7f122ac00700 1 -- 172.21.15.99:0/2390785981 shutdown_connections 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.633+0000 7f122ac00700 1 --2- 172.21.15.99:0/2390785981 >> v2:172.21.15.133:3300/0 conn(0x7f12240ce2c0 0x7f12240ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.633+0000 7f122ac00700 1 --2- 172.21.15.99:0/2390785981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224143940 0x7f1224070760 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.633+0000 7f122ac00700 1 -- 172.21.15.99:0/2390785981 >> 172.21.15.99:0/2390785981 conn(0x7f1224066540 msgr2=0x7f1224066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.633+0000 7f122ac00700 1 -- 172.21.15.99:0/2390785981 shutdown_connections 2024-03-29T09:10:09.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.633+0000 7f122ac00700 1 -- 172.21.15.99:0/2390785981 wait complete. 2024-03-29T09:10:09.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.634+0000 7f122ac00700 1 Processor -- start 2024-03-29T09:10:09.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.635+0000 7f122ac00700 1 -- start start 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.635+0000 7f122ac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224070ca0 0x7f12241e3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.635+0000 7f122ac00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12240ce2c0 0x7f1224139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f122ac00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1224143940 0x7f12241e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f122ac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241e6910 con 0x7f1224070ca0 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f122ac00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12241e4190 con 0x7f12240ce2c0 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f122ac00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12241e4470 con 0x7f1224143940 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224070ca0 0x7f12241e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224070ca0 0x7f12241e3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36460/0 (socket says 172.21.15.99:36460) 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 -- 172.21.15.99:0/2445540977 learned_addr learned my addr 172.21.15.99:0/2445540977 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1228c00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1224143940 0x7f12241e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:09.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223400700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12240ce2c0 0x7f1224139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 -- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1224143940 msgr2=0x7f12241e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1224143940 0x7f12241e5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 -- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12240ce2c0 msgr2=0x7f1224139710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12240ce2c0 0x7f1224139710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 -- 172.21.15.99:0/2445540977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121400b7f0 con 0x7f1224070ca0 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1228c00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1224143940 0x7f12241e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223400700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12240ce2c0 0x7f1224139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.636+0000 7f1223e00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224070ca0 0x7f12241e3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1214000c00 tx=0x7f1214004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.637+0000 7f121be00700 1 -- 172.21.15.99:0/2445540977 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214003da0 con 0x7f1224070ca0 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.637+0000 7f121be00700 1 -- 172.21.15.99:0/2445540977 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1214010470 con 0x7f1224070ca0 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.637+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12241e4750 con 0x7f1224070ca0 2024-03-29T09:10:09.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.637+0000 7f121be00700 1 -- 172.21.15.99:0/2445540977 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214010820 con 0x7f1224070ca0 2024-03-29T09:10:09.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.637+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12241e4d10 con 0x7f1224070ca0 2024-03-29T09:10:09.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.638+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12240cd1d0 con 0x7f1224070ca0 2024-03-29T09:10:09.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.639+0000 7f121be00700 1 -- 172.21.15.99:0/2445540977 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1214019400 con 0x7f1224070ca0 2024-03-29T09:10:09.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.640+0000 7f121be00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f120406ac90 0x7f120406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:09.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.640+0000 7f121be00700 1 -- 172.21.15.99:0/2445540977 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f121408aa90 con 0x7f1224070ca0 2024-03-29T09:10:09.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.640+0000 7f1223400700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f120406ac90 0x7f120406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:09.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.641+0000 7f1223400700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f120406ac90 0x7f120406d150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12241e5bc0 tx=0x7f1210009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:09.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.644+0000 7f121be00700 1 -- 172.21.15.99:0/2445540977 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f12140559b0 con 0x7f1224070ca0 2024-03-29T09:10:09.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.846+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12240cdaa0 con 0x7f1224070ca0 2024-03-29T09:10:09.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.848+0000 7f121be00700 1 -- 172.21.15.99:0/2445540977 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f12240cdaa0 con 0x7f1224070ca0 2024-03-29T09:10:09.848 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:09.848 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f120406ac90 msgr2=0x7f120406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f120406ac90 0x7f120406d150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12241e5bc0 tx=0x7f1210009040 comp rx=0 tx=0).stop 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224070ca0 msgr2=0x7f12241e3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224070ca0 0x7f12241e3940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1214000c00 tx=0x7f1214004750 comp rx=0 tx=0).stop 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 shutdown_connections 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f120406ac90 0x7f120406d150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1224143940 0x7f12241e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1224070ca0 0x7f12241e3940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 --2- 172.21.15.99:0/2445540977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12240ce2c0 0x7f1224139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:09.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.850+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 >> 172.21.15.99:0/2445540977 conn(0x7f1224066540 msgr2=0x7f122415cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:09.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.851+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 shutdown_connections 2024-03-29T09:10:09.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:09.851+0000 7f122ac00700 1 -- 172.21.15.99:0/2445540977 wait complete. 2024-03-29T09:10:09.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:09.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:10.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.064+0000 7fca1fa00700 1 Processor -- start 2024-03-29T09:10:10.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.064+0000 7fca1fa00700 1 -- start start 2024-03-29T09:10:10.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.065+0000 7fca1fa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18160770 0x7fca180b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca1fa00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fca180d0d60 0x7fca180d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca1fa00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca180d3810 con 0x7fca180d3380 2024-03-29T09:10:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca1fa00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca180d1520 con 0x7fca180d0d60 2024-03-29T09:10:10.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca1fa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca180d1800 con 0x7fca18160770 2024-03-29T09:10:10.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca17e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fca180d0d60 0x7fca180d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca1d000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18160770 0x7fca180b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca1d000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18160770 0x7fca180b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:39170/0 (socket says 172.21.15.99:39170) 2024-03-29T09:10:10.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.066+0000 7fca1d000700 1 -- 172.21.15.99:0/2494570409 learned_addr learned my addr 172.21.15.99:0/2494570409 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:10.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca1d000700 1 -- 172.21.15.99:0/2494570409 >> v2:172.21.15.133:3300/0 conn(0x7fca180d0d60 msgr2=0x7fca180d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:10.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca1d000700 1 --2- 172.21.15.99:0/2494570409 >> v2:172.21.15.133:3300/0 conn(0x7fca180d0d60 0x7fca180d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca1d000700 1 -- 172.21.15.99:0/2494570409 >> v1:172.21.15.133:6789/0 conn(0x7fca180d3380 legacy=0x7fca180cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca1d000700 1 -- 172.21.15.99:0/2494570409 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180d1ae0 con 0x7fca18160770 2024-03-29T09:10:10.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca1d000700 1 --2- 172.21.15.99:0/2494570409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18160770 0x7fca180b3cc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0098f0 tx=0x7fca0c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a32b2094402ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca17400700 1 -- 172.21.15.99:0/2494570409 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c011830 con 0x7fca18160770 2024-03-29T09:10:10.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca17400700 1 -- 172.21.15.99:0/2494570409 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c011990 con 0x7fca18160770 2024-03-29T09:10:10.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.067+0000 7fca17400700 1 -- 172.21.15.99:0/2494570409 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c011d20 con 0x7fca18160770 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 -- 172.21.15.99:0/2494570409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18160770 msgr2=0x7fca180b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 --2- 172.21.15.99:0/2494570409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18160770 0x7fca180b3cc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0098f0 tx=0x7fca0c009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 -- 172.21.15.99:0/2494570409 shutdown_connections 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 --2- 172.21.15.99:0/2494570409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca18160770 0x7fca180b3cc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 --2- 172.21.15.99:0/2494570409 >> v2:172.21.15.133:3300/0 conn(0x7fca180d0d60 0x7fca180d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 -- 172.21.15.99:0/2494570409 >> 172.21.15.99:0/2494570409 conn(0x7fca1808afd0 msgr2=0x7fca1809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 -- 172.21.15.99:0/2494570409 shutdown_connections 2024-03-29T09:10:10.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.068+0000 7fca1fa00700 1 -- 172.21.15.99:0/2494570409 wait complete. 2024-03-29T09:10:10.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.070+0000 7fca1fa00700 1 Processor -- start 2024-03-29T09:10:10.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.070+0000 7fca1fa00700 1 -- start start 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1fa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca180d0d60 0x7fca180ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1fa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca180d3380 0x7fca1814bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1fa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca18160770 0x7fca1814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1fa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca180b8860 con 0x7fca180d0d60 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1fa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca180ca7a0 con 0x7fca18160770 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1fa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca180ccff0 con 0x7fca180d3380 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca180d0d60 0x7fca180ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca180d0d60 0x7fca180ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36472/0 (socket says 172.21.15.99:36472) 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 -- 172.21.15.99:0/3162460589 learned_addr learned my addr 172.21.15.99:0/3162460589 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca17e00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca180d3380 0x7fca1814bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1da00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca18160770 0x7fca1814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 -- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca180d3380 msgr2=0x7fca1814bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca180d3380 0x7fca1814bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 -- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca18160770 msgr2=0x7fca1814c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca18160770 0x7fca1814c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.071+0000 7fca1d000700 1 -- 172.21.15.99:0/3162460589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca0c009550 con 0x7fca180d0d60 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.072+0000 7fca1d000700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca180d0d60 0x7fca180ca260 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009c40 tx=0x7fca0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.072+0000 7fca14c00700 1 -- 172.21.15.99:0/3162460589 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c0036a0 con 0x7fca180d0d60 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.072+0000 7fca14c00700 1 -- 172.21.15.99:0/3162460589 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c01e070 con 0x7fca180d0d60 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.072+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca180b7c50 con 0x7fca180d0d60 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.072+0000 7fca14c00700 1 -- 172.21.15.99:0/3162460589 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c003a60 con 0x7fca180d0d60 2024-03-29T09:10:10.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.072+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca180b8210 con 0x7fca180d0d60 2024-03-29T09:10:10.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.073+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca180b8a10 con 0x7fca180d0d60 2024-03-29T09:10:10.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.074+0000 7fca14c00700 1 -- 172.21.15.99:0/3162460589 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fca0c003bc0 con 0x7fca180d0d60 2024-03-29T09:10:10.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.075+0000 7fca14c00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc9f806ac90 0x7fc9f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.075+0000 7fca14c00700 1 -- 172.21.15.99:0/3162460589 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fca0c08a540 con 0x7fca180d0d60 2024-03-29T09:10:10.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.075+0000 7fca17e00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc9f806ac90 0x7fc9f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.076+0000 7fca17e00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc9f806ac90 0x7fc9f806d150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca180c9ce0 tx=0x7fca04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.079+0000 7fca14c00700 1 -- 172.21.15.99:0/3162460589 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fca0c055460 con 0x7fca180d0d60 2024-03-29T09:10:10.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.282+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca180678b0 con 0x7fca180d0d60 2024-03-29T09:10:10.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.282+0000 7fca14c00700 1 -- 172.21.15.99:0/3162460589 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca0c058a80 con 0x7fca180d0d60 2024-03-29T09:10:10.282 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.284+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc9f806ac90 msgr2=0x7fc9f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.284+0000 7fca1fa00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc9f806ac90 0x7fc9f806d150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca180c9ce0 tx=0x7fca04008040 comp rx=0 tx=0).stop 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca180d0d60 msgr2=0x7fca180ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca180d0d60 0x7fca180ca260 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009c40 tx=0x7fca0c004750 comp rx=0 tx=0).stop 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 shutdown_connections 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc9f806ac90 0x7fc9f806d150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca18160770 0x7fca1814c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca180d0d60 0x7fca180ca260 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 --2- 172.21.15.99:0/3162460589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca180d3380 0x7fca1814bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 >> 172.21.15.99:0/3162460589 conn(0x7fca1808afd0 msgr2=0x7fca180b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 shutdown_connections 2024-03-29T09:10:10.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.285+0000 7fca1fa00700 1 -- 172.21.15.99:0/3162460589 wait complete. 2024-03-29T09:10:10.303 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:10.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.494+0000 7fb3fa000700 1 Processor -- start 2024-03-29T09:10:10.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.495+0000 7fb3fa000700 1 -- start start 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3fa000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3f40d1760 0x7fb3f40d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3fa000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3f40cdcb0 0x7fb3f40d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3fa000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f4161ae0 con 0x7fb3f40d12b0 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3fa000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f40d0b10 con 0x7fb3f40d1760 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3fa000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f4160f10 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3f3400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3f40d1760 0x7fb3f40d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3f3400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3f40d1760 0x7fb3f40d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36482/0 (socket says 172.21.15.99:36482) 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3f3400700 1 -- 172.21.15.99:0/3413489742 learned_addr learned my addr 172.21.15.99:0/3413489742 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3f2a00700 1 --2- 172.21.15.99:0/3413489742 >> v2:172.21.15.133:3300/0 conn(0x7fb3f40cdcb0 0x7fb3f40d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3f3400700 1 -- 172.21.15.99:0/3413489742 >> v2:172.21.15.133:3300/0 conn(0x7fb3f40cdcb0 msgr2=0x7fb3f40d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.496+0000 7fb3f3400700 1 --2- 172.21.15.99:0/3413489742 >> v2:172.21.15.133:3300/0 conn(0x7fb3f40cdcb0 0x7fb3f40d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3f3400700 1 -- 172.21.15.99:0/3413489742 >> v1:172.21.15.133:6789/0 conn(0x7fb3f40d12b0 legacy=0x7fb3f40b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:10.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3f3400700 1 -- 172.21.15.99:0/3413489742 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f40d2080 con 0x7fb3f40d1760 2024-03-29T09:10:10.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3f3400700 1 --2- 172.21.15.99:0/3413489742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40d1760 0x7fb3f40d1b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb3f408d160 tx=0x7fb3e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84b2072b7550479 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3f2000700 1 -- 172.21.15.99:0/3413489742 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4004230 con 0x7fb3f40d1760 2024-03-29T09:10:10.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3f2000700 1 -- 172.21.15.99:0/3413489742 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e4004390 con 0x7fb3f40d1760 2024-03-29T09:10:10.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3f2000700 1 -- 172.21.15.99:0/3413489742 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e40036a0 con 0x7fb3f40d1760 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3fa000700 1 -- 172.21.15.99:0/3413489742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40d1760 msgr2=0x7fb3f40d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.497+0000 7fb3fa000700 1 --2- 172.21.15.99:0/3413489742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40d1760 0x7fb3f40d1b40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb3f408d160 tx=0x7fb3e4009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.498+0000 7fb3fa000700 1 -- 172.21.15.99:0/3413489742 shutdown_connections 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.498+0000 7fb3fa000700 1 --2- 172.21.15.99:0/3413489742 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40d1760 0x7fb3f40d1b40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.498+0000 7fb3fa000700 1 --2- 172.21.15.99:0/3413489742 >> v2:172.21.15.133:3300/0 conn(0x7fb3f40cdcb0 0x7fb3f40d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.498+0000 7fb3fa000700 1 -- 172.21.15.99:0/3413489742 >> 172.21.15.99:0/3413489742 conn(0x7fb3f408afd0 msgr2=0x7fb3f409c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.498+0000 7fb3fa000700 1 -- 172.21.15.99:0/3413489742 shutdown_connections 2024-03-29T09:10:10.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.498+0000 7fb3fa000700 1 -- 172.21.15.99:0/3413489742 wait complete. 2024-03-29T09:10:10.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.499+0000 7fb3fa000700 1 Processor -- start 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.499+0000 7fb3fa000700 1 -- start start 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.500+0000 7fb3fa000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40cdcb0 0x7fb3f40d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.500+0000 7fb3fa000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f40d12b0 0x7fb3f41516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.500+0000 7fb3fa000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3f40d1760 0x7fb3f40c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.500+0000 7fb3fa000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f415dac0 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.500+0000 7fb3fa000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f4151bf0 con 0x7fb3f40d12b0 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.500+0000 7fb3fa000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f40c7520 con 0x7fb3f40d1760 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40cdcb0 0x7fb3f40d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f2a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f40d12b0 0x7fb3f41516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40cdcb0 0x7fb3f40d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36488/0 (socket says 172.21.15.99:36488) 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 -- 172.21.15.99:0/2151117836 learned_addr learned my addr 172.21.15.99:0/2151117836 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:10.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3e00700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3f40d1760 0x7fb3f40c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 -- 172.21.15.99:0/2151117836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3f40d1760 msgr2=0x7fb3f40c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3f40d1760 0x7fb3f40c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 -- 172.21.15.99:0/2151117836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f40d12b0 msgr2=0x7fb3f41516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f40d12b0 0x7fb3f41516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 -- 172.21.15.99:0/2151117836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4009550 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.501+0000 7fb3f3400700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40cdcb0 0x7fb3f40d0030 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009130 tx=0x7fb3e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.502+0000 7fb3eb400700 1 -- 172.21.15.99:0/2151117836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e4003da0 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.502+0000 7fb3eb400700 1 -- 172.21.15.99:0/2151117836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e4011630 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.502+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f40c7800 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.502+0000 7fb3eb400700 1 -- 172.21.15.99:0/2151117836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e40119e0 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.502+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f40c7210 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.503+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f40c7a40 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.504+0000 7fb3eb400700 1 -- 172.21.15.99:0/2151117836 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb3e4011b40 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.504+0000 7fb3eb400700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb3d406ac90 0x7fb3d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.504+0000 7fb3eb400700 1 -- 172.21.15.99:0/2151117836 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb3e408ab70 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.505+0000 7fb3f2a00700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb3d406ac90 0x7fb3d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.505+0000 7fb3f2a00700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb3d406ac90 0x7fb3d406d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40d0570 tx=0x7fb3e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.509+0000 7fb3eb400700 1 -- 172.21.15.99:0/2151117836 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb3f40c7a40 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.775+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3f415d5b0 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.775+0000 7fb3eb400700 1 -- 172.21.15.99:0/2151117836 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3f415d5b0 con 0x7fb3f40cdcb0 2024-03-29T09:10:10.776 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb3d406ac90 msgr2=0x7fb3d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb3d406ac90 0x7fb3d406d150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40d0570 tx=0x7fb3e0008040 comp rx=0 tx=0).stop 2024-03-29T09:10:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40cdcb0 msgr2=0x7fb3f40d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40cdcb0 0x7fb3f40d0030 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4009130 tx=0x7fb3e4004750 comp rx=0 tx=0).stop 2024-03-29T09:10:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 shutdown_connections 2024-03-29T09:10:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb3d406ac90 0x7fb3d406d150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3f40d1760 0x7fb3f40c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3f40cdcb0 0x7fb3f40d0030 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 --2- 172.21.15.99:0/2151117836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f40d12b0 0x7fb3f41516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 >> 172.21.15.99:0/2151117836 conn(0x7fb3f408afd0 msgr2=0x7fb3f40b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:10.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 shutdown_connections 2024-03-29T09:10:10.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.778+0000 7fb3fa000700 1 -- 172.21.15.99:0/2151117836 wait complete. 2024-03-29T09:10:10.797 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:10.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.989+0000 7f79a5c00700 1 Processor -- start 2024-03-29T09:10:10.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.989+0000 7f79a5c00700 1 -- start start 2024-03-29T09:10:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f79a5c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79a015d800 0x7f79a00d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f79a5c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79a00d1e90 0x7f79a00d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f79a5c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a00d38d0 con 0x7f79a015d420 2024-03-29T09:10:10.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f79a5c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a00d12b0 con 0x7f79a015d800 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f79a5c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a00afdd0 con 0x7f79a00d1e90 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f799e000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f79a015d800 0x7f79a00d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f799ea00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f79a015d420 0x7f79a00d1780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34882/0 (socket says 172.21.15.99:34882) 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.990+0000 7f799ea00700 1 -- 172.21.15.99:0/2664878261 learned_addr learned my addr 172.21.15.99:0/2664878261 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799f400700 1 --2- 172.21.15.99:0/2664878261 >> v2:172.21.15.133:3300/0 conn(0x7f79a00d1e90 0x7f79a00d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799e000700 1 -- 172.21.15.99:0/2664878261 >> v2:172.21.15.133:3300/0 conn(0x7f79a00d1e90 msgr2=0x7f79a00d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799e000700 1 --2- 172.21.15.99:0/2664878261 >> v2:172.21.15.133:3300/0 conn(0x7f79a00d1e90 0x7f79a00d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799e000700 1 -- 172.21.15.99:0/2664878261 >> v1:172.21.15.99:6789/0 conn(0x7f79a015d420 legacy=0x7f79a00d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799e000700 1 -- 172.21.15.99:0/2664878261 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a00d3b00 con 0x7f79a015d800 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799d600700 1 -- 172.21.15.99:0/2664878261 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330879076 0 0) 0x7f79a00d38d0 con 0x7f79a015d420 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799e000700 1 --2- 172.21.15.99:0/2664878261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a015d800 0x7f79a00d2e50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f79a00cdc00 tx=0x7f798c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26379781fe99fc14 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.991+0000 7f799d600700 1 -- 172.21.15.99:0/2664878261 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c011830 con 0x7f79a015d800 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f799d600700 1 -- 172.21.15.99:0/2664878261 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c011990 con 0x7f79a015d800 2024-03-29T09:10:10.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f799d600700 1 -- 172.21.15.99:0/2664878261 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c011d20 con 0x7f79a015d800 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 -- 172.21.15.99:0/2664878261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a015d800 msgr2=0x7f79a00d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 --2- 172.21.15.99:0/2664878261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a015d800 0x7f79a00d2e50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f79a00cdc00 tx=0x7f798c009b20 comp rx=0 tx=0).stop 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 -- 172.21.15.99:0/2664878261 shutdown_connections 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 --2- 172.21.15.99:0/2664878261 >> v2:172.21.15.133:3300/0 conn(0x7f79a00d1e90 0x7f79a00d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 --2- 172.21.15.99:0/2664878261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a015d800 0x7f79a00d2e50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 -- 172.21.15.99:0/2664878261 >> 172.21.15.99:0/2664878261 conn(0x7f79a008afd0 msgr2=0x7f79a009c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 -- 172.21.15.99:0/2664878261 shutdown_connections 2024-03-29T09:10:10.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.992+0000 7f79a5c00700 1 -- 172.21.15.99:0/2664878261 wait complete. 2024-03-29T09:10:10.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.994+0000 7f79a5c00700 1 Processor -- start 2024-03-29T09:10:10.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.994+0000 7f79a5c00700 1 -- start start 2024-03-29T09:10:10.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f79a5c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a00d1e90 0x7f79a00cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f79a5c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a015d420 0x7f79a01f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f79a5c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a01f4330 0x7f79a01f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f79a5c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a00cc8e0 con 0x7f79a01f4330 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f79a5c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a00ca310 con 0x7f79a00d1e90 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f79a5c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a00ca5f0 con 0x7f79a015d420 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a01f4330 0x7f79a01f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a01f4330 0x7f79a01f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36508/0 (socket says 172.21.15.99:36508) 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799f400700 1 -- 172.21.15.99:0/3919250612 learned_addr learned my addr 172.21.15.99:0/3919250612 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799ea00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a00d1e90 0x7f79a00cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799e000700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a015d420 0x7f79a01f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799f400700 1 -- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a015d420 msgr2=0x7f79a01f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799f400700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a015d420 0x7f79a01f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.995+0000 7f799f400700 1 -- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a00d1e90 msgr2=0x7f79a00cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:10.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f799f400700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a00d1e90 0x7f79a00cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f799f400700 1 -- 172.21.15.99:0/3919250612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0094a0 con 0x7f79a01f4330 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f799ea00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a00d1e90 0x7f79a00cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f799f400700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a01f4330 0x7f79a01f3380 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f799800efc0 tx=0x7f799800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f7996a00700 1 -- 172.21.15.99:0/3919250612 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998009ea0 con 0x7f79a01f4330 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f7996a00700 1 -- 172.21.15.99:0/3919250612 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799800f040 con 0x7f79a01f4330 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a00ca8d0 con 0x7f79a01f4330 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f7996a00700 1 -- 172.21.15.99:0/3919250612 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998014480 con 0x7f79a01f4330 2024-03-29T09:10:10.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.996+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a00cae90 con 0x7f79a01f4330 2024-03-29T09:10:10.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.997+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a00cb0b0 con 0x7f79a01f4330 2024-03-29T09:10:10.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.998+0000 7f7996a00700 1 -- 172.21.15.99:0/3919250612 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f79980145e0 con 0x7f79a01f4330 2024-03-29T09:10:10.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.999+0000 7f7996a00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798006ac90 0x7f798006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:10.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.999+0000 7f7996a00700 1 -- 172.21.15.99:0/3919250612 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7998089f10 con 0x7f79a01f4330 2024-03-29T09:10:10.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:10.999+0000 7f799ea00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798006ac90 0x7f798006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.000+0000 7f799ea00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798006ac90 0x7f798006d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7990000f60 tx=0x7f7990007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:11.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.003+0000 7f7996a00700 1 -- 172.21.15.99:0/3919250612 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7998054d90 con 0x7f79a01f4330 2024-03-29T09:10:11.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.211+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79a00747f0 con 0x7f79a01f4330 2024-03-29T09:10:11.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.211+0000 7f7996a00700 1 -- 172.21.15.99:0/3919250612 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7998018020 con 0x7f79a01f4330 2024-03-29T09:10:11.210 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798006ac90 msgr2=0x7f798006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798006ac90 0x7f798006d150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7990000f60 tx=0x7f7990007cc0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a01f4330 msgr2=0x7f79a01f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a01f4330 0x7f79a01f3380 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f799800efc0 tx=0x7f799800c3a0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 shutdown_connections 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f798006ac90 0x7f798006d150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79a01f4330 0x7f79a01f3380 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79a00d1e90 0x7f79a00cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 --2- 172.21.15.99:0/3919250612 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79a015d420 0x7f79a01f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 >> 172.21.15.99:0/3919250612 conn(0x7f79a008afd0 msgr2=0x7f79a006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 shutdown_connections 2024-03-29T09:10:11.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.214+0000 7f79a5c00700 1 -- 172.21.15.99:0/3919250612 wait complete. 2024-03-29T09:10:11.231 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:11.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.424+0000 7f19eee00700 1 Processor -- start 2024-03-29T09:10:11.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.424+0000 7f19eee00700 1 -- start start 2024-03-29T09:10:11.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19eee00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f19e815d420 0x7f19e80d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19eee00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f19e815d800 0x7f19e80d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19eee00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e80afdd0 con 0x7f19e80d2e50 2024-03-29T09:10:11.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19eee00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e80d12b0 con 0x7f19e815d800 2024-03-29T09:10:11.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19eee00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e80d3b00 con 0x7f19e815d420 2024-03-29T09:10:11.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19e7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f19e815d800 0x7f19e80d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19e7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f19e815d800 0x7f19e80d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36516/0 (socket says 172.21.15.99:36516) 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.425+0000 7f19e7400700 1 -- 172.21.15.99:0/1464300903 learned_addr learned my addr 172.21.15.99:0/1464300903 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e7e00700 1 --2- 172.21.15.99:0/1464300903 >> v2:172.21.15.133:3300/0 conn(0x7f19e815d420 0x7f19e80d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e7400700 1 -- 172.21.15.99:0/1464300903 >> v2:172.21.15.133:3300/0 conn(0x7f19e815d420 msgr2=0x7f19e80d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e7400700 1 --2- 172.21.15.99:0/1464300903 >> v2:172.21.15.133:3300/0 conn(0x7f19e815d420 0x7f19e80d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e7400700 1 -- 172.21.15.99:0/1464300903 >> v1:172.21.15.133:6789/0 conn(0x7f19e80d2e50 legacy=0x7f19e80d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e7400700 1 -- 172.21.15.99:0/1464300903 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80b0070 con 0x7f19e815d800 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e7400700 1 --2- 172.21.15.99:0/1464300903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e815d800 0x7f19e80d1cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f19e80d0c30 tx=0x7f19d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a142d3124f55ebce server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e6a00700 1 -- 172.21.15.99:0/1464300903 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4004230 con 0x7f19e815d800 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e6a00700 1 -- 172.21.15.99:0/1464300903 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4004390 con 0x7f19e815d800 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.426+0000 7f19e6a00700 1 -- 172.21.15.99:0/1464300903 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d40036a0 con 0x7f19e815d800 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 -- 172.21.15.99:0/1464300903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e815d800 msgr2=0x7f19e80d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 --2- 172.21.15.99:0/1464300903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e815d800 0x7f19e80d1cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f19e80d0c30 tx=0x7f19d4009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 -- 172.21.15.99:0/1464300903 shutdown_connections 2024-03-29T09:10:11.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 --2- 172.21.15.99:0/1464300903 >> v2:172.21.15.133:3300/0 conn(0x7f19e815d420 0x7f19e80d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 --2- 172.21.15.99:0/1464300903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e815d800 0x7f19e80d1cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 -- 172.21.15.99:0/1464300903 >> 172.21.15.99:0/1464300903 conn(0x7f19e808afd0 msgr2=0x7f19e809c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:11.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 -- 172.21.15.99:0/1464300903 shutdown_connections 2024-03-29T09:10:11.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.427+0000 7f19eee00700 1 -- 172.21.15.99:0/1464300903 wait complete. 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.429+0000 7f19eee00700 1 Processor -- start 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.429+0000 7f19eee00700 1 -- start start 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19eee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e80d2e50 0x7f19e81606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19eee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e815d420 0x7f19e815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19eee00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e80c8a00 0x7f19e815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19eee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80c96c0 con 0x7f19e80d2e50 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19eee00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e815efc0 con 0x7f19e80c8a00 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19eee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e815e130 con 0x7f19e815d420 2024-03-29T09:10:11.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e80d2e50 0x7f19e81606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e80d2e50 0x7f19e81606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36520/0 (socket says 172.21.15.99:36520) 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 -- 172.21.15.99:0/4232722843 learned_addr learned my addr 172.21.15.99:0/4232722843 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7400700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e815d420 0x7f19e815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19ece00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e80c8a00 0x7f19e815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 -- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e815d420 msgr2=0x7f19e815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e815d420 0x7f19e815dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 -- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e80c8a00 msgr2=0x7f19e815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e80c8a00 0x7f19e815adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7e00700 1 -- 172.21.15.99:0/4232722843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4009550 con 0x7f19e80d2e50 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19e7400700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e815d420 0x7f19e815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.430+0000 7f19ece00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e80c8a00 0x7f19e815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.431+0000 7f19e7e00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e80d2e50 0x7f19e81606a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e810 tx=0x7f19d800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:11.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.431+0000 7f19dfe00700 1 -- 172.21.15.99:0/4232722843 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8013610 con 0x7f19e80d2e50 2024-03-29T09:10:11.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.431+0000 7f19dfe00700 1 -- 172.21.15.99:0/4232722843 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d8013770 con 0x7f19e80d2e50 2024-03-29T09:10:11.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.431+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e815e3b0 con 0x7f19e80d2e50 2024-03-29T09:10:11.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.431+0000 7f19dfe00700 1 -- 172.21.15.99:0/4232722843 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8013b20 con 0x7f19e80d2e50 2024-03-29T09:10:11.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.431+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e815e970 con 0x7f19e80d2e50 2024-03-29T09:10:11.433 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.432+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e8073f50 con 0x7f19e80d2e50 2024-03-29T09:10:11.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.433+0000 7f19dfe00700 1 -- 172.21.15.99:0/4232722843 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f19d8013c80 con 0x7f19e80d2e50 2024-03-29T09:10:11.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.434+0000 7f19dfe00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19c806ac90 0x7f19c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.434+0000 7f19dfe00700 1 -- 172.21.15.99:0/4232722843 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f19d8089390 con 0x7f19e80d2e50 2024-03-29T09:10:11.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.434+0000 7f19e7400700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19c806ac90 0x7f19c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.434+0000 7f19e7400700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19c806ac90 0x7f19c806d150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009130 tx=0x7f19d4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:11.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.438+0000 7f19dfe00700 1 -- 172.21.15.99:0/4232722843 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f19d8054190 con 0x7f19e80d2e50 2024-03-29T09:10:11.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.639+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f19e80678b0 con 0x7f19e80d2e50 2024-03-29T09:10:11.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.640+0000 7f19dfe00700 1 -- 172.21.15.99:0/4232722843 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19d80577b0 con 0x7f19e80d2e50 2024-03-29T09:10:11.639 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:11.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19c806ac90 msgr2=0x7f19c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19c806ac90 0x7f19c806d150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009130 tx=0x7f19d4008030 comp rx=0 tx=0).stop 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e80d2e50 msgr2=0x7f19e81606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e80d2e50 0x7f19e81606a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19d800e810 tx=0x7f19d800ebd0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 shutdown_connections 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f19c806ac90 0x7f19c806d150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19e80c8a00 0x7f19e815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19e80d2e50 0x7f19e81606a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 --2- 172.21.15.99:0/4232722843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f19e815d420 0x7f19e815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 >> 172.21.15.99:0/4232722843 conn(0x7f19e808afd0 msgr2=0x7f19e806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 shutdown_connections 2024-03-29T09:10:11.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.642+0000 7f19eee00700 1 -- 172.21.15.99:0/4232722843 wait complete. 2024-03-29T09:10:11.659 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:11.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.851+0000 7f9309e00700 1 Processor -- start 2024-03-29T09:10:11.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.851+0000 7f9309e00700 1 -- start start 2024-03-29T09:10:11.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9309e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93041608e0 0x7f93040d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9309e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9304160cc0 0x7f93040ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9309e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93040cddc0 con 0x7f93040d3a70 2024-03-29T09:10:11.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9309e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93040ce0a0 con 0x7f9304160cc0 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9309e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93040ce380 con 0x7f93041608e0 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9302000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9304160cc0 0x7f93040ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9303400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f93040d3a70 0x7f93040d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34884/0 (socket says 172.21.15.99:34884) 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9303400700 1 -- 172.21.15.99:0/2510791032 learned_addr learned my addr 172.21.15.99:0/2510791032 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9302a00700 1 --2- 172.21.15.99:0/2510791032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93041608e0 0x7f93040d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9302000700 1 -- 172.21.15.99:0/2510791032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93041608e0 msgr2=0x7f93040d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9302000700 1 --2- 172.21.15.99:0/2510791032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93041608e0 0x7f93040d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9302000700 1 -- 172.21.15.99:0/2510791032 >> v1:172.21.15.99:6789/0 conn(0x7f93040d3a70 legacy=0x7f93040d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9302000700 1 -- 172.21.15.99:0/2510791032 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93040ce660 con 0x7f9304160cc0 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.853+0000 7f9302a00700 1 --2- 172.21.15.99:0/2510791032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93041608e0 0x7f93040d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:11.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.854+0000 7f9302000700 1 --2- 172.21.15.99:0/2510791032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304160cc0 0x7f93040ceb80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009840 tx=0x7f92f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91246739958e709 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:11.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.854+0000 7f9301600700 1 -- 172.21.15.99:0/2510791032 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0011830 con 0x7f9304160cc0 2024-03-29T09:10:11.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.854+0000 7f9301600700 1 -- 172.21.15.99:0/2510791032 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f0011990 con 0x7f9304160cc0 2024-03-29T09:10:11.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.854+0000 7f9301600700 1 -- 172.21.15.99:0/2510791032 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0011d20 con 0x7f9304160cc0 2024-03-29T09:10:11.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.854+0000 7f9309e00700 1 -- 172.21.15.99:0/2510791032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304160cc0 msgr2=0x7f93040ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.854+0000 7f9309e00700 1 --2- 172.21.15.99:0/2510791032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304160cc0 0x7f93040ceb80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009840 tx=0x7f92f0009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.855+0000 7f9309e00700 1 -- 172.21.15.99:0/2510791032 shutdown_connections 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.855+0000 7f9309e00700 1 --2- 172.21.15.99:0/2510791032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93041608e0 0x7f93040d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.855+0000 7f9309e00700 1 --2- 172.21.15.99:0/2510791032 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9304160cc0 0x7f93040ceb80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.855+0000 7f9309e00700 1 -- 172.21.15.99:0/2510791032 >> 172.21.15.99:0/2510791032 conn(0x7f930409c490 msgr2=0x7f93040d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.855+0000 7f9309e00700 1 -- 172.21.15.99:0/2510791032 shutdown_connections 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.855+0000 7f9309e00700 1 -- 172.21.15.99:0/2510791032 wait complete. 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.856+0000 7f9309e00700 1 Processor -- start 2024-03-29T09:10:11.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.856+0000 7f9309e00700 1 -- start start 2024-03-29T09:10:11.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.857+0000 7f9309e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93040d3a70 0x7f93040ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.857+0000 7f9309e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041608e0 0x7f93041574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.857+0000 7f9309e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304160cc0 0x7f93040cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.857+0000 7f9309e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304158960 con 0x7f93041608e0 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.857+0000 7f9309e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304157a30 con 0x7f9304160cc0 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.857+0000 7f9309e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9304157ce0 con 0x7f93040d3a70 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041608e0 0x7f93041574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041608e0 0x7f93041574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36542/0 (socket says 172.21.15.99:36542) 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 -- 172.21.15.99:0/514510572 learned_addr learned my addr 172.21.15.99:0/514510572 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 -- 172.21.15.99:0/514510572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93040d3a70 msgr2=0x7f93040ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93040d3a70 0x7f93040ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 -- 172.21.15.99:0/514510572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304160cc0 msgr2=0x7f93040cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304160cc0 0x7f93040cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 -- 172.21.15.99:0/514510572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00094a0 con 0x7f93041608e0 2024-03-29T09:10:11.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f9302000700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041608e0 0x7f93041574f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:11.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.858+0000 7f92fb400700 1 -- 172.21.15.99:0/514510572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f00036a0 con 0x7f93041608e0 2024-03-29T09:10:11.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.859+0000 7f92fb400700 1 -- 172.21.15.99:0/514510572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f001e070 con 0x7f93041608e0 2024-03-29T09:10:11.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.859+0000 7f92fb400700 1 -- 172.21.15.99:0/514510572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0003a60 con 0x7f93041608e0 2024-03-29T09:10:11.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.859+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9304157fc0 con 0x7f93041608e0 2024-03-29T09:10:11.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.859+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9304158580 con 0x7f93041608e0 2024-03-29T09:10:11.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.860+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304158aa0 con 0x7f93041608e0 2024-03-29T09:10:11.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.860+0000 7f92fb400700 1 -- 172.21.15.99:0/514510572 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f92f0003bc0 con 0x7f93041608e0 2024-03-29T09:10:11.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.861+0000 7f92fb400700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92e406ac90 0x7f92e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:11.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.861+0000 7f92fb400700 1 -- 172.21.15.99:0/514510572 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f92f008a770 con 0x7f93041608e0 2024-03-29T09:10:11.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.862+0000 7f9302a00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92e406ac90 0x7f92e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:11.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.862+0000 7f9302a00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92e406ac90 0x7f92e406d150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009990 tx=0x7f92f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:11.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:11.866+0000 7f92fb400700 1 -- 172.21.15.99:0/514510572 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9304158aa0 con 0x7f93041608e0 2024-03-29T09:10:12.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.086+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9304158aa0 con 0x7f93041608e0 2024-03-29T09:10:12.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.086+0000 7f92fb400700 1 -- 172.21.15.99:0/514510572 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9304158aa0 con 0x7f93041608e0 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.088+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92e406ac90 msgr2=0x7f92e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.088+0000 7f9309e00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92e406ac90 0x7f92e406d150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009990 tx=0x7f92f4008040 comp rx=0 tx=0).stop 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.088+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041608e0 msgr2=0x7f93041574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.088+0000 7f9309e00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041608e0 0x7f93041574f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0004750 comp rx=0 tx=0).stop 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.088+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 shutdown_connections 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.089+0000 7f9309e00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92e406ac90 0x7f92e406d150 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.089+0000 7f9309e00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304160cc0 0x7f93040cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.089+0000 7f9309e00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93040d3a70 0x7f93040ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.089+0000 7f9309e00700 1 --2- 172.21.15.99:0/514510572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f93041608e0 0x7f93041574f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.089+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 >> 172.21.15.99:0/514510572 conn(0x7f930409c490 msgr2=0x7f93040b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:12.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.089+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 shutdown_connections 2024-03-29T09:10:12.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.089+0000 7f9309e00700 1 -- 172.21.15.99:0/514510572 wait complete. 2024-03-29T09:10:12.089 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:12.106 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:10:12.106 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:12.152 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:12.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.342+0000 7f4927600700 1 Processor -- start 2024-03-29T09:10:12.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.343+0000 7f4927600700 1 -- start start 2024-03-29T09:10:12.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4927600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49201608e0 0x7f49200d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4927600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4920160cc0 0x7f49200ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4927600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49200cddc0 con 0x7f49200d3a70 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4927600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49200ce0a0 con 0x7f4920160cc0 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4927600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49200ce380 con 0x7f49201608e0 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4925600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f49200d3a70 0x7f49200d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34898/0 (socket says 172.21.15.99:34898) 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4925600700 1 -- 172.21.15.99:0/4279314705 learned_addr learned my addr 172.21.15.99:0/4279314705 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f491fe00700 1 --2- 172.21.15.99:0/4279314705 >> v2:172.21.15.133:3300/0 conn(0x7f4920160cc0 0x7f49200ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f4924c00700 1 --2- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49201608e0 0x7f49200d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f491f400700 1 -- 172.21.15.99:0/4279314705 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195833536 0 0) 0x7f49200cddc0 con 0x7f49200d3a70 2024-03-29T09:10:12.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.344+0000 7f491f400700 1 -- 172.21.15.99:0/4279314705 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49080035b0 con 0x7f49200d3a70 2024-03-29T09:10:12.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491fe00700 1 -- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49201608e0 msgr2=0x7f49200d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:12.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491fe00700 1 --2- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49201608e0 0x7f49200d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491fe00700 1 -- 172.21.15.99:0/4279314705 >> v1:172.21.15.99:6789/0 conn(0x7f49200d3a70 legacy=0x7f49200d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491fe00700 1 -- 172.21.15.99:0/4279314705 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200ce660 con 0x7f4920160cc0 2024-03-29T09:10:12.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491fe00700 1 --2- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 0x7f49200ceb80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f490c009840 tx=0x7f490c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37dc64c56d46ee30 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:12.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491f400700 1 -- 172.21.15.99:0/4279314705 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c011830 con 0x7f4920160cc0 2024-03-29T09:10:12.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491f400700 1 -- 172.21.15.99:0/4279314705 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c011990 con 0x7f4920160cc0 2024-03-29T09:10:12.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.345+0000 7f491f400700 1 -- 172.21.15.99:0/4279314705 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c011d00 con 0x7f4920160cc0 2024-03-29T09:10:12.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 -- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 msgr2=0x7f49200ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 --2- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 0x7f49200ceb80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f490c009840 tx=0x7f490c009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:12.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 -- 172.21.15.99:0/4279314705 shutdown_connections 2024-03-29T09:10:12.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 --2- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49201608e0 0x7f49200d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 --2- 172.21.15.99:0/4279314705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 0x7f49200ceb80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 -- 172.21.15.99:0/4279314705 >> 172.21.15.99:0/4279314705 conn(0x7f492009c490 msgr2=0x7f49200d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:12.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 -- 172.21.15.99:0/4279314705 shutdown_connections 2024-03-29T09:10:12.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.346+0000 7f4927600700 1 -- 172.21.15.99:0/4279314705 wait complete. 2024-03-29T09:10:12.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.348+0000 7f4927600700 1 Processor -- start 2024-03-29T09:10:12.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.348+0000 7f4927600700 1 -- start start 2024-03-29T09:10:12.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4927600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200d3a70 0x7f49200c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4927600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49201608e0 0x7f492006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4927600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 0x7f492006c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4927600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200ccef0 con 0x7f49201608e0 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4927600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200cd1d0 con 0x7f4920160cc0 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4927600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49200cd4b0 con 0x7f49200d3a70 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49201608e0 0x7f492006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49201608e0 0x7f492006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36558/0 (socket says 172.21.15.99:36558) 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 -- 172.21.15.99:0/1789018572 learned_addr learned my addr 172.21.15.99:0/1789018572 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4924c00700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200d3a70 0x7f49200c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f4925600700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 0x7f492006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 -- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200d3a70 msgr2=0x7f49200c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200d3a70 0x7f49200c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 -- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 msgr2=0x7f492006c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 0x7f492006c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.349+0000 7f491fe00700 1 -- 172.21.15.99:0/1789018572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0094a0 con 0x7f49201608e0 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.350+0000 7f491fe00700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49201608e0 0x7f492006bc10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f490c009fd0 tx=0x7f490c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.350+0000 7f491cc00700 1 -- 172.21.15.99:0/1789018572 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c004550 con 0x7f49201608e0 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.350+0000 7f491cc00700 1 -- 172.21.15.99:0/1789018572 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c01e070 con 0x7f49201608e0 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.350+0000 7f491cc00700 1 -- 172.21.15.99:0/1789018572 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0038c0 con 0x7f49201608e0 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.350+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492015a600 con 0x7f49201608e0 2024-03-29T09:10:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.350+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4920156140 con 0x7f49201608e0 2024-03-29T09:10:12.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.351+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49200cf700 con 0x7f49201608e0 2024-03-29T09:10:12.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.352+0000 7f491cc00700 1 -- 172.21.15.99:0/1789018572 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f490c003a20 con 0x7f49201608e0 2024-03-29T09:10:12.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.353+0000 7f491cc00700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f490006ac90 0x7f490006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.353+0000 7f491cc00700 1 -- 172.21.15.99:0/1789018572 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f490c08a600 con 0x7f49201608e0 2024-03-29T09:10:12.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.353+0000 7f4924c00700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f490006ac90 0x7f490006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.353+0000 7f4924c00700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f490006ac90 0x7f490006d150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4914009be0 tx=0x7f4914009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:12.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.357+0000 7f491cc00700 1 -- 172.21.15.99:0/1789018572 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f490c0553f0 con 0x7f49201608e0 2024-03-29T09:10:12.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.561+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49200b2270 con 0x7f49201608e0 2024-03-29T09:10:12.561 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.562+0000 7f491cc00700 1 -- 172.21.15.99:0/1789018572 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f490c058a10 con 0x7f49201608e0 2024-03-29T09:10:12.561 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:12.562 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f490006ac90 msgr2=0x7f490006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f490006ac90 0x7f490006d150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4914009be0 tx=0x7f4914009210 comp rx=0 tx=0).stop 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49201608e0 msgr2=0x7f492006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49201608e0 0x7f492006bc10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f490c009fd0 tx=0x7f490c004750 comp rx=0 tx=0).stop 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 shutdown_connections 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f490006ac90 0x7f490006d150 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4920160cc0 0x7f492006c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200d3a70 0x7f49200c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 --2- 172.21.15.99:0/1789018572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49201608e0 0x7f492006bc10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 >> 172.21.15.99:0/1789018572 conn(0x7f492009c490 msgr2=0x7f49200d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 shutdown_connections 2024-03-29T09:10:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.565+0000 7f4927600700 1 -- 172.21.15.99:0/1789018572 wait complete. 2024-03-29T09:10:12.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:12.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:12.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.776+0000 7f845a600700 1 Processor -- start 2024-03-29T09:10:12.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.776+0000 7f845a600700 1 -- start start 2024-03-29T09:10:12.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.777+0000 7f845a600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8454159e60 0x7f84540cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.777+0000 7f845a600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8454070d30 con 0x7f84540cb200 2024-03-29T09:10:12.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.777+0000 7f845a600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84540ce850 con 0x7f845415a240 2024-03-29T09:10:12.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.777+0000 7f845a600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845415e000 con 0x7f8454159e60 2024-03-29T09:10:12.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8453e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f84540cb200 0x7f84540ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34914/0 (socket says 172.21.15.99:34914) 2024-03-29T09:10:12.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8453e00700 1 -- 172.21.15.99:0/3298080831 learned_addr learned my addr 172.21.15.99:0/3298080831 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:12.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8453400700 1 --2- 172.21.15.99:0/3298080831 >> v2:172.21.15.133:3300/0 conn(0x7f8454159e60 0x7f84540cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8452000700 1 -- 172.21.15.99:0/3298080831 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032286146 0 0) 0x7f8454070d30 con 0x7f84540cb200 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8452000700 1 -- 172.21.15.99:0/3298080831 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84380035b0 con 0x7f84540cb200 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8452000700 1 -- 172.21.15.99:0/3298080831 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271665416 0 0) 0x7f84540ce850 con 0x7f845415a240 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8452000700 1 -- 172.21.15.99:0/3298080831 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8454070d30 con 0x7f845415a240 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8453400700 1 -- 172.21.15.99:0/3298080831 >> v1:172.21.15.133:6789/0 conn(0x7f845415a240 legacy=0x7f84540cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8453400700 1 -- 172.21.15.99:0/3298080831 >> v1:172.21.15.99:6789/0 conn(0x7f84540cb200 legacy=0x7f84540ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.778+0000 7f8453400700 1 -- 172.21.15.99:0/3298080831 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845415e2e0 con 0x7f8454159e60 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f8453400700 1 --2- 172.21.15.99:0/3298080831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8454159e60 0x7f84540cb760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8444009840 tx=0x7f8444009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca1cc07b22f8eef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f8452000700 1 -- 172.21.15.99:0/3298080831 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444011830 con 0x7f8454159e60 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f8452000700 1 -- 172.21.15.99:0/3298080831 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8444011990 con 0x7f8454159e60 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f8452000700 1 -- 172.21.15.99:0/3298080831 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444011d20 con 0x7f8454159e60 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f845a600700 1 -- 172.21.15.99:0/3298080831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8454159e60 msgr2=0x7f84540cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f845a600700 1 --2- 172.21.15.99:0/3298080831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8454159e60 0x7f84540cb760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8444009840 tx=0x7f8444009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f845a600700 1 -- 172.21.15.99:0/3298080831 shutdown_connections 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f845a600700 1 --2- 172.21.15.99:0/3298080831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8454159e60 0x7f84540cb760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.779+0000 7f845a600700 1 -- 172.21.15.99:0/3298080831 >> 172.21.15.99:0/3298080831 conn(0x7f84540b0a20 msgr2=0x7f84540b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:12.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.780+0000 7f845a600700 1 -- 172.21.15.99:0/3298080831 shutdown_connections 2024-03-29T09:10:12.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.780+0000 7f845a600700 1 -- 172.21.15.99:0/3298080831 wait complete. 2024-03-29T09:10:12.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.781+0000 7f845a600700 1 Processor -- start 2024-03-29T09:10:12.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.781+0000 7f845a600700 1 -- start start 2024-03-29T09:10:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.782+0000 7f845a600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cb200 0x7f845415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.782+0000 7f845a600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8454159e60 0x7f8454147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.782+0000 7f845a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f845415a240 0x7f84540b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.782+0000 7f845a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454157fe0 con 0x7f845415a240 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.782+0000 7f845a600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845415b3d0 con 0x7f84540cb200 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.782+0000 7f845a600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845415b6b0 con 0x7f8454159e60 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.782+0000 7f8453e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f845415a240 0x7f84540b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f845415a240 0x7f84540b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36560/0 (socket says 172.21.15.99:36560) 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 -- 172.21.15.99:0/4054825704 learned_addr learned my addr 172.21.15.99:0/4054825704 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453400700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cb200 0x7f845415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8452a00700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8454159e60 0x7f8454147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 -- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8454159e60 msgr2=0x7f8454147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8454159e60 0x7f8454147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 -- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cb200 msgr2=0x7f845415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cb200 0x7f845415c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 -- 172.21.15.99:0/4054825704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440094a0 con 0x7f845415a240 2024-03-29T09:10:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8452a00700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8454159e60 0x7f8454147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f8453e00700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f845415a240 0x7f84540b1d20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f844c00cbc0 tx=0x7f844c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f844b400700 1 -- 172.21.15.99:0/4054825704 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00ab50 con 0x7f845415a240 2024-03-29T09:10:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f844b400700 1 -- 172.21.15.99:0/4054825704 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c00acb0 con 0x7f845415a240 2024-03-29T09:10:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845415b990 con 0x7f845415a240 2024-03-29T09:10:12.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.783+0000 7f844b400700 1 -- 172.21.15.99:0/4054825704 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c012600 con 0x7f845415a240 2024-03-29T09:10:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.784+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845415bf50 con 0x7f845415a240 2024-03-29T09:10:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.785+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845415c170 con 0x7f845415a240 2024-03-29T09:10:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.785+0000 7f844b400700 1 -- 172.21.15.99:0/4054825704 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f844c01b3b0 con 0x7f845415a240 2024-03-29T09:10:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.786+0000 7f844b400700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f843406ac90 0x7f843406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.786+0000 7f844b400700 1 -- 172.21.15.99:0/4054825704 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f844c0893b0 con 0x7f845415a240 2024-03-29T09:10:12.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.786+0000 7f8453400700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f843406ac90 0x7f843406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:12.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.787+0000 7f8453400700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f843406ac90 0x7f843406d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8444009fd0 tx=0x7f844401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:12.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.791+0000 7f844b400700 1 -- 172.21.15.99:0/4054825704 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f844c054250 con 0x7f845415a240 2024-03-29T09:10:12.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.994+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84540678b0 con 0x7f845415a240 2024-03-29T09:10:12.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.995+0000 7f844b400700 1 -- 172.21.15.99:0/4054825704 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f844c057870 con 0x7f845415a240 2024-03-29T09:10:12.994 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:12.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f843406ac90 msgr2=0x7f843406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f843406ac90 0x7f843406d150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8444009fd0 tx=0x7f844401c040 comp rx=0 tx=0).stop 2024-03-29T09:10:12.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f845415a240 msgr2=0x7f84540b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:12.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f845415a240 0x7f84540b1d20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f844c00cbc0 tx=0x7f844c00cf80 comp rx=0 tx=0).stop 2024-03-29T09:10:12.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 shutdown_connections 2024-03-29T09:10:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f843406ac90 0x7f843406d150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f845415a240 0x7f84540b1d20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84540cb200 0x7f845415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 --2- 172.21.15.99:0/4054825704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8454159e60 0x7f8454147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 >> 172.21.15.99:0/4054825704 conn(0x7f84540b0a20 msgr2=0x7f8454070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 shutdown_connections 2024-03-29T09:10:12.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:12.997+0000 7f845a600700 1 -- 172.21.15.99:0/4054825704 wait complete. 2024-03-29T09:10:13.014 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.209+0000 7fafbca00700 1 Processor -- start 2024-03-29T09:10:13.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.209+0000 7fafbca00700 1 -- start start 2024-03-29T09:10:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.210+0000 7fafbca00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb8160770 0x7fafb80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.210+0000 7fafbca00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fafb80d0d60 0x7fafb80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.210+0000 7fafbca00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafb80d3380 0x7fafb80cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.210+0000 7fafbca00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb80d3810 con 0x7fafb80d3380 2024-03-29T09:10:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.210+0000 7fafbca00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb80d1520 con 0x7fafb80d0d60 2024-03-29T09:10:13.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.210+0000 7fafbca00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80d1800 con 0x7fafb8160770 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafb80d3380 0x7fafb80cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafafe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fafb80d0d60 0x7fafb80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb5e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb8160770 0x7fafb80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafafe00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fafb80d0d60 0x7fafb80d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:50058/0 (socket says 172.21.15.99:50058) 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafafe00700 1 -- 172.21.15.99:0/3394437649 learned_addr learned my addr 172.21.15.99:0/3394437649 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fafb80d3380 0x7fafb80cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36568/0 (socket says 172.21.15.99:36568) 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb6800700 1 -- 172.21.15.99:0/3394437649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb8160770 msgr2=0x7fafb80b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb6800700 1 --2- 172.21.15.99:0/3394437649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb8160770 0x7fafb80b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb6800700 1 -- 172.21.15.99:0/3394437649 >> v2:172.21.15.133:3300/0 conn(0x7fafb80d0d60 msgr2=0x7fafb80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:13.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb6800700 1 --2- 172.21.15.99:0/3394437649 >> v2:172.21.15.133:3300/0 conn(0x7fafb80d0d60 0x7fafb80d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb6800700 1 -- 172.21.15.99:0/3394437649 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80d1ae0 con 0x7fafb80d3380 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.211+0000 7fafb5e00700 1 --2- 172.21.15.99:0/3394437649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb8160770 0x7fafb80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.212+0000 7fafb6800700 1 --2- 172.21.15.99:0/3394437649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb80cfa80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fafb80b4be0 tx=0x7fafb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666d8e0d2b5d6811 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.212+0000 7fafb5400700 1 -- 172.21.15.99:0/3394437649 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000e070 con 0x7fafb80d3380 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.212+0000 7fafb5400700 1 -- 172.21.15.99:0/3394437649 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb000bb20 con 0x7fafb80d3380 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.212+0000 7fafb5400700 1 -- 172.21.15.99:0/3394437649 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0012430 con 0x7fafb80d3380 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.212+0000 7fafbca00700 1 -- 172.21.15.99:0/3394437649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 msgr2=0x7fafb80cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.212+0000 7fafbca00700 1 --2- 172.21.15.99:0/3394437649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb80cfa80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fafb80b4be0 tx=0x7fafb000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.213+0000 7fafbca00700 1 -- 172.21.15.99:0/3394437649 shutdown_connections 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.213+0000 7fafbca00700 1 --2- 172.21.15.99:0/3394437649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb8160770 0x7fafb80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.213+0000 7fafbca00700 1 --2- 172.21.15.99:0/3394437649 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb80cfa80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.213+0000 7fafbca00700 1 --2- 172.21.15.99:0/3394437649 >> v2:172.21.15.133:3300/0 conn(0x7fafb80d0d60 0x7fafb80d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.213+0000 7fafbca00700 1 -- 172.21.15.99:0/3394437649 >> 172.21.15.99:0/3394437649 conn(0x7fafb808afd0 msgr2=0x7fafb809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.213+0000 7fafbca00700 1 -- 172.21.15.99:0/3394437649 shutdown_connections 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.213+0000 7fafbca00700 1 -- 172.21.15.99:0/3394437649 wait complete. 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.214+0000 7fafbca00700 1 Processor -- start 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.215+0000 7fafbca00700 1 -- start start 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.215+0000 7fafbca00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb80d0d60 0x7fafb814c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafbca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafbca00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafb8160770 0x7fafb815c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafbca00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80b5d50 con 0x7fafb80d3380 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafbca00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80b5fa0 con 0x7fafb8160770 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafbca00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80b6280 con 0x7fafb80d0d60 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb815bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36572/0 (socket says 172.21.15.99:36572) 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 -- 172.21.15.99:0/2478966773 learned_addr learned my addr 172.21.15.99:0/2478966773 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:13.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafb5e00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb80d0d60 0x7fafb814c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafb6800700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafb8160770 0x7fafb815c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 -- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb80d0d60 msgr2=0x7fafb814c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb80d0d60 0x7fafb814c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 -- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafb8160770 msgr2=0x7fafb815c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafb8160770 0x7fafb815c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 -- 172.21.15.99:0/2478966773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa80094a0 con 0x7fafb80d3380 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafb5e00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb80d0d60 0x7fafb814c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafb6800700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafb8160770 0x7fafb815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.216+0000 7fafafe00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb815bd90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fafa000e910 tx=0x7fafa000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.217+0000 7fafae000700 1 -- 172.21.15.99:0/2478966773 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa00096a0 con 0x7fafb80d3380 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.217+0000 7fafae000700 1 -- 172.21.15.99:0/2478966773 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa000c850 con 0x7fafb80d3380 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.217+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb0009a70 con 0x7fafb80d3380 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.217+0000 7fafae000700 1 -- 172.21.15.99:0/2478966773 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa000cc00 con 0x7fafb80d3380 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.217+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb80b6840 con 0x7fafb80d3380 2024-03-29T09:10:13.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.218+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb80b6e90 con 0x7fafb80d3380 2024-03-29T09:10:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.219+0000 7fafae000700 1 -- 172.21.15.99:0/2478966773 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fafa000cd60 con 0x7fafb80d3380 2024-03-29T09:10:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.220+0000 7fafae000700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7faf9406ac90 0x7faf9406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.220+0000 7fafae000700 1 -- 172.21.15.99:0/2478966773 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fafa0089770 con 0x7fafb80d3380 2024-03-29T09:10:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.220+0000 7fafb5e00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7faf9406ac90 0x7faf9406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.221+0000 7fafb5e00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7faf9406ac90 0x7faf9406d150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009bd0 tx=0x7fafa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:13.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.224+0000 7fafae000700 1 -- 172.21.15.99:0/2478966773 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fafa0054560 con 0x7fafb80d3380 2024-03-29T09:10:13.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.428+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fafb80678b0 con 0x7fafb80d3380 2024-03-29T09:10:13.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.429+0000 7fafae000700 1 -- 172.21.15.99:0/2478966773 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fafa0057b80 con 0x7fafb80d3380 2024-03-29T09:10:13.428 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.431+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7faf9406ac90 msgr2=0x7faf9406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.431+0000 7fafbca00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7faf9406ac90 0x7faf9406d150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fafa8009bd0 tx=0x7fafa8008040 comp rx=0 tx=0).stop 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.431+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 msgr2=0x7fafb815bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.431+0000 7fafbca00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb815bd90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fafa000e910 tx=0x7fafa000ecd0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 shutdown_connections 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7faf9406ac90 0x7faf9406d150 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafb8160770 0x7fafb815c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fafb80d0d60 0x7fafb814c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 --2- 172.21.15.99:0/2478966773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fafb80d3380 0x7fafb815bd90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 >> 172.21.15.99:0/2478966773 conn(0x7fafb808afd0 msgr2=0x7fafb806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:13.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 shutdown_connections 2024-03-29T09:10:13.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.432+0000 7fafbca00700 1 -- 172.21.15.99:0/2478966773 wait complete. 2024-03-29T09:10:13.451 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:13.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.644+0000 7f3544e00700 1 Processor -- start 2024-03-29T09:10:13.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.644+0000 7f3544e00700 1 -- start start 2024-03-29T09:10:13.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f3544e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3540160770 0x7f35400b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f3544e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35400d0d60 0x7f35400d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f3544e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35400d3810 con 0x7f35400d3380 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f3544e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35400d1520 con 0x7f35400d0d60 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f3544e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35400d1800 con 0x7f3540160770 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f353d400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35400d0d60 0x7f35400d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f353d400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35400d0d60 0x7f35400d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36574/0 (socket says 172.21.15.99:36574) 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f353d400700 1 -- 172.21.15.99:0/2702080984 learned_addr learned my addr 172.21.15.99:0/2702080984 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.645+0000 7f353de00700 1 --2- 172.21.15.99:0/2702080984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3540160770 0x7f35400b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353d400700 1 -- 172.21.15.99:0/2702080984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3540160770 msgr2=0x7f35400b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353d400700 1 --2- 172.21.15.99:0/2702080984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3540160770 0x7f35400b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353d400700 1 -- 172.21.15.99:0/2702080984 >> v1:172.21.15.133:6789/0 conn(0x7f35400d3380 legacy=0x7f35400cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353d400700 1 -- 172.21.15.99:0/2702080984 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35400d1ae0 con 0x7f35400d0d60 2024-03-29T09:10:13.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353d400700 1 --2- 172.21.15.99:0/2702080984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35400d0d60 0x7f35400d2e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f352c0098f0 tx=0x7f352c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc302ba38c604b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:13.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353ca00700 1 -- 172.21.15.99:0/2702080984 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00b8a0 con 0x7f35400d0d60 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353ca00700 1 -- 172.21.15.99:0/2702080984 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352c00ba00 con 0x7f35400d0d60 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.646+0000 7f353ca00700 1 -- 172.21.15.99:0/2702080984 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352c00bd90 con 0x7f35400d0d60 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 -- 172.21.15.99:0/2702080984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35400d0d60 msgr2=0x7f35400d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 --2- 172.21.15.99:0/2702080984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35400d0d60 0x7f35400d2e40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f352c0098f0 tx=0x7f352c009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 -- 172.21.15.99:0/2702080984 shutdown_connections 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 --2- 172.21.15.99:0/2702080984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3540160770 0x7f35400b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 --2- 172.21.15.99:0/2702080984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35400d0d60 0x7f35400d2e40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 -- 172.21.15.99:0/2702080984 >> 172.21.15.99:0/2702080984 conn(0x7f354008afd0 msgr2=0x7f354009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:13.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 -- 172.21.15.99:0/2702080984 shutdown_connections 2024-03-29T09:10:13.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.647+0000 7f3544e00700 1 -- 172.21.15.99:0/2702080984 wait complete. 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.648+0000 7f3544e00700 1 Processor -- start 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.649+0000 7f3544e00700 1 -- start start 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.649+0000 7f3544e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35400d0d60 0x7f35400c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.649+0000 7f3544e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35400d3380 0x7f35400b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.649+0000 7f3544e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3540160770 0x7f35400cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.649+0000 7f3544e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3540151e30 con 0x7f3540160770 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f3544e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3540152040 con 0x7f35400d3380 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f3544e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3540152320 con 0x7f35400d0d60 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3540160770 0x7f35400cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3540160770 0x7f35400cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36578/0 (socket says 172.21.15.99:36578) 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 -- 172.21.15.99:0/2005573445 learned_addr learned my addr 172.21.15.99:0/2005573445 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:13.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353d400700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35400d3380 0x7f35400b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353de00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35400d0d60 0x7f35400c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 -- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35400d0d60 msgr2=0x7f35400c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35400d0d60 0x7f35400c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 -- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35400d3380 msgr2=0x7f35400b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35400d3380 0x7f35400b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 -- 172.21.15.99:0/2005573445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c009550 con 0x7f3540160770 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.650+0000 7f353e800700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3540160770 0x7f35400cc430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f353800cc50 tx=0x7f353800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.651+0000 7f3536000700 1 -- 172.21.15.99:0/2005573445 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538003c00 con 0x7f3540160770 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.651+0000 7f3536000700 1 -- 172.21.15.99:0/2005573445 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3538003d60 con 0x7f3540160770 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.651+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3540152600 con 0x7f3540160770 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.651+0000 7f3536000700 1 -- 172.21.15.99:0/2005573445 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800aab0 con 0x7f3540160770 2024-03-29T09:10:13.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.651+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3540152bc0 con 0x7f3540160770 2024-03-29T09:10:13.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.652+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3540152de0 con 0x7f3540160770 2024-03-29T09:10:13.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.653+0000 7f3536000700 1 -- 172.21.15.99:0/2005573445 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f353800ac10 con 0x7f3540160770 2024-03-29T09:10:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.654+0000 7f3536000700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f352006ac90 0x7f352006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.654+0000 7f3536000700 1 -- 172.21.15.99:0/2005573445 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f35380896f0 con 0x7f3540160770 2024-03-29T09:10:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.654+0000 7f353de00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f352006ac90 0x7f352006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:13.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.655+0000 7f353de00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f352006ac90 0x7f352006d150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35400ccff0 tx=0x7f3530008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:13.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.658+0000 7f3536000700 1 -- 172.21.15.99:0/2005573445 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f35380544e0 con 0x7f3540160770 2024-03-29T09:10:13.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.859+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35400678b0 con 0x7f3540160770 2024-03-29T09:10:13.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.859+0000 7f3536000700 1 -- 172.21.15.99:0/2005573445 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3538057b00 con 0x7f3540160770 2024-03-29T09:10:13.858 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.861+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f352006ac90 msgr2=0x7f352006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.861+0000 7f3544e00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f352006ac90 0x7f352006d150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35400ccff0 tx=0x7f3530008040 comp rx=0 tx=0).stop 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.861+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3540160770 msgr2=0x7f35400cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.861+0000 7f3544e00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3540160770 0x7f35400cc430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f353800cc50 tx=0x7f353800cf60 comp rx=0 tx=0).stop 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 shutdown_connections 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f352006ac90 0x7f352006d150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3540160770 0x7f35400cc430 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35400d0d60 0x7f35400c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 --2- 172.21.15.99:0/2005573445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35400d3380 0x7f35400b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:13.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 >> 172.21.15.99:0/2005573445 conn(0x7f354008afd0 msgr2=0x7f354006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:13.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 shutdown_connections 2024-03-29T09:10:13.862 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:13.862+0000 7f3544e00700 1 -- 172.21.15.99:0/2005573445 wait complete. 2024-03-29T09:10:13.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:14.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.072+0000 7fe3ec600700 1 Processor -- start 2024-03-29T09:10:14.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.072+0000 7fe3ec600700 1 -- start start 2024-03-29T09:10:14.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3ec600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3e4160770 0x7fe3e40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3ec600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d3550 0x7fe3e40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3ec600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40d1520 con 0x7fe3e40d0d60 2024-03-29T09:10:14.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3ec600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e40d1800 con 0x7fe3e4160770 2024-03-29T09:10:14.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3ec600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40d1ae0 con 0x7fe3e40d3550 2024-03-29T09:10:14.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3e9c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3e4160770 0x7fe3e40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3e9c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe3e4160770 0x7fe3e40b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36590/0 (socket says 172.21.15.99:36590) 2024-03-29T09:10:14.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3e9c00700 1 -- 172.21.15.99:0/315737998 learned_addr learned my addr 172.21.15.99:0/315737998 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:14.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.073+0000 7fe3ea600700 1 --2- 172.21.15.99:0/315737998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d3550 0x7fe3e40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e9c00700 1 -- 172.21.15.99:0/315737998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d3550 msgr2=0x7fe3e40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:14.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e9c00700 1 --2- 172.21.15.99:0/315737998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d3550 0x7fe3e40ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e9c00700 1 -- 172.21.15.99:0/315737998 >> v1:172.21.15.99:6789/0 conn(0x7fe3e40d0d60 legacy=0x7fe3e40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e9c00700 1 -- 172.21.15.99:0/315737998 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40d1dc0 con 0x7fe3e4160770 2024-03-29T09:10:14.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e9c00700 1 --2- 172.21.15.99:0/315737998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e4160770 0x7fe3e40b3cc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80098f0 tx=0x7fe3d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6068cfdd80e37dc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:14.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e3e00700 1 -- 172.21.15.99:0/315737998 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8011830 con 0x7fe3e4160770 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e3e00700 1 -- 172.21.15.99:0/315737998 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8011990 con 0x7fe3e4160770 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.074+0000 7fe3e3e00700 1 -- 172.21.15.99:0/315737998 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8011d20 con 0x7fe3e4160770 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 -- 172.21.15.99:0/315737998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e4160770 msgr2=0x7fe3e40b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 --2- 172.21.15.99:0/315737998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e4160770 0x7fe3e40b3cc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80098f0 tx=0x7fe3d8009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 -- 172.21.15.99:0/315737998 shutdown_connections 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 --2- 172.21.15.99:0/315737998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d3550 0x7fe3e40ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 --2- 172.21.15.99:0/315737998 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e4160770 0x7fe3e40b3cc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 -- 172.21.15.99:0/315737998 >> 172.21.15.99:0/315737998 conn(0x7fe3e408afd0 msgr2=0x7fe3e409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 -- 172.21.15.99:0/315737998 shutdown_connections 2024-03-29T09:10:14.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.075+0000 7fe3ec600700 1 -- 172.21.15.99:0/315737998 wait complete. 2024-03-29T09:10:14.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.077+0000 7fe3ec600700 1 Processor -- start 2024-03-29T09:10:14.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.077+0000 7fe3ec600700 1 -- start start 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ec600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d0d60 0x7fe3e40b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ec600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e40d3550 0x7fe3e41524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ec600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e4160770 0x7fe3e415e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ec600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40b7870 con 0x7fe3e40d3550 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ec600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40b7b10 con 0x7fe3e4160770 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ec600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40b7df0 con 0x7fe3e40d0d60 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e40d3550 0x7fe3e41524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e40d3550 0x7fe3e41524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36600/0 (socket says 172.21.15.99:36600) 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 -- 172.21.15.99:0/2521946448 learned_addr learned my addr 172.21.15.99:0/2521946448 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9c00700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d0d60 0x7fe3e40b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ea600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e4160770 0x7fe3e415e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 -- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d0d60 msgr2=0x7fe3e40b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d0d60 0x7fe3e40b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 -- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e4160770 msgr2=0x7fe3e415e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e4160770 0x7fe3e415e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9200700 1 -- 172.21.15.99:0/2521946448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d8009550 con 0x7fe3e40d3550 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3e9c00700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d0d60 0x7fe3e40b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.078+0000 7fe3ea600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e4160770 0x7fe3e415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.079+0000 7fe3e9200700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e40d3550 0x7fe3e41524d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c4e0 tx=0x7fe3d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.079+0000 7fe3e1600700 1 -- 172.21.15.99:0/2521946448 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00052d0 con 0x7fe3e40d3550 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.079+0000 7fe3e1600700 1 -- 172.21.15.99:0/2521946448 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0005430 con 0x7fe3e40d3550 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.079+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e40b80d0 con 0x7fe3e40d3550 2024-03-29T09:10:14.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.079+0000 7fe3e1600700 1 -- 172.21.15.99:0/2521946448 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00038c0 con 0x7fe3e40d3550 2024-03-29T09:10:14.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.079+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e40b8690 con 0x7fe3e40d3550 2024-03-29T09:10:14.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.080+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e40678b0 con 0x7fe3e40d3550 2024-03-29T09:10:14.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.081+0000 7fe3e1600700 1 -- 172.21.15.99:0/2521946448 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe3d0003a20 con 0x7fe3e40d3550 2024-03-29T09:10:14.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.082+0000 7fe3e1600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3c406ac90 0x7fe3c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.082+0000 7fe3e1600700 1 -- 172.21.15.99:0/2521946448 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe3d0088bd0 con 0x7fe3e40d3550 2024-03-29T09:10:14.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.082+0000 7fe3e9c00700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3c406ac90 0x7fe3c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.083+0000 7fe3e9c00700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3c406ac90 0x7fe3c406d150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8014040 tx=0x7fe3d8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:14.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.086+0000 7fe3e1600700 1 -- 172.21.15.99:0/2521946448 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe3d00539c0 con 0x7fe3e40d3550 2024-03-29T09:10:14.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.292+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe3e40d3160 con 0x7fe3e40d3550 2024-03-29T09:10:14.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.292+0000 7fe3e1600700 1 -- 172.21.15.99:0/2521946448 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe3d00170a0 con 0x7fe3e40d3550 2024-03-29T09:10:14.291 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:14.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3c406ac90 msgr2=0x7fe3c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3c406ac90 0x7fe3c406d150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8014040 tx=0x7fe3d8008030 comp rx=0 tx=0).stop 2024-03-29T09:10:14.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e40d3550 msgr2=0x7fe3e41524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e40d3550 0x7fe3e41524d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c4e0 tx=0x7fe3d000c8a0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 shutdown_connections 2024-03-29T09:10:14.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3c406ac90 0x7fe3c406d150 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3e4160770 0x7fe3e415e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3e40d0d60 0x7fe3e40b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 --2- 172.21.15.99:0/2521946448 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3e40d3550 0x7fe3e41524d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 >> 172.21.15.99:0/2521946448 conn(0x7fe3e408afd0 msgr2=0x7fe3e415c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:14.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 shutdown_connections 2024-03-29T09:10:14.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.295+0000 7fe3ec600700 1 -- 172.21.15.99:0/2521946448 wait complete. 2024-03-29T09:10:14.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:14.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.503+0000 7f4e53200700 1 Processor -- start 2024-03-29T09:10:14.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.503+0000 7f4e53200700 1 -- start start 2024-03-29T09:10:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e53200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e4c160cc0 0x7f4e4c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e53200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0cddc0 con 0x7f4e4c1608e0 2024-03-29T09:10:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e53200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e4c0ce0a0 con 0x7f4e4c0d1b20 2024-03-29T09:10:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e53200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c0ce380 con 0x7f4e4c160cc0 2024-03-29T09:10:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e4be00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4e4c1608e0 0x7f4e4c0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34938/0 (socket says 172.21.15.99:34938) 2024-03-29T09:10:14.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e4be00700 1 -- 172.21.15.99:0/4210817361 learned_addr learned my addr 172.21.15.99:0/4210817361 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:14.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e51200700 1 --2- 172.21.15.99:0/4210817361 >> v2:172.21.15.133:3300/0 conn(0x7f4e4c160cc0 0x7f4e4c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.504+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235467340 0 0) 0x7f4e4c0cddc0 con 0x7f4e4c1608e0 2024-03-29T09:10:14.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e300035b0 con 0x7f4e4c1608e0 2024-03-29T09:10:14.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3778527076 0 0) 0x7f4e300035b0 con 0x7f4e4c1608e0 2024-03-29T09:10:14.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 >> v2:172.21.15.133:3300/0 conn(0x7f4e4c160cc0 msgr2=0x7f4e4c0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:14.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 --2- 172.21.15.99:0/4210817361 >> v2:172.21.15.133:3300/0 conn(0x7f4e4c160cc0 0x7f4e4c0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 >> v1:172.21.15.133:6789/0 conn(0x7f4e4c0d1b20 legacy=0x7f4e4c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0ce660 con 0x7f4e4c1608e0 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f4e3c001670 con 0x7f4e4c1608e0 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 >> v1:172.21.15.99:6789/0 conn(0x7f4e4c1608e0 legacy=0x7f4e4c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30003a50 0x7f4e30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30004850 0x7f4e30004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e30005210 0x7f4e3000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0ce380 con 0x7f4e30005210 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3000c0f0 con 0x7f4e30003a50 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3000c2b0 con 0x7f4e30004850 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e51200700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e30005210 0x7f4e3000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4be00700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30003a50 0x7f4e30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.505+0000 7f4e4b400700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30004850 0x7f4e30004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e51200700 1 -- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30004850 msgr2=0x7f4e30004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:14.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e51200700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30004850 0x7f4e30004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e51200700 1 -- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30003a50 msgr2=0x7f4e30003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e51200700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e30003a50 0x7f4e30003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e51200700 1 -- 172.21.15.99:0/4210817361 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e300035b0 con 0x7f4e30005210 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e51200700 1 -- 172.21.15.99:0/4210817361 reap_dead start 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e51200700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e30005210 0x7f4e3000ba10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009fc0 tx=0x7f4e30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbf2aac98199d98 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44003a20 con 0x7f4e30005210 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4400f040 con 0x7f4e30005210 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.506+0000 7f4e4aa00700 1 -- 172.21.15.99:0/4210817361 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e44003da0 con 0x7f4e30005210 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.507+0000 7f4e53200700 1 -- 172.21.15.99:0/4210817361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e30005210 msgr2=0x7f4e3000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.507+0000 7f4e53200700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e30005210 0x7f4e3000ba10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009fc0 tx=0x7f4e30005880 comp rx=0 tx=0).stop 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.507+0000 7f4e53200700 1 -- 172.21.15.99:0/4210817361 shutdown_connections 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.507+0000 7f4e53200700 1 --2- 172.21.15.99:0/4210817361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e30005210 0x7f4e3000ba10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.507+0000 7f4e53200700 1 -- 172.21.15.99:0/4210817361 >> 172.21.15.99:0/4210817361 conn(0x7f4e4c09c490 msgr2=0x7f4e4c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:14.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.507+0000 7f4e53200700 1 -- 172.21.15.99:0/4210817361 shutdown_connections 2024-03-29T09:10:14.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.507+0000 7f4e53200700 1 -- 172.21.15.99:0/4210817361 wait complete. 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.509+0000 7f4e53200700 1 Processor -- start 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.509+0000 7f4e53200700 1 -- start start 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.510+0000 7f4e53200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30005210 0x7f4e4c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.510+0000 7f4e53200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0cb870 0x7f4e4c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.510+0000 7f4e53200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c157aa0 0x7f4e4c157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e53200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c15ce30 con 0x7f4e4c0cb870 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e53200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c1f28b0 con 0x7f4e4c157aa0 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e53200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0cc070 con 0x7f4e30005210 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0cb870 0x7f4e4c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0cb870 0x7f4e4c157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36620/0 (socket says 172.21.15.99:36620) 2024-03-29T09:10:14.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 -- 172.21.15.99:0/147895388 learned_addr learned my addr 172.21.15.99:0/147895388 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4be00700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30005210 0x7f4e4c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e51200700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c157aa0 0x7f4e4c157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 -- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30005210 msgr2=0x7f4e4c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30005210 0x7f4e4c15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 -- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c157aa0 msgr2=0x7f4e4c157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c157aa0 0x7f4e4c157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 -- 172.21.15.99:0/147895388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0cc350 con 0x7f4e4c0cb870 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4be00700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30005210 0x7f4e4c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.511+0000 7f4e4b400700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0cb870 0x7f4e4c157560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800f820 tx=0x7f4e3800fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.512+0000 7f4e43e00700 1 -- 172.21.15.99:0/147895388 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380052e0 con 0x7f4e4c0cb870 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.512+0000 7f4e43e00700 1 -- 172.21.15.99:0/147895388 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e38005460 con 0x7f4e4c0cb870 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.512+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c0cc630 con 0x7f4e4c0cb870 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.512+0000 7f4e43e00700 1 -- 172.21.15.99:0/147895388 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380038c0 con 0x7f4e4c0cb870 2024-03-29T09:10:14.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.512+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c0ccbf0 con 0x7f4e4c0cb870 2024-03-29T09:10:14.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.513+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c0ccf60 con 0x7f4e4c0cb870 2024-03-29T09:10:14.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.514+0000 7f4e43e00700 1 -- 172.21.15.99:0/147895388 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4e38003a20 con 0x7f4e4c0cb870 2024-03-29T09:10:14.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.515+0000 7f4e43e00700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:14.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.515+0000 7f4e43e00700 1 -- 172.21.15.99:0/147895388 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4e38089420 con 0x7f4e4c0cb870 2024-03-29T09:10:14.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.515+0000 7f4e4be00700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:14.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.516+0000 7f4e4be00700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:14.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.519+0000 7f4e43e00700 1 -- 172.21.15.99:0/147895388 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4e380542b0 con 0x7f4e4c0cb870 2024-03-29T09:10:14.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.737+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e4c0d1990 con 0x7f4e4c0cb870 2024-03-29T09:10:14.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.738+0000 7f4e43e00700 1 -- 172.21.15.99:0/147895388 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e380578d0 con 0x7f4e4c0cb870 2024-03-29T09:10:14.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 msgr2=0x7f4e2c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c000c00 tx=0x7f4e3c009040 comp rx=0 tx=0).stop 2024-03-29T09:10:14.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0cb870 msgr2=0x7f4e4c157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:14.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0cb870 0x7f4e4c157560 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800f820 tx=0x7f4e3800fbe0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 shutdown_connections 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e2c06ac90 0x7f4e2c06d150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e4c157aa0 0x7f4e4c157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e30005210 0x7f4e4c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 --2- 172.21.15.99:0/147895388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e4c0cb870 0x7f4e4c157560 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 >> 172.21.15.99:0/147895388 conn(0x7f4e4c09c490 msgr2=0x7f4e4c06b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 shutdown_connections 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:14.740+0000 7f4e53200700 1 -- 172.21.15.99:0/147895388 wait complete. 2024-03-29T09:10:14.740 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:14.757 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-03-29T09:10:14.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:14.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-29T09:10:14.832 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:14.832 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:10:14.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:14.832 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T09:10:14.886 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:14.886 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:14.928 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:14.929 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:15.011 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:15.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.241+0000 7f8ab1400700 1 Processor -- start 2024-03-29T09:10:15.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.241+0000 7f8ab1400700 1 -- start start 2024-03-29T09:10:15.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.242+0000 7f8ab1400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8aac1608e0 0x7f8aac0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8ab1400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8aac0d1b20 0x7f8aac0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8ab1400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0cddc0 con 0x7f8aac160cc0 2024-03-29T09:10:15.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8ab1400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0ce0a0 con 0x7f8aac0d1b20 2024-03-29T09:10:15.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8ab1400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0ce380 con 0x7f8aac1608e0 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aab200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8aac0d1b20 0x7f8aac0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aa9e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8aac160cc0 0x7f8aac0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34942/0 (socket says 172.21.15.99:34942) 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aa9e00700 1 -- 172.21.15.99:0/2671698050 learned_addr learned my addr 172.21.15.99:0/2671698050 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aaa800700 1 --2- 172.21.15.99:0/2671698050 >> v2:172.21.15.133:3300/0 conn(0x7f8aac1608e0 0x7f8aac0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aab200700 1 -- 172.21.15.99:0/2671698050 >> v2:172.21.15.133:3300/0 conn(0x7f8aac1608e0 msgr2=0x7f8aac0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aab200700 1 --2- 172.21.15.99:0/2671698050 >> v2:172.21.15.133:3300/0 conn(0x7f8aac1608e0 0x7f8aac0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aab200700 1 -- 172.21.15.99:0/2671698050 >> v1:172.21.15.99:6789/0 conn(0x7f8aac160cc0 legacy=0x7f8aac0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aa9400700 1 -- 172.21.15.99:0/2671698050 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377507858 0 0) 0x7f8aac0cddc0 con 0x7f8aac160cc0 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.243+0000 7f8aab200700 1 -- 172.21.15.99:0/2671698050 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0ce660 con 0x7f8aac0d1b20 2024-03-29T09:10:15.243 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8aab200700 1 --2- 172.21.15.99:0/2671698050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac0d1b20 0x7f8aac0ceb80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8aac08d790 tx=0x7f8aa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2142a404b7d2a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8aa9400700 1 -- 172.21.15.99:0/2671698050 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400e070 con 0x7f8aac0d1b20 2024-03-29T09:10:15.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8aa9400700 1 -- 172.21.15.99:0/2671698050 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400bb20 con 0x7f8aac0d1b20 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8aa9400700 1 -- 172.21.15.99:0/2671698050 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4012430 con 0x7f8aac0d1b20 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8ab1400700 1 -- 172.21.15.99:0/2671698050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac0d1b20 msgr2=0x7f8aac0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8ab1400700 1 --2- 172.21.15.99:0/2671698050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac0d1b20 0x7f8aac0ceb80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8aac08d790 tx=0x7f8aa400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8ab1400700 1 -- 172.21.15.99:0/2671698050 shutdown_connections 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8ab1400700 1 --2- 172.21.15.99:0/2671698050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac0d1b20 0x7f8aac0ceb80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8ab1400700 1 --2- 172.21.15.99:0/2671698050 >> v2:172.21.15.133:3300/0 conn(0x7f8aac1608e0 0x7f8aac0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.244+0000 7f8ab1400700 1 -- 172.21.15.99:0/2671698050 >> 172.21.15.99:0/2671698050 conn(0x7f8aac09c490 msgr2=0x7f8aac0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.245+0000 7f8ab1400700 1 -- 172.21.15.99:0/2671698050 shutdown_connections 2024-03-29T09:10:15.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.245+0000 7f8ab1400700 1 -- 172.21.15.99:0/2671698050 wait complete. 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.246+0000 7f8ab1400700 1 Processor -- start 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.246+0000 7f8ab1400700 1 -- start start 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.247+0000 7f8ab1400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aac0d1b20 0x7f8aac155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.247+0000 7f8ab1400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8aac1608e0 0x7f8aac1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.247+0000 7f8ab1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac160cc0 0x7f8aac158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.247+0000 7f8ab1400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac153c10 con 0x7f8aac160cc0 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.247+0000 7f8ab1400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac158ca0 con 0x7f8aac0d1b20 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.247+0000 7f8ab1400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac156050 con 0x7f8aac1608e0 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac160cc0 0x7f8aac158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aaa800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aac0d1b20 0x7f8aac155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac160cc0 0x7f8aac158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36632/0 (socket says 172.21.15.99:36632) 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 -- 172.21.15.99:0/1264555928 learned_addr learned my addr 172.21.15.99:0/1264555928 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aa9e00700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8aac1608e0 0x7f8aac1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 -- 172.21.15.99:0/1264555928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8aac1608e0 msgr2=0x7f8aac1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8aac1608e0 0x7f8aac1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 -- 172.21.15.99:0/1264555928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aac0d1b20 msgr2=0x7f8aac155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aac0d1b20 0x7f8aac155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 -- 172.21.15.99:0/1264555928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa4009a70 con 0x7f8aac160cc0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aab200700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac160cc0 0x7f8aac158760 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4014040 tx=0x7f8aa4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.248+0000 7f8aa2a00700 1 -- 172.21.15.99:0/1264555928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4012430 con 0x7f8aac160cc0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.249+0000 7f8aa2a00700 1 -- 172.21.15.99:0/1264555928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4020070 con 0x7f8aac160cc0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.249+0000 7f8aa2a00700 1 -- 172.21.15.99:0/1264555928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4004280 con 0x7f8aac160cc0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.249+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac156330 con 0x7f8aac160cc0 2024-03-29T09:10:15.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.249+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac1568f0 con 0x7f8aac160cc0 2024-03-29T09:10:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.250+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac0677a0 con 0x7f8aac160cc0 2024-03-29T09:10:15.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.250+0000 7f8aa2a00700 1 -- 172.21.15.99:0/1264555928 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8aa40043e0 con 0x7f8aac160cc0 2024-03-29T09:10:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.251+0000 7f8aa2a00700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8a8c06acc0 0x7f8a8c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.251+0000 7f8aa2a00700 1 -- 172.21.15.99:0/1264555928 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8aa4022070 con 0x7f8aac160cc0 2024-03-29T09:10:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.252+0000 7f8aaa800700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8a8c06acc0 0x7f8a8c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.252+0000 7f8aaa800700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8a8c06acc0 0x7f8a8c06d180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009be0 tx=0x7f8a9c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.256+0000 7f8aa2a00700 1 -- 172.21.15.99:0/1264555928 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8aa40560f0 con 0x7f8aac160cc0 2024-03-29T09:10:15.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.460+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8aac0b4990 con 0x7f8aac160cc0 2024-03-29T09:10:15.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.462+0000 7f8aa2a00700 1 -- 172.21.15.99:0/1264555928 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f8aa4059710 con 0x7f8aac160cc0 2024-03-29T09:10:15.461 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:15.461 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:15.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8a8c06acc0 msgr2=0x7f8a8c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8a8c06acc0 0x7f8a8c06d180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009be0 tx=0x7f8a9c009210 comp rx=0 tx=0).stop 2024-03-29T09:10:15.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac160cc0 msgr2=0x7f8aac158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac160cc0 0x7f8aac158760 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4014040 tx=0x7f8aa4004ad0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 shutdown_connections 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8a8c06acc0 0x7f8a8c06d180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8aac160cc0 0x7f8aac158760 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aac0d1b20 0x7f8aac155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 --2- 172.21.15.99:0/1264555928 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8aac1608e0 0x7f8aac1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 >> 172.21.15.99:0/1264555928 conn(0x7f8aac09c490 msgr2=0x7f8aac06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 shutdown_connections 2024-03-29T09:10:15.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.464+0000 7f8ab1400700 1 -- 172.21.15.99:0/1264555928 wait complete. 2024-03-29T09:10:15.482 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:15.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.677+0000 7f1105600700 1 Processor -- start 2024-03-29T09:10:15.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.677+0000 7f1105600700 1 -- start start 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.678+0000 7f1105600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f11000cf090 0x7f1100167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.678+0000 7f1105600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11000d1520 con 0x7f11000b3cc0 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.678+0000 7f1105600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11000d1800 con 0x7f1100160770 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.678+0000 7f1105600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11000b4140 con 0x7f11000cf090 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.678+0000 7f10ff400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f11000cf090 0x7f1100167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.678+0000 7f10fe000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f11000b3cc0 0x7f11000d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34946/0 (socket says 172.21.15.99:34946) 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.678+0000 7f10fe000700 1 -- 172.21.15.99:0/2673495214 learned_addr learned my addr 172.21.15.99:0/2673495214 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10ff400700 1 -- 172.21.15.99:0/2673495214 >> v1:172.21.15.133:6789/0 conn(0x7f1100160770 legacy=0x7f11000d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:15.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10ff400700 1 -- 172.21.15.99:0/2673495214 >> v1:172.21.15.99:6789/0 conn(0x7f11000b3cc0 legacy=0x7f11000d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10ff400700 1 -- 172.21.15.99:0/2673495214 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000d1ae0 con 0x7f11000cf090 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10fd600700 1 -- 172.21.15.99:0/2673495214 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306456773 0 0) 0x7f11000d1520 con 0x7f11000b3cc0 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10ff400700 1 --2- 172.21.15.99:0/2673495214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f1100167f40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009d90 tx=0x7f10f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27066a89909b106 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10fd600700 1 -- 172.21.15.99:0/2673495214 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800e070 con 0x7f11000cf090 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10fd600700 1 -- 172.21.15.99:0/2673495214 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f800bb90 con 0x7f11000cf090 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.679+0000 7f10fd600700 1 -- 172.21.15.99:0/2673495214 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8003a20 con 0x7f11000cf090 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.680+0000 7f1105600700 1 -- 172.21.15.99:0/2673495214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 msgr2=0x7f1100167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.680+0000 7f1105600700 1 --2- 172.21.15.99:0/2673495214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f1100167f40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009d90 tx=0x7f10f800b330 comp rx=0 tx=0).stop 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.680+0000 7f1105600700 1 -- 172.21.15.99:0/2673495214 shutdown_connections 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.680+0000 7f1105600700 1 --2- 172.21.15.99:0/2673495214 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f1100167f40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.680+0000 7f1105600700 1 -- 172.21.15.99:0/2673495214 >> 172.21.15.99:0/2673495214 conn(0x7f110008afd0 msgr2=0x7f110009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.680+0000 7f1105600700 1 -- 172.21.15.99:0/2673495214 shutdown_connections 2024-03-29T09:10:15.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.680+0000 7f1105600700 1 -- 172.21.15.99:0/2673495214 wait complete. 2024-03-29T09:10:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.682+0000 7f1105600700 1 Processor -- start 2024-03-29T09:10:15.681 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.682+0000 7f1105600700 1 -- start start 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f1105600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000b3cc0 0x7f11000b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f1105600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f110014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f1105600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1100160770 0x7f110014dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f1105600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000b0060 con 0x7f11000cf090 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f1105600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000b8830 con 0x7f11000b3cc0 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f1105600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000b8b10 con 0x7f1100160770 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f110014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f110014d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36646/0 (socket says 172.21.15.99:36646) 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 -- 172.21.15.99:0/2635541142 learned_addr learned my addr 172.21.15.99:0/2635541142 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10ff400700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1100160770 0x7f110014dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fea00700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000b3cc0 0x7f11000b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 -- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1100160770 msgr2=0x7f110014dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1100160770 0x7f110014dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 -- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000b3cc0 msgr2=0x7f11000b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000b3cc0 0x7f11000b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.683+0000 7f10fe000700 1 -- 172.21.15.99:0/2635541142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8009a70 con 0x7f11000cf090 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.684+0000 7f10fe000700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f110014d6b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0011e0 tx=0x7f10ec00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.684+0000 7f10f6a00700 1 -- 172.21.15.99:0/2635541142 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec004c80 con 0x7f11000cf090 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.684+0000 7f10f6a00700 1 -- 172.21.15.99:0/2635541142 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec004e00 con 0x7f11000cf090 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.684+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11000b8df0 con 0x7f11000cf090 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.684+0000 7f10f6a00700 1 -- 172.21.15.99:0/2635541142 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec007310 con 0x7f11000cf090 2024-03-29T09:10:15.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.684+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11000b84e0 con 0x7f11000cf090 2024-03-29T09:10:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.685+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11000b9010 con 0x7f11000cf090 2024-03-29T09:10:15.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.686+0000 7f10f6a00700 1 -- 172.21.15.99:0/2635541142 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f10ec007470 con 0x7f11000cf090 2024-03-29T09:10:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.687+0000 7f10f6a00700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.687+0000 7f10f6a00700 1 -- 172.21.15.99:0/2635541142 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f10ec089550 con 0x7f11000cf090 2024-03-29T09:10:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.687+0000 7f10fea00700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:15.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.688+0000 7f10fea00700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:15.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.691+0000 7f10f6a00700 1 -- 172.21.15.99:0/2635541142 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f10ec0542c0 con 0x7f11000cf090 2024-03-29T09:10:15.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.894+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11000678b0 con 0x7f11000cf090 2024-03-29T09:10:15.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.895+0000 7f10f6a00700 1 -- 172.21.15.99:0/2635541142 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10ec0578e0 con 0x7f11000cf090 2024-03-29T09:10:15.894 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 msgr2=0x7f10e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f0008040 comp rx=0 tx=0).stop 2024-03-29T09:10:15.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 msgr2=0x7f110014d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f110014d6b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0011e0 tx=0x7f10ec00cae0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 shutdown_connections 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1100160770 0x7f110014dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11000b3cc0 0x7f11000b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 --2- 172.21.15.99:0/2635541142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11000cf090 0x7f110014d6b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 >> 172.21.15.99:0/2635541142 conn(0x7f110008afd0 msgr2=0x7f110015e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 shutdown_connections 2024-03-29T09:10:15.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:15.897+0000 7f1105600700 1 -- 172.21.15.99:0/2635541142 wait complete. 2024-03-29T09:10:15.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:16.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.109+0000 7fb30b800700 1 Processor -- start 2024-03-29T09:10:16.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.110+0000 7fb30b800700 1 -- start start 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb30b800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb3040b3cc0 0x7fb3040d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb30b800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3040d3380 0x7fb3040ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb30b800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3040d3810 con 0x7fb304160770 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb30b800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3040b4140 con 0x7fb3040d3380 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb30b800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3040d1520 con 0x7fb3040b3cc0 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb309800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb3040d3380 0x7fb3040ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb308e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb304160770 0x7fb3040d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34950/0 (socket says 172.21.15.99:34950) 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb308e00700 1 -- 172.21.15.99:0/1939437679 learned_addr learned my addr 172.21.15.99:0/1939437679 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb309800700 1 -- 172.21.15.99:0/1939437679 >> v2:172.21.15.133:3300/0 conn(0x7fb3040b3cc0 msgr2=0x7fb3040d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb309800700 1 --2- 172.21.15.99:0/1939437679 >> v2:172.21.15.133:3300/0 conn(0x7fb3040b3cc0 0x7fb3040d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb309800700 1 -- 172.21.15.99:0/1939437679 >> v1:172.21.15.99:6789/0 conn(0x7fb304160770 legacy=0x7fb3040d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb309800700 1 -- 172.21.15.99:0/1939437679 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3040d1800 con 0x7fb3040d3380 2024-03-29T09:10:16.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.111+0000 7fb303400700 1 -- 172.21.15.99:0/1939437679 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168304553 0 0) 0x7fb3040d3810 con 0x7fb304160770 2024-03-29T09:10:16.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb309800700 1 --2- 172.21.15.99:0/1939437679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3040d3380 0x7fb3040ce080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3040b5cf0 tx=0x7fb2fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e19cb35a678089 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb303400700 1 -- 172.21.15.99:0/1939437679 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00e070 con 0x7fb3040d3380 2024-03-29T09:10:16.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb303400700 1 -- 172.21.15.99:0/1939437679 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc00bbd0 con 0x7fb3040d3380 2024-03-29T09:10:16.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb303400700 1 -- 172.21.15.99:0/1939437679 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc012430 con 0x7fb3040d3380 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb30b800700 1 -- 172.21.15.99:0/1939437679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3040d3380 msgr2=0x7fb3040ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb30b800700 1 --2- 172.21.15.99:0/1939437679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3040d3380 0x7fb3040ce080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb3040b5cf0 tx=0x7fb2fc00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb30b800700 1 -- 172.21.15.99:0/1939437679 shutdown_connections 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb30b800700 1 --2- 172.21.15.99:0/1939437679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3040d3380 0x7fb3040ce080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb30b800700 1 --2- 172.21.15.99:0/1939437679 >> v2:172.21.15.133:3300/0 conn(0x7fb3040b3cc0 0x7fb3040d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.112+0000 7fb30b800700 1 -- 172.21.15.99:0/1939437679 >> 172.21.15.99:0/1939437679 conn(0x7fb30408afd0 msgr2=0x7fb30409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.113+0000 7fb30b800700 1 -- 172.21.15.99:0/1939437679 shutdown_connections 2024-03-29T09:10:16.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.113+0000 7fb30b800700 1 -- 172.21.15.99:0/1939437679 wait complete. 2024-03-29T09:10:16.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.114+0000 7fb30b800700 1 Processor -- start 2024-03-29T09:10:16.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.114+0000 7fb30b800700 1 -- start start 2024-03-29T09:10:16.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.115+0000 7fb30b800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3040b3cc0 0x7fb3040a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.115+0000 7fb30b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3040d3380 0x7fb3040cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.115+0000 7fb30b800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb304160770 0x7fb3040cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.115+0000 7fb30b800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30409dcd0 con 0x7fb304160770 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.115+0000 7fb30b800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041f3460 con 0x7fb3040b3cc0 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.115+0000 7fb30b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3041f5e70 con 0x7fb3040d3380 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb304160770 0x7fb3040cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb308e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3040b3cc0 0x7fb3040a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb304160770 0x7fb3040cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36662/0 (socket says 172.21.15.99:36662) 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 -- 172.21.15.99:0/3783714167 learned_addr learned my addr 172.21.15.99:0/3783714167 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb303e00700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3040d3380 0x7fb3040cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 -- 172.21.15.99:0/3783714167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3040d3380 msgr2=0x7fb3040cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3040d3380 0x7fb3040cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 -- 172.21.15.99:0/3783714167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3040b3cc0 msgr2=0x7fb3040a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3040b3cc0 0x7fb3040a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.116+0000 7fb309800700 1 -- 172.21.15.99:0/3783714167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc009a70 con 0x7fb304160770 2024-03-29T09:10:16.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.117+0000 7fb309800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb304160770 0x7fb3040cfd30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00b370 tx=0x7fb2fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.118+0000 7fb300c00700 1 -- 172.21.15.99:0/3783714167 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc012430 con 0x7fb304160770 2024-03-29T09:10:16.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.118+0000 7fb300c00700 1 -- 172.21.15.99:0/3783714167 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc020070 con 0x7fb304160770 2024-03-29T09:10:16.117 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.118+0000 7fb300c00700 1 -- 172.21.15.99:0/3783714167 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc012ea0 con 0x7fb304160770 2024-03-29T09:10:16.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.118+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3041f6150 con 0x7fb304160770 2024-03-29T09:10:16.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.118+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3041f6950 con 0x7fb304160770 2024-03-29T09:10:16.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.119+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3041f6670 con 0x7fb304160770 2024-03-29T09:10:16.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.120+0000 7fb300c00700 1 -- 172.21.15.99:0/3783714167 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb2fc004280 con 0x7fb304160770 2024-03-29T09:10:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.121+0000 7fb300c00700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2e406ac90 0x7fb2e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.121+0000 7fb300c00700 1 -- 172.21.15.99:0/3783714167 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb2fc022070 con 0x7fb304160770 2024-03-29T09:10:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.121+0000 7fb308e00700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2e406ac90 0x7fb2e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.122+0000 7fb308e00700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2e406ac90 0x7fb2e406d150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000f60 tx=0x7fb2f8007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.125+0000 7fb300c00700 1 -- 172.21.15.99:0/3783714167 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb3041f6670 con 0x7fb304160770 2024-03-29T09:10:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.328+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3041f6670 con 0x7fb304160770 2024-03-29T09:10:16.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.328+0000 7fb300c00700 1 -- 172.21.15.99:0/3783714167 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb3041f6670 con 0x7fb304160770 2024-03-29T09:10:16.327 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2e406ac90 msgr2=0x7fb2e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2e406ac90 0x7fb2e406d150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8000f60 tx=0x7fb2f8007e70 comp rx=0 tx=0).stop 2024-03-29T09:10:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb304160770 msgr2=0x7fb3040cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb304160770 0x7fb3040cfd30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00b370 tx=0x7fb2fc004ad0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 shutdown_connections 2024-03-29T09:10:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2e406ac90 0x7fb2e406d150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb304160770 0x7fb3040cfd30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3040b3cc0 0x7fb3040a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 --2- 172.21.15.99:0/3783714167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3040d3380 0x7fb3040cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 >> 172.21.15.99:0/3783714167 conn(0x7fb30408afd0 msgr2=0x7fb30406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 shutdown_connections 2024-03-29T09:10:16.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.331+0000 7fb30b800700 1 -- 172.21.15.99:0/3783714167 wait complete. 2024-03-29T09:10:16.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:16.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.544+0000 7f6e8d400700 1 Processor -- start 2024-03-29T09:10:16.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.544+0000 7f6e8d400700 1 -- start start 2024-03-29T09:10:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e8d400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e880cf090 0x7f6e88167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e8d400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e880d1520 con 0x7f6e88160770 2024-03-29T09:10:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e8d400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e880d1800 con 0x7f6e880b3cc0 2024-03-29T09:10:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e8d400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e880b4140 con 0x7f6e880cf090 2024-03-29T09:10:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e87200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e880cf090 0x7f6e88167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e87200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6e880cf090 0x7f6e88167f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36664/0 (socket says 172.21.15.99:36664) 2024-03-29T09:10:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e87200700 1 -- 172.21.15.99:0/4151873323 learned_addr learned my addr 172.21.15.99:0/4151873323 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:16.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.545+0000 7f6e86800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6e88160770 0x7f6e880d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34954/0 (socket says 172.21.15.99:34954) 2024-03-29T09:10:16.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e87200700 1 -- 172.21.15.99:0/4151873323 >> v1:172.21.15.133:6789/0 conn(0x7f6e880b3cc0 legacy=0x7f6e880d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:16.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e87200700 1 -- 172.21.15.99:0/4151873323 >> v1:172.21.15.99:6789/0 conn(0x7f6e88160770 legacy=0x7f6e880d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e85400700 1 -- 172.21.15.99:0/4151873323 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523553064 0 0) 0x7f6e880d1520 con 0x7f6e88160770 2024-03-29T09:10:16.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e87200700 1 -- 172.21.15.99:0/4151873323 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e880d1ae0 con 0x7f6e880cf090 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e87200700 1 --2- 172.21.15.99:0/4151873323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880cf090 0x7f6e88167f40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009e50 tx=0x7f6e8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa20164f98d6ff1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e85400700 1 -- 172.21.15.99:0/4151873323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8000e070 con 0x7f6e880cf090 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e85400700 1 -- 172.21.15.99:0/4151873323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8000b9c0 con 0x7f6e880cf090 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.546+0000 7f6e85400700 1 -- 172.21.15.99:0/4151873323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8000bd70 con 0x7f6e880cf090 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.547+0000 7f6e8d400700 1 -- 172.21.15.99:0/4151873323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880cf090 msgr2=0x7f6e88167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.547+0000 7f6e8d400700 1 --2- 172.21.15.99:0/4151873323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880cf090 0x7f6e88167f40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009e50 tx=0x7f6e8000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.547+0000 7f6e8d400700 1 -- 172.21.15.99:0/4151873323 shutdown_connections 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.547+0000 7f6e8d400700 1 --2- 172.21.15.99:0/4151873323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880cf090 0x7f6e88167f40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.547+0000 7f6e8d400700 1 -- 172.21.15.99:0/4151873323 >> 172.21.15.99:0/4151873323 conn(0x7f6e8808afd0 msgr2=0x7f6e8809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.547+0000 7f6e8d400700 1 -- 172.21.15.99:0/4151873323 shutdown_connections 2024-03-29T09:10:16.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.547+0000 7f6e8d400700 1 -- 172.21.15.99:0/4151873323 wait complete. 2024-03-29T09:10:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.549+0000 7f6e8d400700 1 Processor -- start 2024-03-29T09:10:16.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.549+0000 7f6e8d400700 1 -- start start 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e8d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880b3cc0 0x7f6e880c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e8d400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e880cf090 0x7f6e8815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e8d400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e88160770 0x7f6e8815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e8d400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e880ccdb0 con 0x7f6e880b3cc0 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e8d400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e880ccfb0 con 0x7f6e880cf090 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e8d400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8815a470 con 0x7f6e88160770 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880b3cc0 0x7f6e880c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880b3cc0 0x7f6e880c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36678/0 (socket says 172.21.15.99:36678) 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 -- 172.21.15.99:0/3970038216 learned_addr learned my addr 172.21.15.99:0/3970038216 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:16.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 -- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e88160770 msgr2=0x7f6e8815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e88160770 0x7f6e8815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 -- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e880cf090 msgr2=0x7f6e8815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e85e00700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e880cf090 0x7f6e8815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e880cf090 0x7f6e8815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e86800700 1 -- 172.21.15.99:0/3970038216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e80009b30 con 0x7f6e880b3cc0 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.550+0000 7f6e85e00700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e880cf090 0x7f6e8815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.551+0000 7f6e86800700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880b3cc0 0x7f6e880c9430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c520 tx=0x7f6e7800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.551+0000 7f6e7ea00700 1 -- 172.21.15.99:0/3970038216 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78004c80 con 0x7f6e880b3cc0 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.551+0000 7f6e7ea00700 1 -- 172.21.15.99:0/3970038216 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e78004e20 con 0x7f6e880b3cc0 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.551+0000 7f6e7ea00700 1 -- 172.21.15.99:0/3970038216 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78007310 con 0x7f6e880b3cc0 2024-03-29T09:10:16.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.551+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e880cd250 con 0x7f6e880b3cc0 2024-03-29T09:10:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.551+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e880b7120 con 0x7f6e880b3cc0 2024-03-29T09:10:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.552+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e880678b0 con 0x7f6e880b3cc0 2024-03-29T09:10:16.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.553+0000 7f6e7ea00700 1 -- 172.21.15.99:0/3970038216 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6e78007470 con 0x7f6e880b3cc0 2024-03-29T09:10:16.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.554+0000 7f6e7ea00700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e6806acc0 0x7f6e6806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.554+0000 7f6e7ea00700 1 -- 172.21.15.99:0/3970038216 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6e78089390 con 0x7f6e880b3cc0 2024-03-29T09:10:16.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.554+0000 7f6e85e00700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e6806acc0 0x7f6e6806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.554+0000 7f6e85e00700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e6806acc0 0x7f6e6806d180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e880b8e50 tx=0x7f6e74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.558+0000 7f6e7ea00700 1 -- 172.21.15.99:0/3970038216 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6e78054180 con 0x7f6e880b3cc0 2024-03-29T09:10:16.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.758+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e880b72e0 con 0x7f6e880b3cc0 2024-03-29T09:10:16.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.759+0000 7f6e7ea00700 1 -- 172.21.15.99:0/3970038216 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e780577a0 con 0x7f6e880b3cc0 2024-03-29T09:10:16.758 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:16.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e6806acc0 msgr2=0x7f6e6806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e6806acc0 0x7f6e6806d180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e880b8e50 tx=0x7f6e74009040 comp rx=0 tx=0).stop 2024-03-29T09:10:16.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880b3cc0 msgr2=0x7f6e880c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880b3cc0 0x7f6e880c9430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c520 tx=0x7f6e7800c8e0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 shutdown_connections 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6e6806acc0 0x7f6e6806d180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e88160770 0x7f6e8815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6e880b3cc0 0x7f6e880c9430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 --2- 172.21.15.99:0/3970038216 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e880cf090 0x7f6e8815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 >> 172.21.15.99:0/3970038216 conn(0x7f6e8808afd0 msgr2=0x7f6e880cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 shutdown_connections 2024-03-29T09:10:16.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.761+0000 7f6e8d400700 1 -- 172.21.15.99:0/3970038216 wait complete. 2024-03-29T09:10:16.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.972+0000 7f77f7a00700 1 Processor -- start 2024-03-29T09:10:16.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.973+0000 7f77f7a00700 1 -- start start 2024-03-29T09:10:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f7a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77f0160770 0x7f77f00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f7a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f00d0d60 0x7f77f00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f7a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f00d3810 con 0x7f77f00d3380 2024-03-29T09:10:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f7a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f00d1520 con 0x7f77f0160770 2024-03-29T09:10:16.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f7a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00d1800 con 0x7f77f00d0d60 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f5000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77f0160770 0x7f77f00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f5000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f77f0160770 0x7f77f00b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36686/0 (socket says 172.21.15.99:36686) 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f5000700 1 -- 172.21.15.99:0/1019242311 learned_addr learned my addr 172.21.15.99:0/1019242311 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f5000700 1 -- 172.21.15.99:0/1019242311 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f00d0d60 msgr2=0x7f77f00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f5000700 1 --2- 172.21.15.99:0/1019242311 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f00d0d60 0x7f77f00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f5000700 1 -- 172.21.15.99:0/1019242311 >> v1:172.21.15.133:6789/0 conn(0x7f77f00d3380 legacy=0x7f77f00cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.974+0000 7f77f5000700 1 -- 172.21.15.99:0/1019242311 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00d1ae0 con 0x7f77f0160770 2024-03-29T09:10:16.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77f5000700 1 --2- 172.21.15.99:0/1019242311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f0160770 0x7f77f00b3cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f77e40098f0 tx=0x7f77e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2d44a18536e5e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77ef400700 1 -- 172.21.15.99:0/1019242311 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4004230 con 0x7f77f0160770 2024-03-29T09:10:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77ef400700 1 -- 172.21.15.99:0/1019242311 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e4004390 con 0x7f77f0160770 2024-03-29T09:10:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77ef400700 1 -- 172.21.15.99:0/1019242311 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40036a0 con 0x7f77f0160770 2024-03-29T09:10:16.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77f7a00700 1 -- 172.21.15.99:0/1019242311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f0160770 msgr2=0x7f77f00b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77f7a00700 1 --2- 172.21.15.99:0/1019242311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f0160770 0x7f77f00b3cc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f77e40098f0 tx=0x7f77e4009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77f7a00700 1 -- 172.21.15.99:0/1019242311 shutdown_connections 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77f7a00700 1 --2- 172.21.15.99:0/1019242311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f0160770 0x7f77f00b3cc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77f7a00700 1 --2- 172.21.15.99:0/1019242311 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f00d0d60 0x7f77f00d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.975+0000 7f77f7a00700 1 -- 172.21.15.99:0/1019242311 >> 172.21.15.99:0/1019242311 conn(0x7f77f008afd0 msgr2=0x7f77f009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.976+0000 7f77f7a00700 1 -- 172.21.15.99:0/1019242311 shutdown_connections 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.976+0000 7f77f7a00700 1 -- 172.21.15.99:0/1019242311 wait complete. 2024-03-29T09:10:16.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.977+0000 7f77f7a00700 1 Processor -- start 2024-03-29T09:10:16.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.977+0000 7f77f7a00700 1 -- start start 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.978+0000 7f77f7a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f00d0d60 0x7f77f00c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.978+0000 7f77f7a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f00d3380 0x7f77f015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.978+0000 7f77f7a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f0160770 0x7f77f00cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.978+0000 7f77f7a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00c9250 con 0x7f77f00d0d60 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.978+0000 7f77f7a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f015beb0 con 0x7f77f00d3380 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.978+0000 7f77f7a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f015c190 con 0x7f77f0160770 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f00d0d60 0x7f77f00c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f00d0d60 0x7f77f00c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36690/0 (socket says 172.21.15.99:36690) 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 -- 172.21.15.99:0/771141908 learned_addr learned my addr 172.21.15.99:0/771141908 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 -- 172.21.15.99:0/771141908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f0160770 msgr2=0x7f77f00cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f0160770 0x7f77f00cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 -- 172.21.15.99:0/771141908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f00d3380 msgr2=0x7f77f015b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f00d3380 0x7f77f015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 -- 172.21.15.99:0/771141908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4009550 con 0x7f77f00d0d60 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77f5000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f00d0d60 0x7f77f00c8d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009130 tx=0x7f77e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77ecc00700 1 -- 172.21.15.99:0/771141908 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e400b860 con 0x7f77f00d0d60 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77ecc00700 1 -- 172.21.15.99:0/771141908 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e400b9e0 con 0x7f77f00d0d60 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.979+0000 7f77ecc00700 1 -- 172.21.15.99:0/771141908 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e400bd90 con 0x7f77f00d0d60 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.980+0000 7f77f7a00700 1 -- 172.21.15.99:0/771141908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f015c470 con 0x7f77f00d0d60 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.980+0000 7f77f7a00700 1 -- 172.21.15.99:0/771141908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f015ca30 con 0x7f77f00d0d60 2024-03-29T09:10:16.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.981+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f00678b0 con 0x7f77f00d0d60 2024-03-29T09:10:16.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.982+0000 7f77ecc00700 1 -- 172.21.15.99:0/771141908 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f77e401b440 con 0x7f77f00d0d60 2024-03-29T09:10:16.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.983+0000 7f77ecc00700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f77d006ac90 0x7f77d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:16.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.983+0000 7f77ecc00700 1 -- 172.21.15.99:0/771141908 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f77e408b090 con 0x7f77f00d0d60 2024-03-29T09:10:16.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.983+0000 7f77efe00700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f77d006ac90 0x7f77d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:16.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.984+0000 7f77efe00700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f77d006ac90 0x7f77d006d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77f00b8cf0 tx=0x7f77dc006d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:16.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:16.987+0000 7f77ecc00700 1 -- 172.21.15.99:0/771141908 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f77e4055fb0 con 0x7f77f00d0d60 2024-03-29T09:10:17.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.192+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f77f00d0ab0 con 0x7f77f00d0d60 2024-03-29T09:10:17.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.192+0000 7f77ecc00700 1 -- 172.21.15.99:0/771141908 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f77e40595d0 con 0x7f77f00d0d60 2024-03-29T09:10:17.191 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f77d006ac90 msgr2=0x7f77d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f77d006ac90 0x7f77d006d150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77f00b8cf0 tx=0x7f77dc006d20 comp rx=0 tx=0).stop 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f00d0d60 msgr2=0x7f77f00c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f00d0d60 0x7f77f00c8d10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009130 tx=0x7f77e4004750 comp rx=0 tx=0).stop 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 shutdown_connections 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f77d006ac90 0x7f77d006d150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77f0160770 0x7f77f00cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77f00d0d60 0x7f77f00c8d10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 --2- 172.21.15.99:0/771141908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77f00d3380 0x7f77f015b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 >> 172.21.15.99:0/771141908 conn(0x7f77f008afd0 msgr2=0x7f77f006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:17.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 shutdown_connections 2024-03-29T09:10:17.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.195+0000 7f77e2000700 1 -- 172.21.15.99:0/771141908 wait complete. 2024-03-29T09:10:17.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:17.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.405+0000 7f4ffb200700 1 Processor -- start 2024-03-29T09:10:17.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.405+0000 7f4ffb200700 1 -- start start 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ffb200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4ff4160cc0 0x7f4ff40d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ffb200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40cddc0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ffb200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40ce0a0 con 0x7f4ff41608e0 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ffb200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff40ce380 con 0x7f4ff4160cc0 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ff9200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4ff4160cc0 0x7f4ff40d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ff9200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4ff4160cc0 0x7f4ff40d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36706/0 (socket says 172.21.15.99:36706) 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ff3400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4ff40d1b20 0x7f4ff40afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:34968/0 (socket says 172.21.15.99:34968) 2024-03-29T09:10:17.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.406+0000 7f4ff9200700 1 -- 172.21.15.99:0/1676650597 learned_addr learned my addr 172.21.15.99:0/1676650597 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff9200700 1 -- 172.21.15.99:0/1676650597 >> v1:172.21.15.133:6789/0 conn(0x7f4ff41608e0 legacy=0x7f4ff40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff9200700 1 -- 172.21.15.99:0/1676650597 >> v1:172.21.15.99:6789/0 conn(0x7f4ff40d1b20 legacy=0x7f4ff40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff9200700 1 -- 172.21.15.99:0/1676650597 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40ce660 con 0x7f4ff4160cc0 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff2a00700 1 -- 172.21.15.99:0/1676650597 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798183370 0 0) 0x7f4ff40cddc0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff9200700 1 --2- 172.21.15.99:0/1676650597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff4160cc0 0x7f4ff40d0150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009e50 tx=0x7f4fec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31425001ac1c9ac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff2a00700 1 -- 172.21.15.99:0/1676650597 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00e070 con 0x7f4ff4160cc0 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff2a00700 1 -- 172.21.15.99:0/1676650597 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec00b9c0 con 0x7f4ff4160cc0 2024-03-29T09:10:17.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.407+0000 7f4ff2a00700 1 -- 172.21.15.99:0/1676650597 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00bd70 con 0x7f4ff4160cc0 2024-03-29T09:10:17.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.408+0000 7f4ffb200700 1 -- 172.21.15.99:0/1676650597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff4160cc0 msgr2=0x7f4ff40d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.408+0000 7f4ffb200700 1 --2- 172.21.15.99:0/1676650597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff4160cc0 0x7f4ff40d0150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009e50 tx=0x7f4fec00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:17.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.408+0000 7f4ffb200700 1 -- 172.21.15.99:0/1676650597 shutdown_connections 2024-03-29T09:10:17.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.408+0000 7f4ffb200700 1 --2- 172.21.15.99:0/1676650597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff4160cc0 0x7f4ff40d0150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.408+0000 7f4ffb200700 1 -- 172.21.15.99:0/1676650597 >> 172.21.15.99:0/1676650597 conn(0x7f4ff409c490 msgr2=0x7f4ff40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:17.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.408+0000 7f4ffb200700 1 -- 172.21.15.99:0/1676650597 shutdown_connections 2024-03-29T09:10:17.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.408+0000 7f4ffb200700 1 -- 172.21.15.99:0/1676650597 wait complete. 2024-03-29T09:10:17.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.410+0000 7f4ffb200700 1 Processor -- start 2024-03-29T09:10:17.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.410+0000 7f4ffb200700 1 -- start start 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ffb200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff40d1b20 0x7f4ff41f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ffb200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ff41608e0 0x7f4ff4156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ffb200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ff4160cc0 0x7f4ff40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ffb200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff41f6640 con 0x7f4ff40d1b20 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ffb200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff41582e0 con 0x7f4ff4160cc0 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ffb200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff41585c0 con 0x7f4ff41608e0 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ff3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff40d1b20 0x7f4ff41f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ff3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff40d1b20 0x7f4ff41f6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36708/0 (socket says 172.21.15.99:36708) 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ff3e00700 1 -- 172.21.15.99:0/2240832973 learned_addr learned my addr 172.21.15.99:0/2240832973 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ff9200700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ff4160cc0 0x7f4ff40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.411+0000 7f4ff3400700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ff41608e0 0x7f4ff4156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ff3e00700 1 -- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ff41608e0 msgr2=0x7f4ff4156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ff3e00700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ff41608e0 0x7f4ff4156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ff3e00700 1 -- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ff4160cc0 msgr2=0x7f4ff40b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ff3e00700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ff4160cc0 0x7f4ff40b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ff3e00700 1 -- 172.21.15.99:0/2240832973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec009b30 con 0x7f4ff40d1b20 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ff3e00700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff40d1b20 0x7f4ff41f6ad0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c4e0 tx=0x7f4fe400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4febe00700 1 -- 172.21.15.99:0/2240832973 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40052d0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4febe00700 1 -- 172.21.15.99:0/2240832973 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4005430 con 0x7f4ff40d1b20 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4febe00700 1 -- 172.21.15.99:0/2240832973 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40038c0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff41f62d0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.412+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff41f7010 con 0x7f4ff40d1b20 2024-03-29T09:10:17.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.413+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff40677a0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.414+0000 7f4febe00700 1 -- 172.21.15.99:0/2240832973 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4fe4003a20 con 0x7f4ff40d1b20 2024-03-29T09:10:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.415+0000 7f4febe00700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4fd406ace0 0x7f4fd406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.416+0000 7f4febe00700 1 -- 172.21.15.99:0/2240832973 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4fe40889d0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.416+0000 7f4ff3400700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4fd406ace0 0x7f4fd406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.416+0000 7f4ff3400700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4fd406ace0 0x7f4fd406d1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0007b10 tx=0x7f4fe0007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:17.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.421+0000 7f4febe00700 1 -- 172.21.15.99:0/2240832973 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4fe40538d0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.639+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ff40d0700 con 0x7f4ff40d1b20 2024-03-29T09:10:17.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.639+0000 7f4febe00700 1 -- 172.21.15.99:0/2240832973 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4fe4056ef0 con 0x7f4ff40d1b20 2024-03-29T09:10:17.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4fd406ace0 msgr2=0x7f4fd406d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4fd406ace0 0x7f4fd406d1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0007b10 tx=0x7f4fe0007a60 comp rx=0 tx=0).stop 2024-03-29T09:10:17.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff40d1b20 msgr2=0x7f4ff41f6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff40d1b20 0x7f4ff41f6ad0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c4e0 tx=0x7f4fe400c7f0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 shutdown_connections 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4fd406ace0 0x7f4fd406d1a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ff4160cc0 0x7f4ff40b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ff40d1b20 0x7f4ff41f6ad0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 --2- 172.21.15.99:0/2240832973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ff41608e0 0x7f4ff4156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 >> 172.21.15.99:0/2240832973 conn(0x7f4ff409c490 msgr2=0x7f4ff406cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 shutdown_connections 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.642+0000 7f4ffb200700 1 -- 172.21.15.99:0/2240832973 wait complete. 2024-03-29T09:10:17.642 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:17.659 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:17.660 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:17.701 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:17.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.933+0000 7f6632800700 1 Processor -- start 2024-03-29T09:10:17.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.933+0000 7f6632800700 1 -- start start 2024-03-29T09:10:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f6632800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f662c1608e0 0x7f662c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f6632800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f6632800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662c0cddc0 con 0x7f662c160cc0 2024-03-29T09:10:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f6632800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662c0ce0a0 con 0x7f662c1608e0 2024-03-29T09:10:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f6632800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0ce380 con 0x7f662c0d1b20 2024-03-29T09:10:17.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f662b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f662c1608e0 0x7f662c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f662aa00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f662c160cc0 0x7f662c0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52570/0 (socket says 172.21.15.99:52570) 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f662b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f662c1608e0 0x7f662c0d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35412/0 (socket says 172.21.15.99:35412) 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.934+0000 7f662aa00700 1 -- 172.21.15.99:0/3485918982 learned_addr learned my addr 172.21.15.99:0/3485918982 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662be00700 1 --2- 172.21.15.99:0/3485918982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662b400700 1 -- 172.21.15.99:0/3485918982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 msgr2=0x7f662c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662b400700 1 --2- 172.21.15.99:0/3485918982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662b400700 1 -- 172.21.15.99:0/3485918982 >> v1:172.21.15.99:6789/0 conn(0x7f662c160cc0 legacy=0x7f662c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662b400700 1 -- 172.21.15.99:0/3485918982 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c0ce660 con 0x7f662c1608e0 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662be00700 1 --2- 172.21.15.99:0/3485918982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662b400700 1 --2- 172.21.15.99:0/3485918982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c1608e0 0x7f662c0d0150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f661c009840 tx=0x7f661c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167963369cdc6d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662a000700 1 -- 172.21.15.99:0/3485918982 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011830 con 0x7f662c1608e0 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662a000700 1 -- 172.21.15.99:0/3485918982 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c011990 con 0x7f662c1608e0 2024-03-29T09:10:17.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.935+0000 7f662a000700 1 -- 172.21.15.99:0/3485918982 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c011d20 con 0x7f662c1608e0 2024-03-29T09:10:17.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 -- 172.21.15.99:0/3485918982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c1608e0 msgr2=0x7f662c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 --2- 172.21.15.99:0/3485918982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c1608e0 0x7f662c0d0150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f661c009840 tx=0x7f661c009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 -- 172.21.15.99:0/3485918982 shutdown_connections 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 --2- 172.21.15.99:0/3485918982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 --2- 172.21.15.99:0/3485918982 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c1608e0 0x7f662c0d0150 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 -- 172.21.15.99:0/3485918982 >> 172.21.15.99:0/3485918982 conn(0x7f662c09c490 msgr2=0x7f662c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 -- 172.21.15.99:0/3485918982 shutdown_connections 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.936+0000 7f6632800700 1 -- 172.21.15.99:0/3485918982 wait complete. 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.938+0000 7f6632800700 1 Processor -- start 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.938+0000 7f6632800700 1 -- start start 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f6632800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c1559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f6632800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f662c1608e0 0x7f662c0cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f6632800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c160cc0 0x7f662c1f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f6632800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0cc760 con 0x7f662c160cc0 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f6632800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15ac30 con 0x7f662c1608e0 2024-03-29T09:10:17.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f6632800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15af10 con 0x7f662c0d1b20 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c160cc0 0x7f662c1f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c160cc0 0x7f662c1f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35414/0 (socket says 172.21.15.99:35414) 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 -- 172.21.15.99:0/1014961689 learned_addr learned my addr 172.21.15.99:0/1014961689 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662b400700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c1559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662aa00700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f662c1608e0 0x7f662c0cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 -- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 msgr2=0x7f662c1559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c1559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 -- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f662c1608e0 msgr2=0x7f662c0cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f662c1608e0 0x7f662c0cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.939+0000 7f662be00700 1 -- 172.21.15.99:0/1014961689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0094a0 con 0x7f662c160cc0 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.940+0000 7f662be00700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c160cc0 0x7f662c1f31d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6624009f70 tx=0x7f662400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.940+0000 7f6623400700 1 -- 172.21.15.99:0/1014961689 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624003a20 con 0x7f662c160cc0 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.940+0000 7f6623400700 1 -- 172.21.15.99:0/1014961689 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662400f040 con 0x7f662c160cc0 2024-03-29T09:10:17.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.940+0000 7f6623400700 1 -- 172.21.15.99:0/1014961689 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624003de0 con 0x7f662c160cc0 2024-03-29T09:10:17.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.940+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662c15b1f0 con 0x7f662c160cc0 2024-03-29T09:10:17.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.940+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c0cc3a0 con 0x7f662c160cc0 2024-03-29T09:10:17.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.941+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c15b710 con 0x7f662c160cc0 2024-03-29T09:10:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.942+0000 7f6623400700 1 -- 172.21.15.99:0/1014961689 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6624004030 con 0x7f662c160cc0 2024-03-29T09:10:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.943+0000 7f6623400700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f660c06ac90 0x7f660c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.943+0000 7f6623400700 1 -- 172.21.15.99:0/1014961689 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6624089840 con 0x7f662c160cc0 2024-03-29T09:10:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.943+0000 7f662b400700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f660c06ac90 0x7f660c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:17.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.943+0000 7f662b400700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f660c06ac90 0x7f660c06d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f661c0090b0 tx=0x7f661c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:17.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:17.947+0000 7f6623400700 1 -- 172.21.15.99:0/1014961689 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f662c15b710 con 0x7f662c160cc0 2024-03-29T09:10:18.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.153+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f662c15b710 con 0x7f662c160cc0 2024-03-29T09:10:18.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.154+0000 7f6623400700 1 -- 172.21.15.99:0/1014961689 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f662c15b710 con 0x7f662c160cc0 2024-03-29T09:10:18.153 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:18.154 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f660c06ac90 msgr2=0x7f660c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f660c06ac90 0x7f660c06d150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f661c0090b0 tx=0x7f661c008030 comp rx=0 tx=0).stop 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c160cc0 msgr2=0x7f662c1f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c160cc0 0x7f662c1f31d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6624009f70 tx=0x7f662400d460 comp rx=0 tx=0).stop 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 shutdown_connections 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f660c06ac90 0x7f660c06d150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f662c160cc0 0x7f662c1f31d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f662c0d1b20 0x7f662c1559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 --2- 172.21.15.99:0/1014961689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f662c1608e0 0x7f662c0cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 >> 172.21.15.99:0/1014961689 conn(0x7f662c09c490 msgr2=0x7f662c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 shutdown_connections 2024-03-29T09:10:18.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.157+0000 7f6632800700 1 -- 172.21.15.99:0/1014961689 wait complete. 2024-03-29T09:10:18.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:18.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:18.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.370+0000 7f59b0c00700 1 Processor -- start 2024-03-29T09:10:18.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.370+0000 7f59b0c00700 1 -- start start 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59b0c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f59ac0b3cc0 0x7f59ac0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59b0c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59ac0d3380 0x7f59ac0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59b0c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ac0d3810 con 0x7f59ac160770 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59b0c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ac0b4140 con 0x7f59ac0d3380 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59b0c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ac0d1520 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59aa800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59ac0d3380 0x7f59ac0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59aa800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f59ac0d3380 0x7f59ac0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35428/0 (socket says 172.21.15.99:35428) 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59aa800700 1 -- 172.21.15.99:0/812313502 learned_addr learned my addr 172.21.15.99:0/812313502 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:18.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.371+0000 7f59a9400700 1 --2- 172.21.15.99:0/812313502 >> v2:172.21.15.133:3300/0 conn(0x7f59ac0b3cc0 0x7f59ac0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59aa800700 1 -- 172.21.15.99:0/812313502 >> v2:172.21.15.133:3300/0 conn(0x7f59ac0b3cc0 msgr2=0x7f59ac0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59aa800700 1 --2- 172.21.15.99:0/812313502 >> v2:172.21.15.133:3300/0 conn(0x7f59ac0b3cc0 0x7f59ac0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59aa800700 1 -- 172.21.15.99:0/812313502 >> v1:172.21.15.133:6789/0 conn(0x7f59ac160770 legacy=0x7f59ac0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59aa800700 1 -- 172.21.15.99:0/812313502 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac0d1800 con 0x7f59ac0d3380 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59aa800700 1 --2- 172.21.15.99:0/812313502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0d3380 0x7f59ac0ce080 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0b5cf0 tx=0x7f59a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b2c622cd119563 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59a8a00700 1 -- 172.21.15.99:0/812313502 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400e070 con 0x7f59ac0d3380 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59a8a00700 1 -- 172.21.15.99:0/812313502 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400b910 con 0x7f59ac0d3380 2024-03-29T09:10:18.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.372+0000 7f59a8a00700 1 -- 172.21.15.99:0/812313502 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400bcc0 con 0x7f59ac0d3380 2024-03-29T09:10:18.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 -- 172.21.15.99:0/812313502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0d3380 msgr2=0x7f59ac0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 --2- 172.21.15.99:0/812313502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0d3380 0x7f59ac0ce080 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0b5cf0 tx=0x7f59a400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 -- 172.21.15.99:0/812313502 shutdown_connections 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 --2- 172.21.15.99:0/812313502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0d3380 0x7f59ac0ce080 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 --2- 172.21.15.99:0/812313502 >> v2:172.21.15.133:3300/0 conn(0x7f59ac0b3cc0 0x7f59ac0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 -- 172.21.15.99:0/812313502 >> 172.21.15.99:0/812313502 conn(0x7f59ac08afd0 msgr2=0x7f59ac09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 -- 172.21.15.99:0/812313502 shutdown_connections 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.373+0000 7f59b0c00700 1 -- 172.21.15.99:0/812313502 wait complete. 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.374+0000 7f59b0c00700 1 Processor -- start 2024-03-29T09:10:18.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.375+0000 7f59b0c00700 1 -- start start 2024-03-29T09:10:18.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.375+0000 7f59b0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b3cc0 0x7f59ac152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.375+0000 7f59b0c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59ac0d3380 0x7f59ac06bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.375+0000 7f59b0c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac160770 0x7f59ac06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.375+0000 7f59b0c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0cc210 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.375+0000 7f59b0c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0cbf30 con 0x7f59ac160770 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.375+0000 7f59b0c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac0b5e80 con 0x7f59ac0d3380 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b3cc0 0x7f59ac152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b3cc0 0x7f59ac152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35436/0 (socket says 172.21.15.99:35436) 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 -- 172.21.15.99:0/2293525282 learned_addr learned my addr 172.21.15.99:0/2293525282 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59aa800700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac160770 0x7f59ac06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9400700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59ac0d3380 0x7f59ac06bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 -- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59ac0d3380 msgr2=0x7f59ac06bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59ac0d3380 0x7f59ac06bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 -- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac160770 msgr2=0x7f59ac06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac160770 0x7f59ac06c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 -- 172.21.15.99:0/2293525282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a4009b30 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59aa800700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac160770 0x7f59ac06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:18.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.376+0000 7f59a9e00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b3cc0 0x7f59ac152ce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c450 tx=0x7f599c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:18.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.377+0000 7f59a2000700 1 -- 172.21.15.99:0/2293525282 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c0052d0 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.377+0000 7f59a2000700 1 -- 172.21.15.99:0/2293525282 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599c005430 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.377+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac0b6160 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.377+0000 7f59a2000700 1 -- 172.21.15.99:0/2293525282 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599c0038c0 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.377+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac150300 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.378+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac0678b0 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.379+0000 7f59a2000700 1 -- 172.21.15.99:0/2293525282 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f599c003a20 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.380+0000 7f59a2000700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f598c06ac90 0x7f598c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.381+0000 7f59a2000700 1 -- 172.21.15.99:0/2293525282 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f599c088a30 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.381+0000 7f59a9400700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f598c06ac90 0x7f598c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.381+0000 7f59a9400700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f598c06ac90 0x7f598c06d150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59ac06bef0 tx=0x7f5998008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:18.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.385+0000 7f59a2000700 1 -- 172.21.15.99:0/2293525282 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f599c0538b0 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.590+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59ac0b21b0 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.589 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.591+0000 7f59a2000700 1 -- 172.21.15.99:0/2293525282 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f599c056ed0 con 0x7f59ac0b3cc0 2024-03-29T09:10:18.590 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f598c06ac90 msgr2=0x7f598c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f598c06ac90 0x7f598c06d150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f59ac06bef0 tx=0x7f5998008040 comp rx=0 tx=0).stop 2024-03-29T09:10:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b3cc0 msgr2=0x7f59ac152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b3cc0 0x7f59ac152ce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f599c00c450 tx=0x7f599c00c810 comp rx=0 tx=0).stop 2024-03-29T09:10:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 shutdown_connections 2024-03-29T09:10:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f598c06ac90 0x7f598c06d150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59ac160770 0x7f59ac06c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59ac0b3cc0 0x7f59ac152ce0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 --2- 172.21.15.99:0/2293525282 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59ac0d3380 0x7f59ac06bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 >> 172.21.15.99:0/2293525282 conn(0x7f59ac08afd0 msgr2=0x7f59ac073250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 shutdown_connections 2024-03-29T09:10:18.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.593+0000 7f59b0c00700 1 -- 172.21.15.99:0/2293525282 wait complete. 2024-03-29T09:10:18.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:18.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.804+0000 7f78b3a00700 1 Processor -- start 2024-03-29T09:10:18.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.804+0000 7f78b3a00700 1 -- start start 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.805+0000 7f78b3a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f78ac0b3cc0 0x7f78ac0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.805+0000 7f78b3a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0d3810 con 0x7f78ac0d3380 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.805+0000 7f78b3a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0d1520 con 0x7f78ac160770 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.805+0000 7f78b3a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac0b4140 con 0x7f78ac0b3cc0 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78b1a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f78ac0d3380 0x7f78ac0ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52582/0 (socket says 172.21.15.99:52582) 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78b1a00700 1 -- 172.21.15.99:0/3763261875 learned_addr learned my addr 172.21.15.99:0/3763261875 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78abe00700 1 --2- 172.21.15.99:0/3763261875 >> v2:172.21.15.133:3300/0 conn(0x7f78ac0b3cc0 0x7f78ac0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875460824 0 0) 0x7f78ac0d3810 con 0x7f78ac0d3380 2024-03-29T09:10:18.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78940035b0 con 0x7f78ac0d3380 2024-03-29T09:10:18.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2552595998 0 0) 0x7f78940035b0 con 0x7f78ac0d3380 2024-03-29T09:10:18.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 >> v2:172.21.15.133:3300/0 conn(0x7f78ac0b3cc0 msgr2=0x7f78ac0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:18.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 --2- 172.21.15.99:0/3763261875 >> v2:172.21.15.133:3300/0 conn(0x7f78ac0b3cc0 0x7f78ac0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 >> v1:172.21.15.133:6789/0 conn(0x7f78ac160770 legacy=0x7f78ac0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0d1800 con 0x7f78ac0d3380 2024-03-29T09:10:18.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.806+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f78a40019f0 con 0x7f78ac0d3380 2024-03-29T09:10:18.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 >> v1:172.21.15.99:6789/0 conn(0x7f78ac0d3380 legacy=0x7f78ac0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78ab400700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894003a00 0x7f7894003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78ab400700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7894004850 0x7f7894004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78ab400700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7894005210 0x7f789400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789400c080 con 0x7f7894004850 2024-03-29T09:10:18.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789400c1f0 con 0x7f7894003a00 2024-03-29T09:10:18.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789400c3b0 con 0x7f7894005210 2024-03-29T09:10:18.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78abe00700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7894004850 0x7f7894004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78b1000700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894003a00 0x7f7894003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78b1a00700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7894005210 0x7f789400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78abe00700 1 -- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7894005210 msgr2=0x7f789400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:18.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78abe00700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7894005210 0x7f789400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78abe00700 1 -- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894003a00 msgr2=0x7f7894003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78abe00700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894003a00 0x7f7894003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78abe00700 1 -- 172.21.15.99:0/3763261875 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940035b0 con 0x7f7894004850 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.807+0000 7f78b1a00700 1 -- 172.21.15.99:0/3763261875 reap_dead start 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78abe00700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7894004850 0x7f7894004cd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7898009b60 tx=0x7f7894005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbb14edf3cb8f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800e6c0 con 0x7f7894004850 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789800e820 con 0x7f7894004850 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78ab400700 1 -- 172.21.15.99:0/3763261875 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789800eb90 con 0x7f7894004850 2024-03-29T09:10:18.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78b3a00700 1 -- 172.21.15.99:0/3763261875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7894004850 msgr2=0x7f7894004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78b3a00700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7894004850 0x7f7894004cd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7898009b60 tx=0x7f7894005880 comp rx=0 tx=0).stop 2024-03-29T09:10:18.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78b3a00700 1 -- 172.21.15.99:0/3763261875 shutdown_connections 2024-03-29T09:10:18.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78b3a00700 1 --2- 172.21.15.99:0/3763261875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7894004850 0x7f7894004cd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.808+0000 7f78b3a00700 1 -- 172.21.15.99:0/3763261875 >> 172.21.15.99:0/3763261875 conn(0x7f78ac08afd0 msgr2=0x7f78ac09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:18.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.809+0000 7f78b3a00700 1 -- 172.21.15.99:0/3763261875 shutdown_connections 2024-03-29T09:10:18.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.809+0000 7f78b3a00700 1 -- 172.21.15.99:0/3763261875 wait complete. 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.810+0000 7f78b3a00700 1 Processor -- start 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.810+0000 7f78b3a00700 1 -- start start 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.811+0000 7f78b3a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894004850 0x7f78ac0c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.811+0000 7f78b3a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78ac1545b0 0x7f78ac0b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.811+0000 7f78b3a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78ac0cc430 0x7f78ac0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.811+0000 7f78b3a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0ccdf0 con 0x7f78ac0cc430 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.811+0000 7f78b3a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0c72a0 con 0x7f7894004850 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.811+0000 7f78b3a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0c7580 con 0x7f78ac1545b0 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78ac0cc430 0x7f78ac0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78ac0cc430 0x7f78ac0cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35452/0 (socket says 172.21.15.99:35452) 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 -- 172.21.15.99:0/2640503003 learned_addr learned my addr 172.21.15.99:0/2640503003 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78abe00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78ac1545b0 0x7f78ac0b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1000700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894004850 0x7f78ac0c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 -- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78ac1545b0 msgr2=0x7f78ac0b6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78ac1545b0 0x7f78ac0b6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 -- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894004850 msgr2=0x7f78ac0c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894004850 0x7f78ac0c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 -- 172.21.15.99:0/2640503003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0c7860 con 0x7f78ac0cc430 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78b1a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78ac0cc430 0x7f78ac0cc8b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f78a4000f80 tx=0x7f78a400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.812+0000 7f78a8c00700 1 -- 172.21.15.99:0/2640503003 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400acb0 con 0x7f78ac0cc430 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.813+0000 7f78a8c00700 1 -- 172.21.15.99:0/2640503003 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a400ae10 con 0x7f78ac0cc430 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.813+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac0c7b40 con 0x7f78ac0cc430 2024-03-29T09:10:18.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.813+0000 7f78a8c00700 1 -- 172.21.15.99:0/2640503003 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a4013630 con 0x7f78ac0cc430 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.813+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac0b6c90 con 0x7f78ac0cc430 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.814+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac0678b0 con 0x7f78ac0cc430 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.815+0000 7f78a8c00700 1 -- 172.21.15.99:0/2640503003 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f78a4013790 con 0x7f78ac0cc430 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.815+0000 7f78a8c00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f788c06ac90 0x7f788c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.815+0000 7f78a8c00700 1 -- 172.21.15.99:0/2640503003 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f78a408a190 con 0x7f78ac0cc430 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.816+0000 7f78b1000700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f788c06ac90 0x7f788c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.816+0000 7f78b1000700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f788c06ac90 0x7f788c06d150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f78ac064450 tx=0x7f78a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:18.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:18.820+0000 7f78a8c00700 1 -- 172.21.15.99:0/2640503003 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f78a4055010 con 0x7f78ac0cc430 2024-03-29T09:10:19.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.021+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78ac0c8120 con 0x7f78ac0cc430 2024-03-29T09:10:19.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.022+0000 7f78a8c00700 1 -- 172.21.15.99:0/2640503003 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78a4058630 con 0x7f78ac0cc430 2024-03-29T09:10:19.021 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f788c06ac90 msgr2=0x7f788c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f788c06ac90 0x7f788c06d150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f78ac064450 tx=0x7f78a0009040 comp rx=0 tx=0).stop 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78ac0cc430 msgr2=0x7f78ac0cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78ac0cc430 0x7f78ac0cc8b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f78a4000f80 tx=0x7f78a400a5f0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 shutdown_connections 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f788c06ac90 0x7f788c06d150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f78ac0cc430 0x7f78ac0cc8b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7894004850 0x7f78ac0c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 --2- 172.21.15.99:0/2640503003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78ac1545b0 0x7f78ac0b6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 >> 172.21.15.99:0/2640503003 conn(0x7f78ac08afd0 msgr2=0x7f78ac0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 shutdown_connections 2024-03-29T09:10:19.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.024+0000 7f78b3a00700 1 -- 172.21.15.99:0/2640503003 wait complete. 2024-03-29T09:10:19.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.237+0000 7fdb62e00700 1 Processor -- start 2024-03-29T09:10:19.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.237+0000 7fdb62e00700 1 -- start start 2024-03-29T09:10:19.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.238+0000 7fdb62e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdb5c160770 0x7fdb5c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.238+0000 7fdb62e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d3550 0x7fdb5c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.238+0000 7fdb62e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5c0d1520 con 0x7fdb5c0d0d60 2024-03-29T09:10:19.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.238+0000 7fdb62e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5c0d1800 con 0x7fdb5c160770 2024-03-29T09:10:19.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.238+0000 7fdb62e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c0d1ae0 con 0x7fdb5c0d3550 2024-03-29T09:10:19.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.239+0000 7fdb60e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d3550 0x7fdb5c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.239+0000 7fdb5be00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdb5c160770 0x7fdb5c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.239+0000 7fdb60e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d3550 0x7fdb5c0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:42528/0 (socket says 172.21.15.99:42528) 2024-03-29T09:10:19.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.239+0000 7fdb5be00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdb5c160770 0x7fdb5c0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:49664/0 (socket says 172.21.15.99:49664) 2024-03-29T09:10:19.240 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.239+0000 7fdb60e00700 1 -- 172.21.15.99:0/1850654834 learned_addr learned my addr 172.21.15.99:0/1850654834 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.239+0000 7fdb5b400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdb5c0d0d60 0x7fdb5c0d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:36364/0 (socket says 172.21.15.99:36364) 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5be00700 1 -- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d3550 msgr2=0x7fdb5c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5be00700 1 --2- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d3550 0x7fdb5c0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5be00700 1 -- 172.21.15.99:0/1850654834 >> v1:172.21.15.133:6789/0 conn(0x7fdb5c0d0d60 legacy=0x7fdb5c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5be00700 1 -- 172.21.15.99:0/1850654834 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c0d1dc0 con 0x7fdb5c160770 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb60e00700 1 --2- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d3550 0x7fdb5c0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5aa00700 1 -- 172.21.15.99:0/1850654834 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399401314 0 0) 0x7fdb5c0d1520 con 0x7fdb5c0d0d60 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5be00700 1 --2- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c160770 0x7fdb5c0b3cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009840 tx=0x7fdb4c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580c7165d1a15a66 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5aa00700 1 -- 172.21.15.99:0/1850654834 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c011830 con 0x7fdb5c160770 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5aa00700 1 -- 172.21.15.99:0/1850654834 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c011990 con 0x7fdb5c160770 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.240+0000 7fdb5aa00700 1 -- 172.21.15.99:0/1850654834 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c011d20 con 0x7fdb5c160770 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 -- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c160770 msgr2=0x7fdb5c0b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c160770 0x7fdb5c0b3cc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009840 tx=0x7fdb4c009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 -- 172.21.15.99:0/1850654834 shutdown_connections 2024-03-29T09:10:19.241 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d3550 0x7fdb5c0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1850654834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c160770 0x7fdb5c0b3cc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 -- 172.21.15.99:0/1850654834 >> 172.21.15.99:0/1850654834 conn(0x7fdb5c08afd0 msgr2=0x7fdb5c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:19.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 -- 172.21.15.99:0/1850654834 shutdown_connections 2024-03-29T09:10:19.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.241+0000 7fdb62e00700 1 -- 172.21.15.99:0/1850654834 wait complete. 2024-03-29T09:10:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.243+0000 7fdb62e00700 1 Processor -- start 2024-03-29T09:10:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.243+0000 7fdb62e00700 1 -- start start 2024-03-29T09:10:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb62e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d0d60 0x7fdb5c14fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb62e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c0d3550 0x7fdb5c1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb62e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c160770 0x7fdb5c1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb62e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c153080 con 0x7fdb5c160770 2024-03-29T09:10:19.244 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb62e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c14e2b0 con 0x7fdb5c0d3550 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb62e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c1501c0 con 0x7fdb5c0d0d60 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb60e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c160770 0x7fdb5c1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb5b400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c0d3550 0x7fdb5c1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb60e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c160770 0x7fdb5c1506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35464/0 (socket says 172.21.15.99:35464) 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb5be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d0d60 0x7fdb5c14fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.244+0000 7fdb60e00700 1 -- 172.21.15.99:0/1728095535 learned_addr learned my addr 172.21.15.99:0/1728095535 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb60e00700 1 -- 172.21.15.99:0/1728095535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d0d60 msgr2=0x7fdb5c14fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb60e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d0d60 0x7fdb5c14fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb60e00700 1 -- 172.21.15.99:0/1728095535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c0d3550 msgr2=0x7fdb5c1525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb60e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c0d3550 0x7fdb5c1525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb60e00700 1 -- 172.21.15.99:0/1728095535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb54009b30 con 0x7fdb5c160770 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb60e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c160770 0x7fdb5c1506d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400efc0 tx=0x7fdb5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb53e00700 1 -- 172.21.15.99:0/1728095535 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400ca80 con 0x7fdb5c160770 2024-03-29T09:10:19.245 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb53e00700 1 -- 172.21.15.99:0/1728095535 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5400f040 con 0x7fdb5c160770 2024-03-29T09:10:19.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb53e00700 1 -- 172.21.15.99:0/1728095535 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5400ce20 con 0x7fdb5c160770 2024-03-29T09:10:19.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c0094a0 con 0x7fdb5c160770 2024-03-29T09:10:19.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.245+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5c14d980 con 0x7fdb5c160770 2024-03-29T09:10:19.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.246+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5c14e460 con 0x7fdb5c160770 2024-03-29T09:10:19.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.247+0000 7fdb53e00700 1 -- 172.21.15.99:0/1728095535 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdb54014480 con 0x7fdb5c160770 2024-03-29T09:10:19.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.248+0000 7fdb53e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdb3c06ac90 0x7fdb3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.248+0000 7fdb53e00700 1 -- 172.21.15.99:0/1728095535 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdb54089e30 con 0x7fdb5c160770 2024-03-29T09:10:19.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.249+0000 7fdb5be00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdb3c06ac90 0x7fdb3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.249+0000 7fdb5be00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdb3c06ac90 0x7fdb3c06d150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0090b0 tx=0x7fdb4c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:19.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.253+0000 7fdb53e00700 1 -- 172.21.15.99:0/1728095535 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdb54054c20 con 0x7fdb5c160770 2024-03-29T09:10:19.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.452+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb5c0678b0 con 0x7fdb5c160770 2024-03-29T09:10:19.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.452+0000 7fdb53e00700 1 -- 172.21.15.99:0/1728095535 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb54058240 con 0x7fdb5c160770 2024-03-29T09:10:19.451 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:19.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdb3c06ac90 msgr2=0x7fdb3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdb3c06ac90 0x7fdb3c06d150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0090b0 tx=0x7fdb4c01c040 comp rx=0 tx=0).stop 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c160770 msgr2=0x7fdb5c1506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c160770 0x7fdb5c1506d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400efc0 tx=0x7fdb5400c3a0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 shutdown_connections 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdb3c06ac90 0x7fdb3c06d150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdb5c160770 0x7fdb5c1506d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdb5c0d0d60 0x7fdb5c14fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 --2- 172.21.15.99:0/1728095535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb5c0d3550 0x7fdb5c1525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.454+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 >> 172.21.15.99:0/1728095535 conn(0x7fdb5c08afd0 msgr2=0x7fdb5c06b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.455+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 shutdown_connections 2024-03-29T09:10:19.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.455+0000 7fdb62e00700 1 -- 172.21.15.99:0/1728095535 wait complete. 2024-03-29T09:10:19.471 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:19.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.665+0000 7f51c9400700 1 Processor -- start 2024-03-29T09:10:19.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.665+0000 7f51c9400700 1 -- start start 2024-03-29T09:10:19.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.666+0000 7f51c9400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51c40d07d0 0x7f51c4071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.666+0000 7f51c9400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40af8f0 con 0x7f51c40ae660 2024-03-29T09:10:19.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.666+0000 7f51c9400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40aed70 con 0x7f51c40ae280 2024-03-29T09:10:19.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.666+0000 7f51c9400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c415d7e0 con 0x7f51c40d07d0 2024-03-29T09:10:19.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.666+0000 7f51c1e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f51c40ae660 0x7f51c40d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52590/0 (socket says 172.21.15.99:52590) 2024-03-29T09:10:19.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.666+0000 7f51c1e00700 1 -- 172.21.15.99:0/348538175 learned_addr learned my addr 172.21.15.99:0/348538175 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:19.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.666+0000 7f51c3200700 1 --2- 172.21.15.99:0/348538175 >> v2:172.21.15.133:3300/0 conn(0x7f51c40d07d0 0x7f51c4071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.667+0000 7f51c1400700 1 -- 172.21.15.99:0/348538175 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105585958 0 0) 0x7f51c40af8f0 con 0x7f51c40ae660 2024-03-29T09:10:19.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.667+0000 7f51c1400700 1 -- 172.21.15.99:0/348538175 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a80035b0 con 0x7f51c40ae660 2024-03-29T09:10:19.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.667+0000 7f51c3200700 1 -- 172.21.15.99:0/348538175 >> v1:172.21.15.133:6789/0 conn(0x7f51c40ae280 legacy=0x7f51c406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.667+0000 7f51c3200700 1 -- 172.21.15.99:0/348538175 >> v1:172.21.15.99:6789/0 conn(0x7f51c40ae660 legacy=0x7f51c40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.667+0000 7f51c3200700 1 -- 172.21.15.99:0/348538175 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40af050 con 0x7f51c40d07d0 2024-03-29T09:10:19.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.667+0000 7f51c3200700 1 --2- 172.21.15.99:0/348538175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40d07d0 0x7f51c4071ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009d90 tx=0x7f51bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827f29accb17a2d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:19.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.667+0000 7f51c1400700 1 -- 172.21.15.99:0/348538175 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00e070 con 0x7f51c40d07d0 2024-03-29T09:10:19.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c1400700 1 -- 172.21.15.99:0/348538175 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc00bb20 con 0x7f51c40d07d0 2024-03-29T09:10:19.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c1400700 1 -- 172.21.15.99:0/348538175 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc012430 con 0x7f51c40d07d0 2024-03-29T09:10:19.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c9400700 1 -- 172.21.15.99:0/348538175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40d07d0 msgr2=0x7f51c4071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c9400700 1 --2- 172.21.15.99:0/348538175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40d07d0 0x7f51c4071ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009d90 tx=0x7f51bc00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:19.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c9400700 1 -- 172.21.15.99:0/348538175 shutdown_connections 2024-03-29T09:10:19.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c9400700 1 --2- 172.21.15.99:0/348538175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40d07d0 0x7f51c4071ea0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c9400700 1 -- 172.21.15.99:0/348538175 >> 172.21.15.99:0/348538175 conn(0x7f51c4066720 msgr2=0x7f51c4066b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:19.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c9400700 1 -- 172.21.15.99:0/348538175 shutdown_connections 2024-03-29T09:10:19.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.668+0000 7f51c9400700 1 -- 172.21.15.99:0/348538175 wait complete. 2024-03-29T09:10:19.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.670+0000 7f51c9400700 1 Processor -- start 2024-03-29T09:10:19.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.670+0000 7f51c9400700 1 -- start start 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c9400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40ae280 0x7f51c415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c40ae660 0x7f51c415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c9400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51c40d07d0 0x7f51c415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c9400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4154590 con 0x7f51c40ae660 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c9400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c415ef80 con 0x7f51c40ae280 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c9400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c41f2750 con 0x7f51c40d07d0 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c40ae660 0x7f51c415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c40ae660 0x7f51c415db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35476/0 (socket says 172.21.15.99:35476) 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 -- 172.21.15.99:0/2566832854 learned_addr learned my addr 172.21.15.99:0/2566832854 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c3200700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51c40d07d0 0x7f51c415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c2800700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40ae280 0x7f51c415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 -- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51c40d07d0 msgr2=0x7f51c415e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51c40d07d0 0x7f51c415e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 -- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40ae280 msgr2=0x7f51c415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40ae280 0x7f51c415cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c1e00700 1 -- 172.21.15.99:0/2566832854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc009a70 con 0x7f51c40ae660 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.671+0000 7f51c3200700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51c40d07d0 0x7f51c415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.672+0000 7f51c1e00700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c40ae660 0x7f51c415db30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005910 tx=0x7f51b000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.672+0000 7f51baa00700 1 -- 172.21.15.99:0/2566832854 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0005d70 con 0x7f51c40ae660 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.672+0000 7f51baa00700 1 -- 172.21.15.99:0/2566832854 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b00070d0 con 0x7f51c40ae660 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.672+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c415e710 con 0x7f51c40ae660 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.672+0000 7f51baa00700 1 -- 172.21.15.99:0/2566832854 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0007460 con 0x7f51c40ae660 2024-03-29T09:10:19.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.672+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c415f1d0 con 0x7f51c40ae660 2024-03-29T09:10:19.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.673+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c4160b50 con 0x7f51c40ae660 2024-03-29T09:10:19.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.674+0000 7f51baa00700 1 -- 172.21.15.99:0/2566832854 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f51b00075c0 con 0x7f51c40ae660 2024-03-29T09:10:19.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.675+0000 7f51baa00700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51a406ac90 0x7f51a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:19.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.675+0000 7f51baa00700 1 -- 172.21.15.99:0/2566832854 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f51b0088ce0 con 0x7f51c40ae660 2024-03-29T09:10:19.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.675+0000 7f51c2800700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51a406ac90 0x7f51a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:19.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.676+0000 7f51c2800700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51a406ac90 0x7f51a406d150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51b4000f60 tx=0x7f51b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:19.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.679+0000 7f51baa00700 1 -- 172.21.15.99:0/2566832854 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f51c4160b50 con 0x7f51c40ae660 2024-03-29T09:10:19.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.882+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51c4160b50 con 0x7f51c40ae660 2024-03-29T09:10:19.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.882+0000 7f51baa00700 1 -- 172.21.15.99:0/2566832854 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51c4160b50 con 0x7f51c40ae660 2024-03-29T09:10:19.881 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:19.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51a406ac90 msgr2=0x7f51a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51a406ac90 0x7f51a406d150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51b4000f60 tx=0x7f51b4009040 comp rx=0 tx=0).stop 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c40ae660 msgr2=0x7f51c415db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c40ae660 0x7f51c415db30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f51b0005910 tx=0x7f51b000ca90 comp rx=0 tx=0).stop 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 shutdown_connections 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51a406ac90 0x7f51a406d150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51c40d07d0 0x7f51c415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c40ae280 0x7f51c415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 --2- 172.21.15.99:0/2566832854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51c40ae660 0x7f51c415db30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 >> 172.21.15.99:0/2566832854 conn(0x7f51c4066720 msgr2=0x7f51c40b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 shutdown_connections 2024-03-29T09:10:19.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:19.885+0000 7f51c9400700 1 -- 172.21.15.99:0/2566832854 wait complete. 2024-03-29T09:10:19.902 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:20.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.092+0000 7f7731600700 1 Processor -- start 2024-03-29T09:10:20.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.093+0000 7f7731600700 1 -- start start 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f7731600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c146150 0x7f772c161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f7731600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f772c06a7d0 0x7f772c06abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f7731600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c06b130 con 0x7f772c145d70 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f7731600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c0b3990 con 0x7f772c06a7d0 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f7731600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0b3c70 con 0x7f772c146150 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f772b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f772c06a7d0 0x7f772c06abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f772b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f772c06a7d0 0x7f772c06abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35482/0 (socket says 172.21.15.99:35482) 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f772b400700 1 -- 172.21.15.99:0/4108338811 learned_addr learned my addr 172.21.15.99:0/4108338811 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f772b400700 1 -- 172.21.15.99:0/4108338811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c146150 msgr2=0x7f772c161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:20.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f772b400700 1 --2- 172.21.15.99:0/4108338811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c146150 0x7f772c161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.094+0000 7f772b400700 1 -- 172.21.15.99:0/4108338811 >> v1:172.21.15.133:6789/0 conn(0x7f772c145d70 legacy=0x7f772c0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.095+0000 7f772b400700 1 -- 172.21.15.99:0/4108338811 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c160f40 con 0x7f772c06a7d0 2024-03-29T09:10:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.095+0000 7f772b400700 1 --2- 172.21.15.99:0/4108338811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c06a7d0 0x7f772c06abf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7724009e50 tx=0x7f772400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91190534bee800b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.095+0000 7f7729600700 1 -- 172.21.15.99:0/4108338811 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400e070 con 0x7f772c06a7d0 2024-03-29T09:10:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.095+0000 7f7729600700 1 -- 172.21.15.99:0/4108338811 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7724003a20 con 0x7f772c06a7d0 2024-03-29T09:10:20.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.095+0000 7f7729600700 1 -- 172.21.15.99:0/4108338811 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724003dd0 con 0x7f772c06a7d0 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.095+0000 7f7731600700 1 -- 172.21.15.99:0/4108338811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c06a7d0 msgr2=0x7f772c06abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.095+0000 7f7731600700 1 --2- 172.21.15.99:0/4108338811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c06a7d0 0x7f772c06abf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7724009e50 tx=0x7f772400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.096+0000 7f7731600700 1 -- 172.21.15.99:0/4108338811 shutdown_connections 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.096+0000 7f7731600700 1 --2- 172.21.15.99:0/4108338811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c06a7d0 0x7f772c06abf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.096+0000 7f7731600700 1 --2- 172.21.15.99:0/4108338811 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c146150 0x7f772c161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.096+0000 7f7731600700 1 -- 172.21.15.99:0/4108338811 >> 172.21.15.99:0/4108338811 conn(0x7f772c066df0 msgr2=0x7f772c06d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.096+0000 7f7731600700 1 -- 172.21.15.99:0/4108338811 shutdown_connections 2024-03-29T09:10:20.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.096+0000 7f7731600700 1 -- 172.21.15.99:0/4108338811 wait complete. 2024-03-29T09:10:20.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.097+0000 7f7731600700 1 Processor -- start 2024-03-29T09:10:20.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.098+0000 7f7731600700 1 -- start start 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f7731600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c06a7d0 0x7f772c1f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f7731600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c145d70 0x7f772c13e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f7731600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772c160570 0x7f772c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f7731600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c14e970 con 0x7f772c145d70 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f7731600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c13e6c0 con 0x7f772c160570 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f7731600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c13e8b0 con 0x7f772c06a7d0 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c145d70 0x7f772c13e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c145d70 0x7f772c13e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35496/0 (socket says 172.21.15.99:35496) 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 -- 172.21.15.99:0/845121168 learned_addr learned my addr 172.21.15.99:0/845121168 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:20.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772b400700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772c160570 0x7f772c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772aa00700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c06a7d0 0x7f772c1f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 -- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c06a7d0 msgr2=0x7f772c1f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c06a7d0 0x7f772c1f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 -- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772c160570 msgr2=0x7f772c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772c160570 0x7f772c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772a000700 1 -- 172.21.15.99:0/845121168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009b30 con 0x7f772c145d70 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.099+0000 7f772aa00700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c06a7d0 0x7f772c1f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.100+0000 7f772a000700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c145d70 0x7f772c13e180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f771800b480 tx=0x7f771800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.100+0000 7f7722a00700 1 -- 172.21.15.99:0/845121168 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718011830 con 0x7f772c145d70 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.100+0000 7f7722a00700 1 -- 172.21.15.99:0/845121168 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7718011990 con 0x7f772c145d70 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.100+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c14dfd0 con 0x7f772c145d70 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.100+0000 7f7722a00700 1 -- 172.21.15.99:0/845121168 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718011d40 con 0x7f772c145d70 2024-03-29T09:10:20.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.100+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c14e590 con 0x7f772c145d70 2024-03-29T09:10:20.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.101+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c14eab0 con 0x7f772c145d70 2024-03-29T09:10:20.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.102+0000 7f7722a00700 1 -- 172.21.15.99:0/845121168 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f771801a400 con 0x7f772c145d70 2024-03-29T09:10:20.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.103+0000 7f7722a00700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f770c06ac90 0x7f770c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.103+0000 7f7722a00700 1 -- 172.21.15.99:0/845121168 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f77180896d0 con 0x7f772c145d70 2024-03-29T09:10:20.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.103+0000 7f772aa00700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f770c06ac90 0x7f770c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.103+0000 7f772aa00700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f770c06ac90 0x7f770c06d150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f771c001750 tx=0x7f771c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:20.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.107+0000 7f7722a00700 1 -- 172.21.15.99:0/845121168 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f772c14eab0 con 0x7f772c145d70 2024-03-29T09:10:20.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.326+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f772c14eab0 con 0x7f772c145d70 2024-03-29T09:10:20.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.326+0000 7f7722a00700 1 -- 172.21.15.99:0/845121168 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f772c14eab0 con 0x7f772c145d70 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.328+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f770c06ac90 msgr2=0x7f770c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.328+0000 7f7731600700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f770c06ac90 0x7f770c06d150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f771c001750 tx=0x7f771c009040 comp rx=0 tx=0).stop 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.328+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c145d70 msgr2=0x7f772c13e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.328+0000 7f7731600700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c145d70 0x7f772c13e180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f771800b480 tx=0x7f771800b840 comp rx=0 tx=0).stop 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.328+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 shutdown_connections 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.328+0000 7f7731600700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f770c06ac90 0x7f770c06d150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.328+0000 7f7731600700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772c160570 0x7f772c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.329+0000 7f7731600700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f772c06a7d0 0x7f772c1f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.329+0000 7f7731600700 1 --2- 172.21.15.99:0/845121168 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f772c145d70 0x7f772c13e180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.329+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 >> 172.21.15.99:0/845121168 conn(0x7f772c066df0 msgr2=0x7f772c06f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.329+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 shutdown_connections 2024-03-29T09:10:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.329+0000 7f7731600700 1 -- 172.21.15.99:0/845121168 wait complete. 2024-03-29T09:10:20.329 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:20.346 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:10:20.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:20.346 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-29T09:10:20.411 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:20.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:10:20.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:20.411 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T09:10:20.453 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:10:20.453 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:20.499 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:10:20.500 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:20.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:20.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.775+0000 7f70cb000700 1 Processor -- start 2024-03-29T09:10:20.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.775+0000 7f70cb000700 1 -- start start 2024-03-29T09:10:20.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.776+0000 7f70cb000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.776+0000 7f70cb000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c40cddc0 con 0x7f70c40d1b20 2024-03-29T09:10:20.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.776+0000 7f70cb000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c40ce0a0 con 0x7f70c41608e0 2024-03-29T09:10:20.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.776+0000 7f70cb000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c40ce380 con 0x7f70c4160cc0 2024-03-29T09:10:20.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.776+0000 7f70c3400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f70c40d1b20 0x7f70c40afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52606/0 (socket says 172.21.15.99:52606) 2024-03-29T09:10:20.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.776+0000 7f70c3400700 1 -- 172.21.15.99:0/3740884952 learned_addr learned my addr 172.21.15.99:0/3740884952 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:20.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c9000700 1 --2- 172.21.15.99:0/3740884952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818250899 0 0) 0x7f70c40cddc0 con 0x7f70c40d1b20 2024-03-29T09:10:20.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70a80035b0 con 0x7f70c40d1b20 2024-03-29T09:10:20.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789536896 0 0) 0x7f70c40ce0a0 con 0x7f70c41608e0 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c40cddc0 con 0x7f70c41608e0 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c9000700 1 -- 172.21.15.99:0/3740884952 >> v1:172.21.15.133:6789/0 conn(0x7f70c41608e0 legacy=0x7f70c40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c9000700 1 -- 172.21.15.99:0/3740884952 >> v1:172.21.15.99:6789/0 conn(0x7f70c40d1b20 legacy=0x7f70c40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889886796 0 0) 0x7f70a80035b0 con 0x7f70c40d1b20 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c9000700 1 -- 172.21.15.99:0/3740884952 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40ce660 con 0x7f70c4160cc0 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.777+0000 7f70c9000700 1 --2- 172.21.15.99:0/3740884952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40d0150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009dc0 tx=0x7f70bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865ffe68f6f19abb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00e070 con 0x7f70c4160cc0 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc003a20 con 0x7f70c4160cc0 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70c2a00700 1 -- 172.21.15.99:0/3740884952 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0001670 con 0x7f70c4160cc0 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70cb000700 1 -- 172.21.15.99:0/3740884952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 msgr2=0x7f70c40d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70cb000700 1 --2- 172.21.15.99:0/3740884952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40d0150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009dc0 tx=0x7f70bc00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70cb000700 1 -- 172.21.15.99:0/3740884952 shutdown_connections 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70cb000700 1 --2- 172.21.15.99:0/3740884952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40d0150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70cb000700 1 -- 172.21.15.99:0/3740884952 >> 172.21.15.99:0/3740884952 conn(0x7f70c409c490 msgr2=0x7f70c40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:20.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70cb000700 1 -- 172.21.15.99:0/3740884952 shutdown_connections 2024-03-29T09:10:20.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.778+0000 7f70cb000700 1 -- 172.21.15.99:0/3740884952 wait complete. 2024-03-29T09:10:20.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.780+0000 7f70cb000700 1 Processor -- start 2024-03-29T09:10:20.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.781+0000 7f70cb000700 1 -- start start 2024-03-29T09:10:20.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70cb000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70c40d1b20 0x7f70c4150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70cb000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41608e0 0x7f70c40b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70cb000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70cb000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c4150a70 con 0x7f70c41608e0 2024-03-29T09:10:20.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70cb000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c40b8b20 con 0x7f70c40d1b20 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70cb000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c40b7f10 con 0x7f70c4160cc0 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70c3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41608e0 0x7f70c40b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70c3400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41608e0 0x7f70c40b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35498/0 (socket says 172.21.15.99:35498) 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70c3400700 1 -- 172.21.15.99:0/2397518562 learned_addr learned my addr 172.21.15.99:0/2397518562 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70c3e00700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70c40d1b20 0x7f70c4150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.782+0000 7f70c9000700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70c3400700 1 -- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 msgr2=0x7f70c40b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70c3400700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70c3400700 1 -- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70c40d1b20 msgr2=0x7f70c4150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70c3400700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70c40d1b20 0x7f70c4150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70c3400700 1 -- 172.21.15.99:0/2397518562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc009a70 con 0x7f70c41608e0 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70c3400700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41608e0 0x7f70c40b5420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c750 tx=0x7f70b000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:20.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70bbe00700 1 -- 172.21.15.99:0/2397518562 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0004c80 con 0x7f70c41608e0 2024-03-29T09:10:20.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70bbe00700 1 -- 172.21.15.99:0/2397518562 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b0004de0 con 0x7f70c41608e0 2024-03-29T09:10:20.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c40b81f0 con 0x7f70c41608e0 2024-03-29T09:10:20.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70bbe00700 1 -- 172.21.15.99:0/2397518562 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0007310 con 0x7f70c41608e0 2024-03-29T09:10:20.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.783+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c40b87b0 con 0x7f70c41608e0 2024-03-29T09:10:20.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.784+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c40677a0 con 0x7f70c41608e0 2024-03-29T09:10:20.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.785+0000 7f70bbe00700 1 -- 172.21.15.99:0/2397518562 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f70b0007470 con 0x7f70c41608e0 2024-03-29T09:10:20.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.786+0000 7f70bbe00700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f70a406acc0 0x7f70a406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:20.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.786+0000 7f70bbe00700 1 -- 172.21.15.99:0/2397518562 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f70b0089370 con 0x7f70c41608e0 2024-03-29T09:10:20.785 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.786+0000 7f70c3e00700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f70a406acc0 0x7f70a406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:20.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.787+0000 7f70c3e00700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f70a406acc0 0x7f70a406d180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f70b4001220 tx=0x7f70b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:20.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.790+0000 7f70bbe00700 1 -- 172.21.15.99:0/2397518562 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f70b00541f0 con 0x7f70c41608e0 2024-03-29T09:10:20.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.996+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70c40d0700 con 0x7f70c41608e0 2024-03-29T09:10:20.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:20.997+0000 7f70bbe00700 1 -- 172.21.15.99:0/2397518562 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f70b0057810 con 0x7f70c41608e0 2024-03-29T09:10:20.996 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:20.996 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f70a406acc0 msgr2=0x7f70a406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f70a406acc0 0x7f70a406d180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f70b4001220 tx=0x7f70b4009040 comp rx=0 tx=0).stop 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41608e0 msgr2=0x7f70c40b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41608e0 0x7f70c40b5420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70b000c750 tx=0x7f70b000ca60 comp rx=0 tx=0).stop 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 shutdown_connections 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f70a406acc0 0x7f70a406d180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70c4160cc0 0x7f70c40b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70c40d1b20 0x7f70c4150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 --2- 172.21.15.99:0/2397518562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70c41608e0 0x7f70c40b5420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:20.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 >> 172.21.15.99:0/2397518562 conn(0x7f70c409c490 msgr2=0x7f70c406ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:21.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 shutdown_connections 2024-03-29T09:10:21.000 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.000+0000 7f70cb000700 1 -- 172.21.15.99:0/2397518562 wait complete. 2024-03-29T09:10:21.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:21.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:21.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.214+0000 7fbbfe800700 1 Processor -- start 2024-03-29T09:10:21.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.214+0000 7fbbfe800700 1 -- start start 2024-03-29T09:10:21.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.215+0000 7fbbfe800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbbf8160770 0x7fbbf80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.215+0000 7fbbfe800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80d1520 con 0x7fbbf80d3550 2024-03-29T09:10:21.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.215+0000 7fbbfe800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80d1800 con 0x7fbbf80d0d60 2024-03-29T09:10:21.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.215+0000 7fbbfe800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80d1ae0 con 0x7fbbf8160770 2024-03-29T09:10:21.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbbf8160770 0x7fbbf80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf7e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbbf80d3550 0x7fbbf80ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52616/0 (socket says 172.21.15.99:52616) 2024-03-29T09:10:21.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf7e00700 1 -- 172.21.15.99:0/3964425643 learned_addr learned my addr 172.21.15.99:0/3964425643 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf7400700 1 -- 172.21.15.99:0/3964425643 >> v1:172.21.15.133:6789/0 conn(0x7fbbf80d0d60 legacy=0x7fbbf80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf7400700 1 -- 172.21.15.99:0/3964425643 >> v1:172.21.15.99:6789/0 conn(0x7fbbf80d3550 legacy=0x7fbbf80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf7400700 1 -- 172.21.15.99:0/3964425643 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80d1dc0 con 0x7fbbf8160770 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf6000700 1 -- 172.21.15.99:0/3964425643 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871869003 0 0) 0x7fbbf80d1520 con 0x7fbbf80d3550 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf7400700 1 --2- 172.21.15.99:0/3964425643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8160770 0x7fbbf80b3cc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009840 tx=0x7fbbe8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56e93db87a2d73e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.216+0000 7fbbf6000700 1 -- 172.21.15.99:0/3964425643 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8004230 con 0x7fbbf8160770 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbf6000700 1 -- 172.21.15.99:0/3964425643 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8004390 con 0x7fbbf8160770 2024-03-29T09:10:21.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbf6000700 1 -- 172.21.15.99:0/3964425643 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe80036a0 con 0x7fbbf8160770 2024-03-29T09:10:21.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbfe800700 1 -- 172.21.15.99:0/3964425643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8160770 msgr2=0x7fbbf80b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3964425643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8160770 0x7fbbf80b3cc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8009840 tx=0x7fbbe8009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:21.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbfe800700 1 -- 172.21.15.99:0/3964425643 shutdown_connections 2024-03-29T09:10:21.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3964425643 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf8160770 0x7fbbf80b3cc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbfe800700 1 -- 172.21.15.99:0/3964425643 >> 172.21.15.99:0/3964425643 conn(0x7fbbf808afd0 msgr2=0x7fbbf809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:21.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbfe800700 1 -- 172.21.15.99:0/3964425643 shutdown_connections 2024-03-29T09:10:21.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.217+0000 7fbbfe800700 1 -- 172.21.15.99:0/3964425643 wait complete. 2024-03-29T09:10:21.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.219+0000 7fbbfe800700 1 Processor -- start 2024-03-29T09:10:21.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.219+0000 7fbbfe800700 1 -- start start 2024-03-29T09:10:21.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbfe800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf80d0d60 0x7fbbf8150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbfe800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbf80d3550 0x7fbbf814d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbfe800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbf8160770 0x7fbbf814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbfe800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf814e9a0 con 0x7fbbf80d0d60 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbfe800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf814dac0 con 0x7fbbf8160770 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbfe800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf814dd20 con 0x7fbbf80d3550 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbf7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf80d0d60 0x7fbbf8150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbf7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf80d0d60 0x7fbbf8150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35510/0 (socket says 172.21.15.99:35510) 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbf7400700 1 -- 172.21.15.99:0/3739227629 learned_addr learned my addr 172.21.15.99:0/3739227629 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbf6a00700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbf80d3550 0x7fbbf814d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.220+0000 7fbbf7e00700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbf8160770 0x7fbbf814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbf7400700 1 -- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbf80d3550 msgr2=0x7fbbf814d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbf7400700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbf80d3550 0x7fbbf814d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbf7400700 1 -- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbf8160770 msgr2=0x7fbbf814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbf7400700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbf8160770 0x7fbbf814edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbf7400700 1 -- 172.21.15.99:0/3739227629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe80094a0 con 0x7fbbf80d0d60 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbf7400700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf80d0d60 0x7fbbf8150140 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80090b0 tx=0x7fbbe80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:21.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbef400700 1 -- 172.21.15.99:0/3739227629 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8011830 con 0x7fbbf80d0d60 2024-03-29T09:10:21.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbef400700 1 -- 172.21.15.99:0/3739227629 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe8011990 con 0x7fbbf80d0d60 2024-03-29T09:10:21.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbef400700 1 -- 172.21.15.99:0/3739227629 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe8011d40 con 0x7fbbf80d0d60 2024-03-29T09:10:21.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.221+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf814e000 con 0x7fbbf80d0d60 2024-03-29T09:10:21.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.222+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf814e5c0 con 0x7fbbf80d0d60 2024-03-29T09:10:21.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.223+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf80678b0 con 0x7fbbf80d0d60 2024-03-29T09:10:21.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.223+0000 7fbbef400700 1 -- 172.21.15.99:0/3739227629 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbbe801b440 con 0x7fbbf80d0d60 2024-03-29T09:10:21.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.225+0000 7fbbef400700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbd806ac90 0x7fbbd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.225+0000 7fbbef400700 1 -- 172.21.15.99:0/3739227629 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbbe808afe0 con 0x7fbbf80d0d60 2024-03-29T09:10:21.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.225+0000 7fbbf6a00700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbd806ac90 0x7fbbd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.226+0000 7fbbf6a00700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbd806ac90 0x7fbbd806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80a1090 tx=0x7fbbe4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:21.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.229+0000 7fbbef400700 1 -- 172.21.15.99:0/3739227629 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbbe8055e80 con 0x7fbbf80d0d60 2024-03-29T09:10:21.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.432+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbbf80b3010 con 0x7fbbf80d0d60 2024-03-29T09:10:21.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.432+0000 7fbbef400700 1 -- 172.21.15.99:0/3739227629 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbbe802a0a0 con 0x7fbbf80d0d60 2024-03-29T09:10:21.432 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:21.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbd806ac90 msgr2=0x7fbbd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbd806ac90 0x7fbbd806d150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbbf80a1090 tx=0x7fbbe4009040 comp rx=0 tx=0).stop 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf80d0d60 msgr2=0x7fbbf8150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf80d0d60 0x7fbbf8150140 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80090b0 tx=0x7fbbe80047c0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 shutdown_connections 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbbd806ac90 0x7fbbd806d150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbf8160770 0x7fbbf814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbf80d0d60 0x7fbbf8150140 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 --2- 172.21.15.99:0/3739227629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbf80d3550 0x7fbbf814d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.435+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 >> 172.21.15.99:0/3739227629 conn(0x7fbbf808afd0 msgr2=0x7fbbf806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.436+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 shutdown_connections 2024-03-29T09:10:21.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.436+0000 7fbbfe800700 1 -- 172.21.15.99:0/3739227629 wait complete. 2024-03-29T09:10:21.455 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.648+0000 7f27bb400700 1 Processor -- start 2024-03-29T09:10:21.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.648+0000 7f27bb400700 1 -- start start 2024-03-29T09:10:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.649+0000 7f27bb400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27b40cf090 0x7f27b4167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.649+0000 7f27bb400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b40d1520 con 0x7f27b4160770 2024-03-29T09:10:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.649+0000 7f27bb400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b40d1800 con 0x7f27b40b3cc0 2024-03-29T09:10:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.649+0000 7f27bb400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b40b4140 con 0x7f27b40cf090 2024-03-29T09:10:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b9400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27b40cf090 0x7f27b4167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b9400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f27b40cf090 0x7f27b4167f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35514/0 (socket says 172.21.15.99:35514) 2024-03-29T09:10:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b9400700 1 -- 172.21.15.99:0/2367749312 learned_addr learned my addr 172.21.15.99:0/2367749312 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b9400700 1 -- 172.21.15.99:0/2367749312 >> v1:172.21.15.133:6789/0 conn(0x7f27b40b3cc0 legacy=0x7f27b40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b9400700 1 -- 172.21.15.99:0/2367749312 >> v1:172.21.15.99:6789/0 conn(0x7f27b4160770 legacy=0x7f27b40d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b9400700 1 -- 172.21.15.99:0/2367749312 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b40d1ae0 con 0x7f27b40cf090 2024-03-29T09:10:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b3400700 1 -- 172.21.15.99:0/2367749312 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066928891 0 0) 0x7f27b40d1520 con 0x7f27b4160770 2024-03-29T09:10:21.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.650+0000 7f27b9400700 1 --2- 172.21.15.99:0/2367749312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b40cf090 0x7f27b4167f40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009e50 tx=0x7f27ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ba7a443c7a7c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27b3400700 1 -- 172.21.15.99:0/2367749312 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00e070 con 0x7f27b40cf090 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27b3400700 1 -- 172.21.15.99:0/2367749312 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac00b9c0 con 0x7f27b40cf090 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27b3400700 1 -- 172.21.15.99:0/2367749312 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00bd70 con 0x7f27b40cf090 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27bb400700 1 -- 172.21.15.99:0/2367749312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b40cf090 msgr2=0x7f27b4167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27bb400700 1 --2- 172.21.15.99:0/2367749312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b40cf090 0x7f27b4167f40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009e50 tx=0x7f27ac00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27bb400700 1 -- 172.21.15.99:0/2367749312 shutdown_connections 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27bb400700 1 --2- 172.21.15.99:0/2367749312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b40cf090 0x7f27b4167f40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27bb400700 1 -- 172.21.15.99:0/2367749312 >> 172.21.15.99:0/2367749312 conn(0x7f27b408afd0 msgr2=0x7f27b409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27bb400700 1 -- 172.21.15.99:0/2367749312 shutdown_connections 2024-03-29T09:10:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.651+0000 7f27bb400700 1 -- 172.21.15.99:0/2367749312 wait complete. 2024-03-29T09:10:21.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.653+0000 7f27bb400700 1 Processor -- start 2024-03-29T09:10:21.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.653+0000 7f27bb400700 1 -- start start 2024-03-29T09:10:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27bb400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27b40b3cc0 0x7f27b40c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27bb400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27b40cf090 0x7f27b415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27bb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4160770 0x7f27b415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27bb400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40ccdb0 con 0x7f27b4160770 2024-03-29T09:10:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27bb400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40ccfb0 con 0x7f27b40cf090 2024-03-29T09:10:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27bb400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b415a470 con 0x7f27b40b3cc0 2024-03-29T09:10:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27b9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4160770 0x7f27b415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27b9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4160770 0x7f27b415be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35518/0 (socket says 172.21.15.99:35518) 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27b9400700 1 -- 172.21.15.99:0/413076531 learned_addr learned my addr 172.21.15.99:0/413076531 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b3e00700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27b40cf090 0x7f27b415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.654+0000 7f27b8a00700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27b40b3cc0 0x7f27b40c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b9400700 1 -- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27b40b3cc0 msgr2=0x7f27b40c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b9400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27b40b3cc0 0x7f27b40c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b9400700 1 -- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27b40cf090 msgr2=0x7f27b415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b9400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27b40cf090 0x7f27b415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b9400700 1 -- 172.21.15.99:0/413076531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac009b30 con 0x7f27b4160770 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b9400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4160770 0x7f27b415be50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f27b40d0d10 tx=0x7f27ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:21.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b0c00700 1 -- 172.21.15.99:0/413076531 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac01c640 con 0x7f27b4160770 2024-03-29T09:10:21.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b0c00700 1 -- 172.21.15.99:0/413076531 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac01c7c0 con 0x7f27b4160770 2024-03-29T09:10:21.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b40cd250 con 0x7f27b4160770 2024-03-29T09:10:21.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27b0c00700 1 -- 172.21.15.99:0/413076531 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac01cb70 con 0x7f27b4160770 2024-03-29T09:10:21.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.655+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b40b7120 con 0x7f27b4160770 2024-03-29T09:10:21.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.656+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b40b7310 con 0x7f27b4160770 2024-03-29T09:10:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.657+0000 7f27b0c00700 1 -- 172.21.15.99:0/413076531 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f27ac01ccd0 con 0x7f27b4160770 2024-03-29T09:10:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.658+0000 7f27b0c00700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f279406acc0 0x7f279406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.658+0000 7f27b0c00700 1 -- 172.21.15.99:0/413076531 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f27ac08aa80 con 0x7f27b4160770 2024-03-29T09:10:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.658+0000 7f27b8a00700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f279406acc0 0x7f279406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.659+0000 7f27b8a00700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f279406acc0 0x7f279406d180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f27a8000f60 tx=0x7f27a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:21.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.662+0000 7f27b0c00700 1 -- 172.21.15.99:0/413076531 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f27ac055920 con 0x7f27b4160770 2024-03-29T09:10:21.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.866+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27b40678b0 con 0x7f27b4160770 2024-03-29T09:10:21.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.866+0000 7f27b0c00700 1 -- 172.21.15.99:0/413076531 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27ac058f40 con 0x7f27b4160770 2024-03-29T09:10:21.865 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f279406acc0 msgr2=0x7f279406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f279406acc0 0x7f279406d180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f27a8000f60 tx=0x7f27a8009040 comp rx=0 tx=0).stop 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4160770 msgr2=0x7f27b415be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4160770 0x7f27b415be50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f27b40d0d10 tx=0x7f27ac004ad0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 shutdown_connections 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f279406acc0 0x7f279406d180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f27b4160770 0x7f27b415be50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27b40b3cc0 0x7f27b40c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 --2- 172.21.15.99:0/413076531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27b40cf090 0x7f27b415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.868+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 >> 172.21.15.99:0/413076531 conn(0x7f27b408afd0 msgr2=0x7f27b40cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.869+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 shutdown_connections 2024-03-29T09:10:21.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:21.869+0000 7f27bb400700 1 -- 172.21.15.99:0/413076531 wait complete. 2024-03-29T09:10:21.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:22.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.080+0000 7f1b20e00700 1 Processor -- start 2024-03-29T09:10:22.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.080+0000 7f1b20e00700 1 -- start start 2024-03-29T09:10:22.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b20e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b20e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b1c0d0d60 0x7f1b1c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b20e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b1c0d3380 0x7f1b1c0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b20e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c0d3810 con 0x7f1b1c0d3380 2024-03-29T09:10:22.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b20e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c0d1520 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b20e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0d1800 con 0x7f1b1c160770 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b1a800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b1c0d3380 0x7f1b1c0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b1a800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b1c0d3380 0x7f1b1c0cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35532/0 (socket says 172.21.15.99:35532) 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b1a800700 1 -- 172.21.15.99:0/4251926648 learned_addr learned my addr 172.21.15.99:0/4251926648 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b13e00700 1 --2- 172.21.15.99:0/4251926648 >> v2:172.21.15.133:3300/0 conn(0x7f1b1c0d0d60 0x7f1b1c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.081+0000 7f1b19e00700 1 --2- 172.21.15.99:0/4251926648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b1a800700 1 -- 172.21.15.99:0/4251926648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 msgr2=0x7f1b1c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b1a800700 1 --2- 172.21.15.99:0/4251926648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b1a800700 1 -- 172.21.15.99:0/4251926648 >> v2:172.21.15.133:3300/0 conn(0x7f1b1c0d0d60 msgr2=0x7f1b1c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b1a800700 1 --2- 172.21.15.99:0/4251926648 >> v2:172.21.15.133:3300/0 conn(0x7f1b1c0d0d60 0x7f1b1c0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b1a800700 1 -- 172.21.15.99:0/4251926648 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c0d1ae0 con 0x7f1b1c0d3380 2024-03-29T09:10:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b1a800700 1 --2- 172.21.15.99:0/4251926648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cfa80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0b4be0 tx=0x7f1b1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d045c87d69c792e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b19400700 1 -- 172.21.15.99:0/4251926648 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400e070 con 0x7f1b1c0d3380 2024-03-29T09:10:22.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b19400700 1 -- 172.21.15.99:0/4251926648 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1400b910 con 0x7f1b1c0d3380 2024-03-29T09:10:22.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.082+0000 7f1b19400700 1 -- 172.21.15.99:0/4251926648 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400bcc0 con 0x7f1b1c0d3380 2024-03-29T09:10:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 -- 172.21.15.99:0/4251926648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d3380 msgr2=0x7f1b1c0cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 --2- 172.21.15.99:0/4251926648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cfa80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0b4be0 tx=0x7f1b1400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 -- 172.21.15.99:0/4251926648 shutdown_connections 2024-03-29T09:10:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 --2- 172.21.15.99:0/4251926648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 --2- 172.21.15.99:0/4251926648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cfa80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 --2- 172.21.15.99:0/4251926648 >> v2:172.21.15.133:3300/0 conn(0x7f1b1c0d0d60 0x7f1b1c0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 -- 172.21.15.99:0/4251926648 >> 172.21.15.99:0/4251926648 conn(0x7f1b1c08afd0 msgr2=0x7f1b1c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 -- 172.21.15.99:0/4251926648 shutdown_connections 2024-03-29T09:10:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.083+0000 7f1b20e00700 1 -- 172.21.15.99:0/4251926648 wait complete. 2024-03-29T09:10:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.085+0000 7f1b20e00700 1 Processor -- start 2024-03-29T09:10:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.085+0000 7f1b20e00700 1 -- start start 2024-03-29T09:10:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b20e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d0d60 0x7f1b1c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b20e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b20e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c14f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b20e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0b8590 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b20e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0b5d50 con 0x7f1b1c0d3380 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b20e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0cb7f0 con 0x7f1b1c160770 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d0d60 0x7f1b1c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d0d60 0x7f1b1c0b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35546/0 (socket says 172.21.15.99:35546) 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 -- 172.21.15.99:0/439749265 learned_addr learned my addr 172.21.15.99:0/439749265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b13e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b1a800700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c14f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 -- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 msgr2=0x7f1b1c14f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c14f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 -- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b1c0d3380 msgr2=0x7f1b1c0cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b19e00700 1 -- 172.21.15.99:0/439749265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b14009b30 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.086+0000 7f1b13e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.087+0000 7f1b19e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d0d60 0x7f1b1c0b8050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e870 tx=0x7f1b0c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.087+0000 7f1b12000700 1 -- 172.21.15.99:0/439749265 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c009590 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.087+0000 7f1b12000700 1 -- 172.21.15.99:0/439749265 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c00ee70 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.087+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c0cba70 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.087+0000 7f1b12000700 1 -- 172.21.15.99:0/439749265 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c00ca50 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.087+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c0cc030 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.088+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c0b6310 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.089+0000 7f1b12000700 1 -- 172.21.15.99:0/439749265 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b0c00cbb0 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.090+0000 7f1b12000700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1af806ac90 0x7f1af806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.090+0000 7f1b12000700 1 -- 172.21.15.99:0/439749265 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1b0c089600 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.090+0000 7f1b13e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1af806ac90 0x7f1af806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.091+0000 7f1b13e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1af806ac90 0x7f1af806d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c15cf20 tx=0x7f1b04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.094+0000 7f1b12000700 1 -- 172.21.15.99:0/439749265 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b0c054480 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.298+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b1c0678b0 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.298+0000 7f1b12000700 1 -- 172.21.15.99:0/439749265 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b0c00ce60 con 0x7f1b1c0d0d60 2024-03-29T09:10:22.297 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:22.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1af806ac90 msgr2=0x7f1af806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1af806ac90 0x7f1af806d150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c15cf20 tx=0x7f1b04008040 comp rx=0 tx=0).stop 2024-03-29T09:10:22.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d0d60 msgr2=0x7f1b1c0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d0d60 0x7f1b1c0b8050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00e870 tx=0x7f1b0c00ec30 comp rx=0 tx=0).stop 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 shutdown_connections 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1af806ac90 0x7f1af806d150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b1c160770 0x7f1b1c14f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b1c0d0d60 0x7f1b1c0b8050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 --2- 172.21.15.99:0/439749265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b1c0d3380 0x7f1b1c0cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 >> 172.21.15.99:0/439749265 conn(0x7f1b1c08afd0 msgr2=0x7f1b1c15ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 shutdown_connections 2024-03-29T09:10:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.301+0000 7f1b20e00700 1 -- 172.21.15.99:0/439749265 wait complete. 2024-03-29T09:10:22.318 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.511+0000 7f0f5ec00700 1 Processor -- start 2024-03-29T09:10:22.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.511+0000 7f0f5ec00700 1 -- start start 2024-03-29T09:10:22.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.512+0000 7f0f5ec00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0f580d1e90 0x7f0f58168130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.512+0000 7f0f5ec00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580afdd0 con 0x7f0f5815d800 2024-03-29T09:10:22.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.512+0000 7f0f5ec00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580d3b00 con 0x7f0f5815d420 2024-03-29T09:10:22.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.512+0000 7f0f5ec00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580d12b0 con 0x7f0f580d1e90 2024-03-29T09:10:22.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f57400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0f5815d800 0x7f0f580d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52642/0 (socket says 172.21.15.99:52642) 2024-03-29T09:10:22.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f57400700 1 -- 172.21.15.99:0/1366374772 learned_addr learned my addr 172.21.15.99:0/1366374772 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f5cc00700 1 --2- 172.21.15.99:0/1366374772 >> v2:172.21.15.133:3300/0 conn(0x7f0f580d1e90 0x7f0f58168130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f56a00700 1 -- 172.21.15.99:0/1366374772 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144292837 0 0) 0x7f0f580afdd0 con 0x7f0f5815d800 2024-03-29T09:10:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f56a00700 1 -- 172.21.15.99:0/1366374772 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f3c0035b0 con 0x7f0f5815d800 2024-03-29T09:10:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f5cc00700 1 -- 172.21.15.99:0/1366374772 >> v1:172.21.15.133:6789/0 conn(0x7f0f5815d420 legacy=0x7f0f580d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f5cc00700 1 -- 172.21.15.99:0/1366374772 >> v1:172.21.15.99:6789/0 conn(0x7f0f5815d800 legacy=0x7f0f580d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.513+0000 7f0f5cc00700 1 -- 172.21.15.99:0/1366374772 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580b0070 con 0x7f0f580d1e90 2024-03-29T09:10:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.514+0000 7f0f5cc00700 1 --2- 172.21.15.99:0/1366374772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f580d1e90 0x7f0f58168130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d90 tx=0x7f0f5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87712a858f7af190 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.514+0000 7f0f56a00700 1 -- 172.21.15.99:0/1366374772 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5000e070 con 0x7f0f580d1e90 2024-03-29T09:10:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.514+0000 7f0f56a00700 1 -- 172.21.15.99:0/1366374772 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5000bb20 con 0x7f0f580d1e90 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.514+0000 7f0f56a00700 1 -- 172.21.15.99:0/1366374772 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50012430 con 0x7f0f580d1e90 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.515+0000 7f0f5ec00700 1 -- 172.21.15.99:0/1366374772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f580d1e90 msgr2=0x7f0f58168130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.515+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/1366374772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f580d1e90 0x7f0f58168130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d90 tx=0x7f0f5000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.515+0000 7f0f5ec00700 1 -- 172.21.15.99:0/1366374772 shutdown_connections 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.515+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/1366374772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f580d1e90 0x7f0f58168130 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.515+0000 7f0f5ec00700 1 -- 172.21.15.99:0/1366374772 >> 172.21.15.99:0/1366374772 conn(0x7f0f5808afd0 msgr2=0x7f0f5809c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.515+0000 7f0f5ec00700 1 -- 172.21.15.99:0/1366374772 shutdown_connections 2024-03-29T09:10:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.515+0000 7f0f5ec00700 1 -- 172.21.15.99:0/1366374772 wait complete. 2024-03-29T09:10:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.517+0000 7f0f5ec00700 1 Processor -- start 2024-03-29T09:10:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.517+0000 7f0f5ec00700 1 -- start start 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.518+0000 7f0f5ec00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f580d1e90 0x7f0f5815fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.518+0000 7f0f5ec00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f5815d420 0x7f0f5815b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.518+0000 7f0f5ec00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f5815ecb0 0x7f0f580ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.518+0000 7f0f5ec00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581f5150 con 0x7f0f580d1e90 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.518+0000 7f0f5ec00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581f4260 con 0x7f0f5815ecb0 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.518+0000 7f0f5ec00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581f4540 con 0x7f0f5815d420 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f580d1e90 0x7f0f5815fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f580d1e90 0x7f0f5815fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35562/0 (socket says 172.21.15.99:35562) 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 -- 172.21.15.99:0/2749148111 learned_addr learned my addr 172.21.15.99:0/2749148111 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f5cc00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f5815ecb0 0x7f0f580ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57400700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f5815d420 0x7f0f5815b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 -- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f5815d420 msgr2=0x7f0f5815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f5815d420 0x7f0f5815b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 -- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f5815ecb0 msgr2=0x7f0f580ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f5815ecb0 0x7f0f580ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 -- 172.21.15.99:0/2749148111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50009a70 con 0x7f0f580d1e90 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f5cc00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f5815ecb0 0x7f0f580ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57400700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f5815d420 0x7f0f5815b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.519+0000 7f0f57e00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f580d1e90 0x7f0f5815fb20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c4e0 tx=0x7f0f4800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.520+0000 7f0f4fe00700 1 -- 172.21.15.99:0/2749148111 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480052d0 con 0x7f0f580d1e90 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.520+0000 7f0f4fe00700 1 -- 172.21.15.99:0/2749148111 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48005430 con 0x7f0f580d1e90 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.520+0000 7f0f4fe00700 1 -- 172.21.15.99:0/2749148111 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480038c0 con 0x7f0f580d1e90 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.520+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581f4820 con 0x7f0f580d1e90 2024-03-29T09:10:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.520+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581f4de0 con 0x7f0f580d1e90 2024-03-29T09:10:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.521+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f580c6b40 con 0x7f0f580d1e90 2024-03-29T09:10:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.522+0000 7f0f4fe00700 1 -- 172.21.15.99:0/2749148111 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0f48003a20 con 0x7f0f580d1e90 2024-03-29T09:10:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.522+0000 7f0f4fe00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0f3806ac90 0x7f0f3806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.523+0000 7f0f4fe00700 1 -- 172.21.15.99:0/2749148111 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0f48088bd0 con 0x7f0f580d1e90 2024-03-29T09:10:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.523+0000 7f0f57400700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0f3806ac90 0x7f0f3806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.523+0000 7f0f57400700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0f3806ac90 0x7f0f3806d150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0f580d39a0 tx=0x7f0f44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.527+0000 7f0f4fe00700 1 -- 172.21.15.99:0/2749148111 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0f480539c0 con 0x7f0f580d1e90 2024-03-29T09:10:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.728+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0f580678b0 con 0x7f0f580d1e90 2024-03-29T09:10:22.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.728+0000 7f0f4fe00700 1 -- 172.21.15.99:0/2749148111 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0f480170a0 con 0x7f0f580d1e90 2024-03-29T09:10:22.728 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0f3806ac90 msgr2=0x7f0f3806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0f3806ac90 0x7f0f3806d150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0f580d39a0 tx=0x7f0f44009040 comp rx=0 tx=0).stop 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f580d1e90 msgr2=0x7f0f5815fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f580d1e90 0x7f0f5815fb20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c4e0 tx=0x7f0f4800c8a0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 shutdown_connections 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0f3806ac90 0x7f0f3806d150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f5815ecb0 0x7f0f580ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0f580d1e90 0x7f0f5815fb20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 --2- 172.21.15.99:0/2749148111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f5815d420 0x7f0f5815b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 >> 172.21.15.99:0/2749148111 conn(0x7f0f5808afd0 msgr2=0x7f0f5806c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:22.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 shutdown_connections 2024-03-29T09:10:22.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.731+0000 7f0f5ec00700 1 -- 172.21.15.99:0/2749148111 wait complete. 2024-03-29T09:10:22.748 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:22.773 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:10:22.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:10:22.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.946+0000 7f3c1a000700 1 Processor -- start 2024-03-29T09:10:22.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.946+0000 7f3c1a000700 1 -- start start 2024-03-29T09:10:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.947+0000 7f3c1a000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3c1415d690 0x7f3c140cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.947+0000 7f3c1a000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3c1415da70 0x7f3c140cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.947+0000 7f3c1a000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.947+0000 7f3c1a000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c140d2110 con 0x7f3c1415da70 2024-03-29T09:10:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.947+0000 7f3c1a000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c140afde0 con 0x7f3c1415d690 2024-03-29T09:10:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.947+0000 7f3c1a000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140cf660 con 0x7f3c140d1740 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3c1415da70 0x7f3c140cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3c1415da70 0x7f3c140cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35578/0 (socket says 172.21.15.99:35578) 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 -- 172.21.15.99:0/3672891265 learned_addr learned my addr 172.21.15.99:0/3672891265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c13e00700 1 --2- 172.21.15.99:0/3672891265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c13400700 1 --2- 172.21.15.99:0/3672891265 >> v2:172.21.15.133:3300/0 conn(0x7f3c1415d690 0x7f3c140cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 -- 172.21.15.99:0/3672891265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 msgr2=0x7f3c140d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 --2- 172.21.15.99:0/3672891265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 -- 172.21.15.99:0/3672891265 >> v2:172.21.15.133:3300/0 conn(0x7f3c1415d690 msgr2=0x7f3c140cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 --2- 172.21.15.99:0/3672891265 >> v2:172.21.15.133:3300/0 conn(0x7f3c1415d690 0x7f3c140cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.948+0000 7f3c12a00700 1 -- 172.21.15.99:0/3672891265 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140d01d0 con 0x7f3c1415da70 2024-03-29T09:10:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c12a00700 1 --2- 172.21.15.99:0/3672891265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415da70 0x7f3c140cf120 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3c140d30b0 tx=0x7f3c00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abadadee80380fbb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c12000700 1 -- 172.21.15.99:0/3672891265 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00011830 con 0x7f3c1415da70 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c12000700 1 -- 172.21.15.99:0/3672891265 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c00011990 con 0x7f3c1415da70 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c12000700 1 -- 172.21.15.99:0/3672891265 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00011d20 con 0x7f3c1415da70 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 -- 172.21.15.99:0/3672891265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415da70 msgr2=0x7f3c140cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 --2- 172.21.15.99:0/3672891265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415da70 0x7f3c140cf120 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3c140d30b0 tx=0x7f3c00009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 -- 172.21.15.99:0/3672891265 shutdown_connections 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 --2- 172.21.15.99:0/3672891265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 --2- 172.21.15.99:0/3672891265 >> v2:172.21.15.133:3300/0 conn(0x7f3c1415d690 0x7f3c140cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 --2- 172.21.15.99:0/3672891265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415da70 0x7f3c140cf120 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 -- 172.21.15.99:0/3672891265 >> 172.21.15.99:0/3672891265 conn(0x7f3c1409c480 msgr2=0x7f3c140d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.949+0000 7f3c1a000700 1 -- 172.21.15.99:0/3672891265 shutdown_connections 2024-03-29T09:10:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.950+0000 7f3c1a000700 1 -- 172.21.15.99:0/3672891265 wait complete. 2024-03-29T09:10:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.951+0000 7f3c1a000700 1 Processor -- start 2024-03-29T09:10:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.951+0000 7f3c1a000700 1 -- start start 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.952+0000 7f3c1a000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.952+0000 7f3c1a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415d690 0x7f3c1414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.952+0000 7f3c1a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1415da70 0x7f3c140b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.952+0000 7f3c1a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1414b710 con 0x7f3c1415d690 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.952+0000 7f3c1a000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1414ae20 con 0x7f3c1415da70 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.952+0000 7f3c1a000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140b8cf0 con 0x7f3c140d1740 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.952+0000 7f3c12a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415d690 0x7f3c1414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415d690 0x7f3c1414d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35580/0 (socket says 172.21.15.99:35580) 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 -- 172.21.15.99:0/1358939668 learned_addr learned my addr 172.21.15.99:0/1358939668 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c13400700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c13e00700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1415da70 0x7f3c140b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 -- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 msgr2=0x7f3c140ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 -- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1415da70 msgr2=0x7f3c140b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1415da70 0x7f3c140b87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 -- 172.21.15.99:0/1358939668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c00009550 con 0x7f3c1415d690 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c12a00700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415d690 0x7f3c1414d860 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3c00000c00 tx=0x7f3c00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c0b400700 1 -- 172.21.15.99:0/1358939668 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00004240 con 0x7f3c1415d690 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c0b400700 1 -- 172.21.15.99:0/1358939668 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0001f070 con 0x7f3c1415d690 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1414b0c0 con 0x7f3c1415d690 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.953+0000 7f3c0b400700 1 -- 172.21.15.99:0/1358939668 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c000036a0 con 0x7f3c1415d690 2024-03-29T09:10:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.954+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c140b8f90 con 0x7f3c1415d690 2024-03-29T09:10:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.955+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c140677a0 con 0x7f3c1415d690 2024-03-29T09:10:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.955+0000 7f3c0b400700 1 -- 172.21.15.99:0/1358939668 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3c00021030 con 0x7f3c1415d690 2024-03-29T09:10:22.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.956+0000 7f3c0b400700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bf406ac90 0x7f3bf406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:22.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.956+0000 7f3c0b400700 1 -- 172.21.15.99:0/1358939668 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3c0008a810 con 0x7f3c1415d690 2024-03-29T09:10:22.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.956+0000 7f3c13400700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bf406ac90 0x7f3bf406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:22.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.957+0000 7f3c13400700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bf406ac90 0x7f3bf406d150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c140cf630 tx=0x7f3c04009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:22.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:22.961+0000 7f3c0b400700 1 -- 172.21.15.99:0/1358939668 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3c00055690 con 0x7f3c1415d690 2024-03-29T09:10:23.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.180+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3c140b43f0 con 0x7f3c1415d690 2024-03-29T09:10:23.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.181+0000 7f3c0b400700 1 -- 172.21.15.99:0/1358939668 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3c00058cb0 con 0x7f3c1415d690 2024-03-29T09:10:23.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bf406ac90 msgr2=0x7f3bf406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bf406ac90 0x7f3bf406d150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c140cf630 tx=0x7f3c04009210 comp rx=0 tx=0).stop 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415d690 msgr2=0x7f3c1414d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415d690 0x7f3c1414d860 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3c00000c00 tx=0x7f3c00004750 comp rx=0 tx=0).stop 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 shutdown_connections 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3bf406ac90 0x7f3bf406d150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c1415da70 0x7f3c140b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c140d1740 0x7f3c140ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 --2- 172.21.15.99:0/1358939668 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3c1415d690 0x7f3c1414d860 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 >> 172.21.15.99:0/1358939668 conn(0x7f3c1409c480 msgr2=0x7f3c1406c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 shutdown_connections 2024-03-29T09:10:23.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.183+0000 7f3c1a000700 1 -- 172.21.15.99:0/1358939668 wait complete. 2024-03-29T09:10:23.184 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:23.200 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:10:23.200 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:23.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:23.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.437+0000 7f6776e00700 1 Processor -- start 2024-03-29T09:10:23.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.437+0000 7f6776e00700 1 -- start start 2024-03-29T09:10:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.438+0000 7f6776e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 0x7f67700d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.438+0000 7f6776e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67700cddc0 con 0x7f67701608e0 2024-03-29T09:10:23.440 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.438+0000 7f6776e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67700ce0a0 con 0x7f6770160cc0 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.438+0000 7f6776e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700ce380 con 0x7f67700d1b20 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.438+0000 7f676fe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f67701608e0 0x7f67700d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52650/0 (socket says 172.21.15.99:52650) 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.438+0000 7f676fe00700 1 -- 172.21.15.99:0/3216493264 learned_addr learned my addr 172.21.15.99:0/3216493264 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.438+0000 7f676f400700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 0x7f67700d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463285922 0 0) 0x7f67700cddc0 con 0x7f67701608e0 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67540035b0 con 0x7f67701608e0 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450992456 0 0) 0x7f67700ce0a0 con 0x7f6770160cc0 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67700cddc0 con 0x7f6770160cc0 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1084202656 0 0) 0x7f67540035b0 con 0x7f67701608e0 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 msgr2=0x7f67700d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 0x7f67700d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 >> v1:172.21.15.133:6789/0 conn(0x7f6770160cc0 legacy=0x7f67700afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67700ce660 con 0x7f67701608e0 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f6760001670 con 0x7f67701608e0 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 >> v1:172.21.15.99:6789/0 conn(0x7f67701608e0 legacy=0x7f67700d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 0x7f6754003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6754004120 0x7f6754004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754004d90 0x7f6754005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700ce380 con 0x7f6754004d90 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675400baf0 con 0x7f6754004120 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.439+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675400bcb0 con 0x7f67700d1b20 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676fe00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 0x7f6754003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f6774e00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754004d90 0x7f6754005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676f400700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6754004120 0x7f6754004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676fe00700 1 -- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6754004120 msgr2=0x7f6754004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676fe00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6754004120 0x7f6754004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676fe00700 1 -- 172.21.15.99:0/3216493264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754004d90 msgr2=0x7f6754005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676fe00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754004d90 0x7f6754005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676fe00700 1 -- 172.21.15.99:0/3216493264 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540035b0 con 0x7f67700d1b20 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676f400700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6754004120 0x7f6754004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f6774e00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754004d90 0x7f6754005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676fe00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 0x7f6754003bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f676000c210 tx=0x7f6754005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ff91b1a96e1bdca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.440+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000cd50 con 0x7f67700d1b20 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760005960 con 0x7f67700d1b20 2024-03-29T09:10:23.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f676ea00700 1 -- 172.21.15.99:0/3216493264 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760005d10 con 0x7f67700d1b20 2024-03-29T09:10:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f6776e00700 1 -- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 msgr2=0x7f6754003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f6776e00700 1 --2- 172.21.15.99:0/3216493264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d1b20 0x7f6754003bc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f676000c210 tx=0x7f6754005880 comp rx=0 tx=0).stop 2024-03-29T09:10:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f6774e00700 1 -- 172.21.15.99:0/3216493264 reap_dead start 2024-03-29T09:10:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f6776e00700 1 -- 172.21.15.99:0/3216493264 shutdown_connections 2024-03-29T09:10:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f6776e00700 1 -- 172.21.15.99:0/3216493264 >> 172.21.15.99:0/3216493264 conn(0x7f677009c490 msgr2=0x7f67700d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f6776e00700 1 -- 172.21.15.99:0/3216493264 shutdown_connections 2024-03-29T09:10:23.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.441+0000 7f6776e00700 1 -- 172.21.15.99:0/3216493264 wait complete. 2024-03-29T09:10:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.443+0000 7f6776e00700 1 Processor -- start 2024-03-29T09:10:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.443+0000 7f6776e00700 1 -- start start 2024-03-29T09:10:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f6776e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67700d1b20 0x7f6770159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f6776e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677015bbd0 0x7f67701f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f6776e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d0d00 0x7f67700d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f6776e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700ca440 con 0x7f67700d1b20 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f6776e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67701f3430 con 0x7f677015bbd0 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f6776e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700d16c0 con 0x7f67700d0d00 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f676fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67700d1b20 0x7f6770159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f676fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67700d1b20 0x7f6770159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35596/0 (socket says 172.21.15.99:35596) 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f676fe00700 1 -- 172.21.15.99:0/3340822016 learned_addr learned my addr 172.21.15.99:0/3340822016 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f676f400700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677015bbd0 0x7f67701f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.444+0000 7f6774e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d0d00 0x7f67700d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f676fe00700 1 -- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d0d00 msgr2=0x7f67700d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f676fe00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d0d00 0x7f67700d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f676fe00700 1 -- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677015bbd0 msgr2=0x7f67701f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f676fe00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677015bbd0 0x7f67701f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f676fe00700 1 -- 172.21.15.99:0/3340822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67701f2a90 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f676fe00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67700d1b20 0x7f6770159b20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f67600018a0 tx=0x7f6760001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f6767e00700 1 -- 172.21.15.99:0/3340822016 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760004870 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f6767e00700 1 -- 172.21.15.99:0/3340822016 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67600049d0 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67701f2d70 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f6767e00700 1 -- 172.21.15.99:0/3340822016 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760004d40 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.445+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67701f2790 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.446+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67700ca3d0 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.447+0000 7f6767e00700 1 -- 172.21.15.99:0/3340822016 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67600070d0 con 0x7f67700d1b20 2024-03-29T09:10:23.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.448+0000 7f6767e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f675006ac90 0x7f675006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.448+0000 7f6767e00700 1 -- 172.21.15.99:0/3340822016 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f676008a720 con 0x7f67700d1b20 2024-03-29T09:10:23.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.448+0000 7f676f400700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f675006ac90 0x7f675006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.449+0000 7f676f400700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f675006ac90 0x7f675006d150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f677015f3d0 tx=0x7f675c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:23.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.452+0000 7f6767e00700 1 -- 172.21.15.99:0/3340822016 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6760055640 con 0x7f67700d1b20 2024-03-29T09:10:23.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.655+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67700677a0 con 0x7f67700d1b20 2024-03-29T09:10:23.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.657+0000 7f6767e00700 1 -- 172.21.15.99:0/3340822016 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f6760058c60 con 0x7f67700d1b20 2024-03-29T09:10:23.656 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:23.656 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f675006ac90 msgr2=0x7f675006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f675006ac90 0x7f675006d150 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f677015f3d0 tx=0x7f675c008040 comp rx=0 tx=0).stop 2024-03-29T09:10:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67700d1b20 msgr2=0x7f6770159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67700d1b20 0x7f6770159b20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f67600018a0 tx=0x7f6760001980 comp rx=0 tx=0).stop 2024-03-29T09:10:23.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 shutdown_connections 2024-03-29T09:10:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f675006ac90 0x7f675006d150 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67700d0d00 0x7f67700d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67700d1b20 0x7f6770159b20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 --2- 172.21.15.99:0/3340822016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f677015bbd0 0x7f67701f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 >> 172.21.15.99:0/3340822016 conn(0x7f677009c490 msgr2=0x7f67700cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 shutdown_connections 2024-03-29T09:10:23.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.659+0000 7f6776e00700 1 -- 172.21.15.99:0/3340822016 wait complete. 2024-03-29T09:10:23.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:23.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:23.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.869+0000 7f720b800700 1 Processor -- start 2024-03-29T09:10:23.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.870+0000 7f720b800700 1 -- start start 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f720b800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f72040b3cc0 0x7f72040d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f720b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 0x7f72040ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f720b800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72040d3810 con 0x7f7204160770 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f720b800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040b4140 con 0x7f72040b3cc0 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f720b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72040d1520 con 0x7f72040d3380 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f7203e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f72040b3cc0 0x7f72040d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f7208e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7204160770 0x7f72040d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52666/0 (socket says 172.21.15.99:52666) 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f7208e00700 1 -- 172.21.15.99:0/688589597 learned_addr learned my addr 172.21.15.99:0/688589597 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f7203e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f72040b3cc0 0x7f72040d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35604/0 (socket says 172.21.15.99:35604) 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.871+0000 7f7209800700 1 --2- 172.21.15.99:0/688589597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 0x7f72040ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203e00700 1 -- 172.21.15.99:0/688589597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 msgr2=0x7f72040ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203e00700 1 --2- 172.21.15.99:0/688589597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 0x7f72040ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203e00700 1 -- 172.21.15.99:0/688589597 >> v1:172.21.15.99:6789/0 conn(0x7f7204160770 legacy=0x7f72040d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203e00700 1 -- 172.21.15.99:0/688589597 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72040d1800 con 0x7f72040b3cc0 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203400700 1 -- 172.21.15.99:0/688589597 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706982619 0 0) 0x7f72040d3810 con 0x7f7204160770 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203e00700 1 --2- 172.21.15.99:0/688589597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f72040d2e40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009840 tx=0x7f71f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481a0e27acc79e79 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203400700 1 -- 172.21.15.99:0/688589597 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011830 con 0x7f72040b3cc0 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203400700 1 -- 172.21.15.99:0/688589597 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0011990 con 0x7f72040b3cc0 2024-03-29T09:10:23.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.872+0000 7f7203400700 1 -- 172.21.15.99:0/688589597 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011d20 con 0x7f72040b3cc0 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 -- 172.21.15.99:0/688589597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 msgr2=0x7f72040d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 --2- 172.21.15.99:0/688589597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f72040d2e40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009840 tx=0x7f71f0009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 -- 172.21.15.99:0/688589597 shutdown_connections 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 --2- 172.21.15.99:0/688589597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 0x7f72040ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 --2- 172.21.15.99:0/688589597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f72040d2e40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 -- 172.21.15.99:0/688589597 >> 172.21.15.99:0/688589597 conn(0x7f720408afd0 msgr2=0x7f720409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 -- 172.21.15.99:0/688589597 shutdown_connections 2024-03-29T09:10:23.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.873+0000 7f720b800700 1 -- 172.21.15.99:0/688589597 wait complete. 2024-03-29T09:10:23.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.874+0000 7f720b800700 1 Processor -- start 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.875+0000 7f720b800700 1 -- start start 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.875+0000 7f720b800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f7204151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.875+0000 7f720b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 0x7f72040a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f720b800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7204160770 0x7f72040cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f720b800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720409d2f0 con 0x7f72040b3cc0 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f720b800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040a0410 con 0x7f7204160770 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f720b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b6960 con 0x7f72040d3380 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f7204151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f7204151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35606/0 (socket says 172.21.15.99:35606) 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 -- 172.21.15.99:0/25975607 learned_addr learned my addr 172.21.15.99:0/25975607 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7209800700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7204160770 0x7f72040cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 -- 172.21.15.99:0/25975607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 msgr2=0x7f72040a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 0x7f72040a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 -- 172.21.15.99:0/25975607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7204160770 msgr2=0x7f72040cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7204160770 0x7f72040cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 -- 172.21.15.99:0/25975607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00094a0 con 0x7f72040b3cc0 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.876+0000 7f7208e00700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f7204151580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c550 tx=0x7f71f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.877+0000 7f7200c00700 1 -- 172.21.15.99:0/25975607 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8004c80 con 0x7f72040b3cc0 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.877+0000 7f7200c00700 1 -- 172.21.15.99:0/25975607 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8004e00 con 0x7f72040b3cc0 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.877+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72040b6c40 con 0x7f72040b3cc0 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.877+0000 7f7200c00700 1 -- 172.21.15.99:0/25975607 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007310 con 0x7f72040b3cc0 2024-03-29T09:10:23.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.877+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7204151ac0 con 0x7f72040b3cc0 2024-03-29T09:10:23.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.878+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720406be50 con 0x7f72040b3cc0 2024-03-29T09:10:23.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.879+0000 7f7200c00700 1 -- 172.21.15.99:0/25975607 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f71f8007470 con 0x7f72040b3cc0 2024-03-29T09:10:23.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.879+0000 7f7200c00700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71e406ac90 0x7f71e406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:23.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.880+0000 7f7200c00700 1 -- 172.21.15.99:0/25975607 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f71f8089450 con 0x7f72040b3cc0 2024-03-29T09:10:23.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.880+0000 7f7203e00700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71e406ac90 0x7f71e406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:23.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.880+0000 7f7203e00700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71e406ac90 0x7f71e406d150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7204155180 tx=0x7f71f0010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:23.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:23.884+0000 7f7200c00700 1 -- 172.21.15.99:0/25975607 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f71f8054240 con 0x7f72040b3cc0 2024-03-29T09:10:24.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.085+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72040678b0 con 0x7f72040b3cc0 2024-03-29T09:10:24.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.085+0000 7f7200c00700 1 -- 172.21.15.99:0/25975607 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71f8057860 con 0x7f72040b3cc0 2024-03-29T09:10:24.084 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71e406ac90 msgr2=0x7f71e406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71e406ac90 0x7f71e406d150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7204155180 tx=0x7f71f0010000 comp rx=0 tx=0).stop 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 msgr2=0x7f7204151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f7204151580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c550 tx=0x7f71f800c910 comp rx=0 tx=0).stop 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 shutdown_connections 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71e406ac90 0x7f71e406d150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7204160770 0x7f72040cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f72040b3cc0 0x7f7204151580 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 --2- 172.21.15.99:0/25975607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72040d3380 0x7f72040a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 >> 172.21.15.99:0/25975607 conn(0x7f720408afd0 msgr2=0x7f720406c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:24.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 shutdown_connections 2024-03-29T09:10:24.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.088+0000 7f720b800700 1 -- 172.21.15.99:0/25975607 wait complete. 2024-03-29T09:10:24.106 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:24.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.301+0000 7f5215e00700 1 Processor -- start 2024-03-29T09:10:24.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.302+0000 7f5215e00700 1 -- start start 2024-03-29T09:10:24.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.303+0000 7f5215e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f521015d800 0x7f52100d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.303+0000 7f5215e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52100d1e90 0x7f52100d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.303+0000 7f5215e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52100d38d0 con 0x7f521015d420 2024-03-29T09:10:24.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.303+0000 7f5215e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52100d12b0 con 0x7f521015d800 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.303+0000 7f5215e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100afdd0 con 0x7f52100d1e90 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.303+0000 7f520e000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f521015d800 0x7f52100d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520e000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f521015d800 0x7f52100d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35612/0 (socket says 172.21.15.99:35612) 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520e000700 1 -- 172.21.15.99:0/995196026 learned_addr learned my addr 172.21.15.99:0/995196026 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520f400700 1 --2- 172.21.15.99:0/995196026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52100d1e90 0x7f52100d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520e000700 1 -- 172.21.15.99:0/995196026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52100d1e90 msgr2=0x7f52100d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520e000700 1 --2- 172.21.15.99:0/995196026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52100d1e90 0x7f52100d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520e000700 1 -- 172.21.15.99:0/995196026 >> v1:172.21.15.133:6789/0 conn(0x7f521015d420 legacy=0x7f52100d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520e000700 1 -- 172.21.15.99:0/995196026 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100d3b00 con 0x7f521015d800 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520e000700 1 --2- 172.21.15.99:0/995196026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521015d800 0x7f52100d2e50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0098f0 tx=0x7f51fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39de2c636af07deb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520d600700 1 -- 172.21.15.99:0/995196026 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc011830 con 0x7f521015d800 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520d600700 1 -- 172.21.15.99:0/995196026 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc011990 con 0x7f521015d800 2024-03-29T09:10:24.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.304+0000 7f520d600700 1 -- 172.21.15.99:0/995196026 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc011d20 con 0x7f521015d800 2024-03-29T09:10:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 -- 172.21.15.99:0/995196026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521015d800 msgr2=0x7f52100d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 --2- 172.21.15.99:0/995196026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521015d800 0x7f52100d2e50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0098f0 tx=0x7f51fc009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:24.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 -- 172.21.15.99:0/995196026 shutdown_connections 2024-03-29T09:10:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 --2- 172.21.15.99:0/995196026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52100d1e90 0x7f52100d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 --2- 172.21.15.99:0/995196026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f521015d800 0x7f52100d2e50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 -- 172.21.15.99:0/995196026 >> 172.21.15.99:0/995196026 conn(0x7f521008afd0 msgr2=0x7f521009c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 -- 172.21.15.99:0/995196026 shutdown_connections 2024-03-29T09:10:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.305+0000 7f5215e00700 1 -- 172.21.15.99:0/995196026 wait complete. 2024-03-29T09:10:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.307+0000 7f5215e00700 1 Processor -- start 2024-03-29T09:10:24.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.308+0000 7f5215e00700 1 -- start start 2024-03-29T09:10:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f5215e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52100d1e90 0x7f52101f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f5215e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f521015d420 0x7f52101f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f5215e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52101f66f0 0x7f52100ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f5215e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52101f6ba0 con 0x7f52100d1e90 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f5215e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521015f7e0 con 0x7f52101f66f0 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f5215e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521015ebd0 con 0x7f521015d420 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52100d1e90 0x7f52101f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52100d1e90 0x7f52101f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35628/0 (socket says 172.21.15.99:35628) 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 -- 172.21.15.99:0/3284823734 learned_addr learned my addr 172.21.15.99:0/3284823734 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520f400700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52101f66f0 0x7f52100ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520e000700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f521015d420 0x7f52101f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 -- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f521015d420 msgr2=0x7f52101f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f521015d420 0x7f52101f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 -- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52101f66f0 msgr2=0x7f52100ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52101f66f0 0x7f52100ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520ea00700 1 -- 172.21.15.99:0/3284823734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009550 con 0x7f52100d1e90 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.309+0000 7f520e000700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f521015d420 0x7f52101f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.310+0000 7f520ea00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52100d1e90 0x7f52101f23c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f520000c510 tx=0x7f520000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.310+0000 7f5207400700 1 -- 172.21.15.99:0/3284823734 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000052d0 con 0x7f52100d1e90 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.310+0000 7f5207400700 1 -- 172.21.15.99:0/3284823734 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200005430 con 0x7f52100d1e90 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.310+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521015eeb0 con 0x7f52100d1e90 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.310+0000 7f5207400700 1 -- 172.21.15.99:0/3284823734 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000038c0 con 0x7f52100d1e90 2024-03-29T09:10:24.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.310+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521015f470 con 0x7f52100d1e90 2024-03-29T09:10:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.311+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52100678b0 con 0x7f52100d1e90 2024-03-29T09:10:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.312+0000 7f5207400700 1 -- 172.21.15.99:0/3284823734 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5200003a20 con 0x7f52100d1e90 2024-03-29T09:10:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.313+0000 7f5207400700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51f006ac90 0x7f51f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.313+0000 7f5207400700 1 -- 172.21.15.99:0/3284823734 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5200088b10 con 0x7f52100d1e90 2024-03-29T09:10:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.313+0000 7f520e000700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51f006ac90 0x7f51f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.314+0000 7f520e000700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51f006ac90 0x7f51f006d150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5210160bc0 tx=0x7f51fc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:24.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.317+0000 7f5207400700 1 -- 172.21.15.99:0/3284823734 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5200053990 con 0x7f52100d1e90 2024-03-29T09:10:24.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.519+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52100b23d0 con 0x7f52100d1e90 2024-03-29T09:10:24.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.520+0000 7f5207400700 1 -- 172.21.15.99:0/3284823734 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5200017020 con 0x7f52100d1e90 2024-03-29T09:10:24.519 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51f006ac90 msgr2=0x7f51f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51f006ac90 0x7f51f006d150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5210160bc0 tx=0x7f51fc008030 comp rx=0 tx=0).stop 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52100d1e90 msgr2=0x7f52101f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52100d1e90 0x7f52101f23c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f520000c510 tx=0x7f520000c8d0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 shutdown_connections 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f51f006ac90 0x7f51f006d150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52101f66f0 0x7f52100ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52100d1e90 0x7f52101f23c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 --2- 172.21.15.99:0/3284823734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f521015d420 0x7f52101f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.522+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 >> 172.21.15.99:0/3284823734 conn(0x7f521008afd0 msgr2=0x7f52100b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:24.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.523+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 shutdown_connections 2024-03-29T09:10:24.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.523+0000 7f5215e00700 1 -- 172.21.15.99:0/3284823734 wait complete. 2024-03-29T09:10:24.540 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:24.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.733+0000 7fc279200700 1 Processor -- start 2024-03-29T09:10:24.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.734+0000 7fc279200700 1 -- start start 2024-03-29T09:10:24.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.735+0000 7fc279200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc27415d800 0x7fc2740d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.735+0000 7fc279200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2740d1e90 0x7fc2740d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.735+0000 7fc279200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2740d38d0 con 0x7fc27415d420 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.735+0000 7fc279200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2740d12b0 con 0x7fc2740d1e90 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.735+0000 7fc279200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740afdd0 con 0x7fc27415d800 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2740d1e90 0x7fc2740d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc2740d1e90 0x7fc2740d3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35638/0 (socket says 172.21.15.99:35638) 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 -- 172.21.15.99:0/253379690 learned_addr learned my addr 172.21.15.99:0/253379690 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc271400700 1 --2- 172.21.15.99:0/253379690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc27415d800 0x7fc2740d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 -- 172.21.15.99:0/253379690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc27415d800 msgr2=0x7fc2740d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 --2- 172.21.15.99:0/253379690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc27415d800 0x7fc2740d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 -- 172.21.15.99:0/253379690 >> v1:172.21.15.133:6789/0 conn(0x7fc27415d420 legacy=0x7fc2740d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 -- 172.21.15.99:0/253379690 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2740d3b00 con 0x7fc2740d1e90 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.736+0000 7fc272800700 1 --2- 172.21.15.99:0/253379690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2740d1e90 0x7fc2740d3390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009e50 tx=0x7fc26c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2898ef8a7ec9ca7a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc270a00700 1 -- 172.21.15.99:0/253379690 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00e070 con 0x7fc2740d1e90 2024-03-29T09:10:24.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc270a00700 1 -- 172.21.15.99:0/253379690 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c00b910 con 0x7fc2740d1e90 2024-03-29T09:10:24.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc270a00700 1 -- 172.21.15.99:0/253379690 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c00bcc0 con 0x7fc2740d1e90 2024-03-29T09:10:24.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 -- 172.21.15.99:0/253379690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2740d1e90 msgr2=0x7fc2740d3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 --2- 172.21.15.99:0/253379690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2740d1e90 0x7fc2740d3390 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009e50 tx=0x7fc26c00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 -- 172.21.15.99:0/253379690 shutdown_connections 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 --2- 172.21.15.99:0/253379690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2740d1e90 0x7fc2740d3390 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 --2- 172.21.15.99:0/253379690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc27415d800 0x7fc2740d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 -- 172.21.15.99:0/253379690 >> 172.21.15.99:0/253379690 conn(0x7fc27408afd0 msgr2=0x7fc27409c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 -- 172.21.15.99:0/253379690 shutdown_connections 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.737+0000 7fc279200700 1 -- 172.21.15.99:0/253379690 wait complete. 2024-03-29T09:10:24.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.739+0000 7fc279200700 1 Processor -- start 2024-03-29T09:10:24.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.739+0000 7fc279200700 1 -- start start 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc279200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2740d1e90 0x7fc274152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc279200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc27415d420 0x7fc2740c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc279200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc274152980 0x7fc27415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc279200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740c7780 con 0x7fc27415d420 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc279200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740c87e0 con 0x7fc2740d1e90 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc279200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740c7a20 con 0x7fc274152980 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc271400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc27415d420 0x7fc2740c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc271400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc27415d420 0x7fc2740c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35646/0 (socket says 172.21.15.99:35646) 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc271400700 1 -- 172.21.15.99:0/1646037000 learned_addr learned my addr 172.21.15.99:0/1646037000 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc272800700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc274152980 0x7fc27415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.740+0000 7fc271e00700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2740d1e90 0x7fc274152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc271400700 1 -- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc274152980 msgr2=0x7fc27415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc271400700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc274152980 0x7fc27415f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc271400700 1 -- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2740d1e90 msgr2=0x7fc274152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc271400700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2740d1e90 0x7fc274152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc271400700 1 -- 172.21.15.99:0/1646037000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c009b30 con 0x7fc27415d420 2024-03-29T09:10:24.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc271400700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc27415d420 0x7fc2740c8be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc26000e870 tx=0x7fc26000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:24.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc26a000700 1 -- 172.21.15.99:0/1646037000 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260009590 con 0x7fc27415d420 2024-03-29T09:10:24.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc26a000700 1 -- 172.21.15.99:0/1646037000 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260013680 con 0x7fc27415d420 2024-03-29T09:10:24.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc26a000700 1 -- 172.21.15.99:0/1646037000 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2600139f0 con 0x7fc27415d420 2024-03-29T09:10:24.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2740c7d00 con 0x7fc27415d420 2024-03-29T09:10:24.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.741+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2740c82c0 con 0x7fc27415d420 2024-03-29T09:10:24.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.742+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2740c84b0 con 0x7fc27415d420 2024-03-29T09:10:24.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.743+0000 7fc26a000700 1 -- 172.21.15.99:0/1646037000 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc260013b50 con 0x7fc27415d420 2024-03-29T09:10:24.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.744+0000 7fc26a000700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc25406ac90 0x7fc25406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:24.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.744+0000 7fc26a000700 1 -- 172.21.15.99:0/1646037000 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc260089600 con 0x7fc27415d420 2024-03-29T09:10:24.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.744+0000 7fc271e00700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc25406ac90 0x7fc25406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:24.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.745+0000 7fc271e00700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc25406ac90 0x7fc25406d150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc264007fd0 tx=0x7fc264009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:24.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.748+0000 7fc26a000700 1 -- 172.21.15.99:0/1646037000 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc2600543f0 con 0x7fc27415d420 2024-03-29T09:10:24.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.955+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2740678b0 con 0x7fc27415d420 2024-03-29T09:10:24.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.955+0000 7fc26a000700 1 -- 172.21.15.99:0/1646037000 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc260057a10 con 0x7fc27415d420 2024-03-29T09:10:24.954 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc25406ac90 msgr2=0x7fc25406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc25406ac90 0x7fc25406d150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc264007fd0 tx=0x7fc264009040 comp rx=0 tx=0).stop 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc27415d420 msgr2=0x7fc2740c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc27415d420 0x7fc2740c8be0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc26000e870 tx=0x7fc26000eb80 comp rx=0 tx=0).stop 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 shutdown_connections 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc25406ac90 0x7fc25406d150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc274152980 0x7fc27415f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2740d1e90 0x7fc274152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 --2- 172.21.15.99:0/1646037000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc27415d420 0x7fc2740c8be0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.957+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 >> 172.21.15.99:0/1646037000 conn(0x7fc27408afd0 msgr2=0x7fc2740d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:24.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.958+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 shutdown_connections 2024-03-29T09:10:24.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:24.958+0000 7fc279200700 1 -- 172.21.15.99:0/1646037000 wait complete. 2024-03-29T09:10:24.975 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:25.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.170+0000 7effb9a00700 1 Processor -- start 2024-03-29T09:10:25.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.171+0000 7effb9a00700 1 -- start start 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb9a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40ae280 0x7effb406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb9a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effb40d07d0 0x7effb4071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb9a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb40af8f0 con 0x7effb40ae660 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb9a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb415d7e0 con 0x7effb40d07d0 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb9a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb40aed70 con 0x7effb40ae280 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb3400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effb40d07d0 0x7effb4071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb2a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40ae280 0x7effb406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb3400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effb40d07d0 0x7effb4071ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:49814/0 (socket says 172.21.15.99:49814) 2024-03-29T09:10:25.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.172+0000 7effb3400700 1 -- 172.21.15.99:0/2389155163 learned_addr learned my addr 172.21.15.99:0/2389155163 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb3400700 1 -- 172.21.15.99:0/2389155163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40ae280 msgr2=0x7effb406d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb3400700 1 --2- 172.21.15.99:0/2389155163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40ae280 0x7effb406d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb3400700 1 -- 172.21.15.99:0/2389155163 >> v1:172.21.15.133:6789/0 conn(0x7effb40ae660 legacy=0x7effb40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb3400700 1 -- 172.21.15.99:0/2389155163 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40af050 con 0x7effb40d07d0 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb3400700 1 --2- 172.21.15.99:0/2389155163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40d07d0 0x7effb4071ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7effac009e50 tx=0x7effac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59b1f68bfec44c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb1600700 1 -- 172.21.15.99:0/2389155163 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac00e070 con 0x7effb40d07d0 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb1600700 1 -- 172.21.15.99:0/2389155163 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac00b910 con 0x7effb40d07d0 2024-03-29T09:10:25.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.173+0000 7effb1600700 1 -- 172.21.15.99:0/2389155163 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac00bcc0 con 0x7effb40d07d0 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 -- 172.21.15.99:0/2389155163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40d07d0 msgr2=0x7effb4071ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 --2- 172.21.15.99:0/2389155163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40d07d0 0x7effb4071ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7effac009e50 tx=0x7effac00b330 comp rx=0 tx=0).stop 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 -- 172.21.15.99:0/2389155163 shutdown_connections 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 --2- 172.21.15.99:0/2389155163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40d07d0 0x7effb4071ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 --2- 172.21.15.99:0/2389155163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40ae280 0x7effb406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 -- 172.21.15.99:0/2389155163 >> 172.21.15.99:0/2389155163 conn(0x7effb4066720 msgr2=0x7effb4066b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 -- 172.21.15.99:0/2389155163 shutdown_connections 2024-03-29T09:10:25.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.174+0000 7effb9a00700 1 -- 172.21.15.99:0/2389155163 wait complete. 2024-03-29T09:10:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.176+0000 7effb9a00700 1 Processor -- start 2024-03-29T09:10:25.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.176+0000 7effb9a00700 1 -- start start 2024-03-29T09:10:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb9a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb40ae280 0x7effb41f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb9a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40ae660 0x7effb40b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb9a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40d07d0 0x7effb40b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb9a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb415e780 con 0x7effb40ae280 2024-03-29T09:10:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb9a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb414f480 con 0x7effb40ae660 2024-03-29T09:10:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb9a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb415de50 con 0x7effb40d07d0 2024-03-29T09:10:25.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb40ae280 0x7effb41f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb40ae280 0x7effb41f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35662/0 (socket says 172.21.15.99:35662) 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb2a00700 1 -- 172.21.15.99:0/3008374589 learned_addr learned my addr 172.21.15.99:0/3008374589 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb3400700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40d07d0 0x7effb40b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.177+0000 7effb2000700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40ae660 0x7effb40b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb2a00700 1 -- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40d07d0 msgr2=0x7effb40b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb2a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40d07d0 0x7effb40b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb2a00700 1 -- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40ae660 msgr2=0x7effb40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb2a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40ae660 0x7effb40b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb2a00700 1 -- 172.21.15.99:0/3008374589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac009b30 con 0x7effb40ae280 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb2a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb40ae280 0x7effb41f20d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7effa400e9b0 tx=0x7effa400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:25.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effaaa00700 1 -- 172.21.15.99:0/3008374589 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4009710 con 0x7effb40ae280 2024-03-29T09:10:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effaaa00700 1 -- 172.21.15.99:0/3008374589 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa400ee50 con 0x7effb40ae280 2024-03-29T09:10:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb415e130 con 0x7effb40ae280 2024-03-29T09:10:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effaaa00700 1 -- 172.21.15.99:0/3008374589 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa400ca50 con 0x7effb40ae280 2024-03-29T09:10:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.178+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb41f2610 con 0x7effb40ae280 2024-03-29T09:10:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.179+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb414c8e0 con 0x7effb40ae280 2024-03-29T09:10:25.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.180+0000 7effaaa00700 1 -- 172.21.15.99:0/3008374589 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7effa400cbb0 con 0x7effb40ae280 2024-03-29T09:10:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.181+0000 7effaaa00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff9406acc0 0x7eff9406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.181+0000 7effaaa00700 1 -- 172.21.15.99:0/3008374589 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7effa40895c0 con 0x7effb40ae280 2024-03-29T09:10:25.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.181+0000 7effb2000700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff9406acc0 0x7eff9406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.182+0000 7effb2000700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff9406acc0 0x7eff9406d180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7effb414c120 tx=0x7effa0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:25.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.185+0000 7effaaa00700 1 -- 172.21.15.99:0/3008374589 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7effa4054450 con 0x7effb40ae280 2024-03-29T09:10:25.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.388+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7effb40b39a0 con 0x7effb40ae280 2024-03-29T09:10:25.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.389+0000 7effaaa00700 1 -- 172.21.15.99:0/3008374589 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7effa405ce60 con 0x7effb40ae280 2024-03-29T09:10:25.388 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff9406acc0 msgr2=0x7eff9406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff9406acc0 0x7eff9406d180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7effb414c120 tx=0x7effa0009040 comp rx=0 tx=0).stop 2024-03-29T09:10:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb40ae280 msgr2=0x7effb41f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb40ae280 0x7effb41f20d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7effa400e9b0 tx=0x7effa400ecc0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 shutdown_connections 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7eff9406acc0 0x7eff9406d180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7effb40d07d0 0x7effb40b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effb40ae280 0x7effb41f20d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 --2- 172.21.15.99:0/3008374589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effb40ae660 0x7effb40b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 >> 172.21.15.99:0/3008374589 conn(0x7effb4066720 msgr2=0x7effb40d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 shutdown_connections 2024-03-29T09:10:25.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.391+0000 7effb9a00700 1 -- 172.21.15.99:0/3008374589 wait complete. 2024-03-29T09:10:25.409 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:25.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.602+0000 7f0790000700 1 Processor -- start 2024-03-29T09:10:25.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.603+0000 7f0790000700 1 -- start start 2024-03-29T09:10:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f0790000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 0x7f07880d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f0790000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07880cddc0 con 0x7f07881608e0 2024-03-29T09:10:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f0790000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07880ce0a0 con 0x7f0788160cc0 2024-03-29T09:10:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f0790000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07880ce380 con 0x7f07880d1b20 2024-03-29T09:10:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f078d600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f07881608e0 0x7f07880d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52682/0 (socket says 172.21.15.99:52682) 2024-03-29T09:10:25.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f078d600700 1 -- 172.21.15.99:0/3329372768 learned_addr learned my addr 172.21.15.99:0/3329372768 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f078cc00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 0x7f07880d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641189922 0 0) 0x7f07880cddc0 con 0x7f07881608e0 2024-03-29T09:10:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.604+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07700035b0 con 0x7f07881608e0 2024-03-29T09:10:25.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423753024 0 0) 0x7f07880ce0a0 con 0x7f0788160cc0 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07880cddc0 con 0x7f0788160cc0 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 218260107 0 0) 0x7f07700035b0 con 0x7f07881608e0 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 msgr2=0x7f07880d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 0x7f07880d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 >> v1:172.21.15.133:6789/0 conn(0x7f0788160cc0 legacy=0x7f07880afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880ce660 con 0x7f07881608e0 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f077c001670 con 0x7f07881608e0 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 >> v1:172.21.15.99:6789/0 conn(0x7f07881608e0 legacy=0x7f07880d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 0x7f0770003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0770004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770004d90 0x7f0770005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07880ce380 con 0x7f0770004100 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077000ba70 con 0x7f0770004d90 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077000bc30 con 0x7f07880d1b20 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f078cc00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0770004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f078d600700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 0x7f0770003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.605+0000 7f078e000700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770004d90 0x7f0770005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078cc00700 1 -- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 msgr2=0x7f0770003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078cc00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880d1b20 0x7f0770003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078cc00700 1 -- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770004d90 msgr2=0x7f0770005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078cc00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770004d90 0x7f0770005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078cc00700 1 -- 172.21.15.99:0/3329372768 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700035b0 con 0x7f0770004100 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078e000700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770004d90 0x7f0770005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078cc00700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0770004850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0774009b60 tx=0x7f0770005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f163ac71ca7cc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:25.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077400e6c0 con 0x7f0770004100 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400e820 con 0x7f0770004100 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f0787e00700 1 -- 172.21.15.99:0/3329372768 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077400eb90 con 0x7f0770004100 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f0790000700 1 -- 172.21.15.99:0/3329372768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 msgr2=0x7f0770004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f0790000700 1 --2- 172.21.15.99:0/3329372768 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0770004850 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0774009b60 tx=0x7f0770005880 comp rx=0 tx=0).stop 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.606+0000 7f078e000700 1 -- 172.21.15.99:0/3329372768 reap_dead start 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.607+0000 7f0790000700 1 -- 172.21.15.99:0/3329372768 shutdown_connections 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.607+0000 7f0790000700 1 -- 172.21.15.99:0/3329372768 >> 172.21.15.99:0/3329372768 conn(0x7f078809c490 msgr2=0x7f07880d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.607+0000 7f0790000700 1 -- 172.21.15.99:0/3329372768 shutdown_connections 2024-03-29T09:10:25.609 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.607+0000 7f0790000700 1 -- 172.21.15.99:0/3329372768 wait complete. 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.608+0000 7f0790000700 1 Processor -- start 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.609+0000 7f0790000700 1 -- start start 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.609+0000 7f0790000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0788156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.609+0000 7f0790000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880b3b90 0x7f078815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f0790000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078815acb0 0x7f078809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f0790000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078815b150 con 0x7f0770004100 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f0790000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078815b430 con 0x7f078815acb0 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f0790000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078815b710 con 0x7f07880b3b90 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0788156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0788156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35668/0 (socket says 172.21.15.99:35668) 2024-03-29T09:10:25.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 -- 172.21.15.99:0/1680990836 learned_addr learned my addr 172.21.15.99:0/1680990836 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078cc00700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880b3b90 0x7f078815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078e000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078815acb0 0x7f078809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 -- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880b3b90 msgr2=0x7f078815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880b3b90 0x7f078815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 -- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078815acb0 msgr2=0x7f078809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078815acb0 0x7f078809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 -- 172.21.15.99:0/1680990836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078815b9f0 con 0x7f0770004100 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078cc00700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880b3b90 0x7f078815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078e000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078815acb0 0x7f078809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.610+0000 7f078d600700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0788156a20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f077c008350 tx=0x7f077c008710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.611+0000 7f0785600700 1 -- 172.21.15.99:0/1680990836 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c008830 con 0x7f0770004100 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.611+0000 7f0785600700 1 -- 172.21.15.99:0/1680990836 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c00f860 con 0x7f0770004100 2024-03-29T09:10:25.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.611+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078815bcd0 con 0x7f0770004100 2024-03-29T09:10:25.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.611+0000 7f0785600700 1 -- 172.21.15.99:0/1680990836 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c00fbf0 con 0x7f0770004100 2024-03-29T09:10:25.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.611+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078809d440 con 0x7f0770004100 2024-03-29T09:10:25.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.612+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07880677a0 con 0x7f0770004100 2024-03-29T09:10:25.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.613+0000 7f0785600700 1 -- 172.21.15.99:0/1680990836 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f077c00fd50 con 0x7f0770004100 2024-03-29T09:10:25.614 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.613+0000 7f0785600700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f076806ac90 0x7f076806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:25.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.614+0000 7f0785600700 1 -- 172.21.15.99:0/1680990836 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f077c0891d0 con 0x7f0770004100 2024-03-29T09:10:25.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.614+0000 7f078cc00700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f076806ac90 0x7f076806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:25.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.614+0000 7f078cc00700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f076806ac90 0x7f076806d150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0788073f70 tx=0x7f0774008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:25.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.618+0000 7f0785600700 1 -- 172.21.15.99:0/1680990836 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f077c0540f0 con 0x7f0770004100 2024-03-29T09:10:25.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.834+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f07880d0700 con 0x7f0770004100 2024-03-29T09:10:25.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.834+0000 7f0785600700 1 -- 172.21.15.99:0/1680990836 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f077c057710 con 0x7f0770004100 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f076806ac90 msgr2=0x7f076806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f076806ac90 0x7f076806d150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0788073f70 tx=0x7f0774008030 comp rx=0 tx=0).stop 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 msgr2=0x7f0788156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0788156a20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f077c008350 tx=0x7f077c008710 comp rx=0 tx=0).stop 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 shutdown_connections 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f076806ac90 0x7f076806d150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078815acb0 0x7f078809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0770004100 0x7f0788156a20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 --2- 172.21.15.99:0/1680990836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07880b3b90 0x7f078815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:25.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 >> 172.21.15.99:0/1680990836 conn(0x7f078809c490 msgr2=0x7f07880d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:25.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 shutdown_connections 2024-03-29T09:10:25.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:25.837+0000 7f0790000700 1 -- 172.21.15.99:0/1680990836 wait complete. 2024-03-29T09:10:25.838 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:25.856 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2024-03-29T09:10:25.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:25.856 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-29T09:10:25.928 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:25.928 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:10:25.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:25.928 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T09:10:25.972 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T09:10:25.973 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:10:26.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.162+0000 7fed4e200700 1 Processor -- start 2024-03-29T09:10:26.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.162+0000 7fed4e200700 1 -- start start 2024-03-29T09:10:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed4e200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fed481608e0 0x7fed480d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed4e200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed480d1b20 0x7fed480ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed4e200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed480cddc0 con 0x7fed48160cc0 2024-03-29T09:10:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed4e200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed480ce0a0 con 0x7fed481608e0 2024-03-29T09:10:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed4e200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed480ce380 con 0x7fed480d1b20 2024-03-29T09:10:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed46a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fed48160cc0 0x7fed480d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52692/0 (socket says 172.21.15.99:52692) 2024-03-29T09:10:26.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed46a00700 1 -- 172.21.15.99:0/3979693941 learned_addr learned my addr 172.21.15.99:0/3979693941 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:26.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed47400700 1 --2- 172.21.15.99:0/3979693941 >> v2:172.21.15.133:3300/0 conn(0x7fed481608e0 0x7fed480d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed47e00700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed480d1b20 0x7fed480ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409136257 0 0) 0x7fed480cddc0 con 0x7fed48160cc0 2024-03-29T09:10:26.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.164+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed2c0035b0 con 0x7fed48160cc0 2024-03-29T09:10:26.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418468744 0 0) 0x7fed2c0035b0 con 0x7fed48160cc0 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed480d1b20 msgr2=0x7fed480ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed480d1b20 0x7fed480ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 >> v2:172.21.15.133:3300/0 conn(0x7fed481608e0 msgr2=0x7fed480d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 --2- 172.21.15.99:0/3979693941 >> v2:172.21.15.133:3300/0 conn(0x7fed481608e0 0x7fed480d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed480ce660 con 0x7fed48160cc0 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fed34001670 con 0x7fed48160cc0 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed47400700 1 --2- 172.21.15.99:0/3979693941 >> v2:172.21.15.133:3300/0 conn(0x7fed481608e0 0x7fed480d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 >> v1:172.21.15.99:6789/0 conn(0x7fed48160cc0 legacy=0x7fed480d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2c003a00 0x7fed2c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c0043f0 0x7fed2c0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480ce0a0 con 0x7fed2c004f40 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed480ce380 con 0x7fed2c0043f0 2024-03-29T09:10:26.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c00bf50 con 0x7fed2c003a00 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed47e00700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed480d1b20 0x7fed480ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed47e00700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed47400700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2c003a00 0x7fed2c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.165+0000 7fed46a00700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c0043f0 0x7fed2c0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed47e00700 1 -- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2c003a00 msgr2=0x7fed2c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed47e00700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed2c003a00 0x7fed2c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed47e00700 1 -- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c0043f0 msgr2=0x7fed2c0048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed47e00700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed2c0043f0 0x7fed2c0048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed47e00700 1 -- 172.21.15.99:0/3979693941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0035b0 con 0x7fed2c004f40 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed47e00700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed2c00ba10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed4000e1f0 tx=0x7fed2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5155ca14e9368319 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4000ec90 con 0x7fed2c004f40 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4000f040 con 0x7fed2c004f40 2024-03-29T09:10:26.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed46000700 1 -- 172.21.15.99:0/3979693941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40003a20 con 0x7fed2c004f40 2024-03-29T09:10:26.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed4e200700 1 -- 172.21.15.99:0/3979693941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 msgr2=0x7fed2c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.166+0000 7fed4e200700 1 --2- 172.21.15.99:0/3979693941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed2c00ba10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fed4000e1f0 tx=0x7fed2c005880 comp rx=0 tx=0).stop 2024-03-29T09:10:26.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.167+0000 7fed47e00700 1 -- 172.21.15.99:0/3979693941 reap_dead start 2024-03-29T09:10:26.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.167+0000 7fed4e200700 1 -- 172.21.15.99:0/3979693941 shutdown_connections 2024-03-29T09:10:26.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.167+0000 7fed4e200700 1 -- 172.21.15.99:0/3979693941 >> 172.21.15.99:0/3979693941 conn(0x7fed4809c490 msgr2=0x7fed480d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:26.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.167+0000 7fed4e200700 1 -- 172.21.15.99:0/3979693941 shutdown_connections 2024-03-29T09:10:26.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.167+0000 7fed4e200700 1 -- 172.21.15.99:0/3979693941 wait complete. 2024-03-29T09:10:26.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.168+0000 7fed4e200700 1 Processor -- start 2024-03-29T09:10:26.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.169+0000 7fed4e200700 1 -- start start 2024-03-29T09:10:26.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.169+0000 7fed4e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed480a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.169+0000 7fed4e200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed481544d0 0x7fed4809e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.169+0000 7fed4e200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed48154930 0x7fed48152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.169+0000 7fed4e200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4809d300 con 0x7fed2c004f40 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.169+0000 7fed4e200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4809d5e0 con 0x7fed48154930 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.169+0000 7fed4e200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4809d8c0 con 0x7fed481544d0 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed480a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed480a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35682/0 (socket says 172.21.15.99:35682) 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 -- 172.21.15.99:0/2657917536 learned_addr learned my addr 172.21.15.99:0/2657917536 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed46a00700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed481544d0 0x7fed4809e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47e00700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed48154930 0x7fed48152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 -- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed481544d0 msgr2=0x7fed4809e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed481544d0 0x7fed4809e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 -- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed48154930 msgr2=0x7fed48152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed48154930 0x7fed48152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 -- 172.21.15.99:0/2657917536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4809dba0 con 0x7fed2c004f40 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47e00700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed48154930 0x7fed48152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.170+0000 7fed47400700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed480a06a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed38011cd0 tx=0x7fed38009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.171+0000 7fed3f400700 1 -- 172.21.15.99:0/2657917536 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3800fa60 con 0x7fed2c004f40 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.171+0000 7fed3f400700 1 -- 172.21.15.99:0/2657917536 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3800fbc0 con 0x7fed2c004f40 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.171+0000 7fed3f400700 1 -- 172.21.15.99:0/2657917536 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38004030 con 0x7fed2c004f40 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.171+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4809de80 con 0x7fed2c004f40 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.171+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed48154f00 con 0x7fed2c004f40 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.172+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4809e070 con 0x7fed2c004f40 2024-03-29T09:10:26.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.173+0000 7fed3f400700 1 -- 172.21.15.99:0/2657917536 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fed38004190 con 0x7fed2c004f40 2024-03-29T09:10:26.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.173+0000 7fed3f400700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fed2806acc0 0x7fed2806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.174+0000 7fed3f400700 1 -- 172.21.15.99:0/2657917536 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fed38089c90 con 0x7fed2c004f40 2024-03-29T09:10:26.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.174+0000 7fed46a00700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fed2806acc0 0x7fed2806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.174+0000 7fed46a00700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fed2806acc0 0x7fed2806d180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed4809fc20 tx=0x7fed34007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:26.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.178+0000 7fed3f400700 1 -- 172.21.15.99:0/2657917536 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fed38054a80 con 0x7fed2c004f40 2024-03-29T09:10:26.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.428+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed480677a0 con 0x7fed2c004f40 2024-03-29T09:10:26.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.428+0000 7fed3f400700 1 -- 172.21.15.99:0/2657917536 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7fed38022070 con 0x7fed2c004f40 2024-03-29T09:10:26.429 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:26.429 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:10:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fed2806acc0 msgr2=0x7fed2806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fed2806acc0 0x7fed2806d180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fed4809fc20 tx=0x7fed34007980 comp rx=0 tx=0).stop 2024-03-29T09:10:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 msgr2=0x7fed480a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.431 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed480a06a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fed38011cd0 tx=0x7fed38009ed0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 shutdown_connections 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fed2806acc0 0x7fed2806d180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed48154930 0x7fed48152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fed2c004f40 0x7fed480a06a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 --2- 172.21.15.99:0/2657917536 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed481544d0 0x7fed4809e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 >> 172.21.15.99:0/2657917536 conn(0x7fed4809c490 msgr2=0x7fed480b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 shutdown_connections 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.431+0000 7fed4e200700 1 -- 172.21.15.99:0/2657917536 wait complete. 2024-03-29T09:10:26.432 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:10:26.449 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:26.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.640+0000 7f4e69000700 1 Processor -- start 2024-03-29T09:10:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.641+0000 7f4e69000700 1 -- start start 2024-03-29T09:10:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e69000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4e6415d690 0x7f4e640cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e69000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e640afde0 con 0x7f4e6415da70 2024-03-29T09:10:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e69000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e640d01d0 con 0x7f4e640d1740 2024-03-29T09:10:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e69000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e640d04b0 con 0x7f4e6415d690 2024-03-29T09:10:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e61400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4e6415da70 0x7f4e640cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:52698/0 (socket says 172.21.15.99:52698) 2024-03-29T09:10:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e61400700 1 -- 172.21.15.99:0/1529852005 learned_addr learned my addr 172.21.15.99:0/1529852005 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e61e00700 1 --2- 172.21.15.99:0/1529852005 >> v2:172.21.15.133:3300/0 conn(0x7f4e6415d690 0x7f4e640cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962104513 0 0) 0x7f4e640afde0 con 0x7f4e6415da70 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.642+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e480035b0 con 0x7f4e6415da70 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3034225908 0 0) 0x7f4e480035b0 con 0x7f4e6415da70 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 >> v2:172.21.15.133:3300/0 conn(0x7f4e6415d690 msgr2=0x7f4e640cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 --2- 172.21.15.99:0/1529852005 >> v2:172.21.15.133:3300/0 conn(0x7f4e6415d690 0x7f4e640cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 >> v1:172.21.15.133:6789/0 conn(0x7f4e640d1740 legacy=0x7f4e640b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640b0030 con 0x7f4e6415da70 2024-03-29T09:10:26.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f4e50001670 con 0x7f4e6415da70 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 >> v1:172.21.15.99:6789/0 conn(0x7f4e6415da70 legacy=0x7f4e640cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e48003a00 0x7f4e48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e48004850 0x7f4e48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48005330 0x7f4e4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640d04b0 con 0x7f4e48003a00 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4800c200 con 0x7f4e48005330 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4800c3c0 con 0x7f4e48004850 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e61e00700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e48003a00 0x7f4e48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e62800700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48005330 0x7f4e4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e61400700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e48004850 0x7f4e48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e61e00700 1 -- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e48004850 msgr2=0x7f4e48004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e61e00700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e48004850 0x7f4e48004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e61e00700 1 -- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48005330 msgr2=0x7f4e4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e61e00700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48005330 0x7f4e4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e61e00700 1 -- 172.21.15.99:0/1529852005 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480035b0 con 0x7f4e48003a00 2024-03-29T09:10:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e62800700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48005330 0x7f4e4800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.643+0000 7f4e62800700 1 -- 172.21.15.99:0/1529852005 reap_dead start 2024-03-29T09:10:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e61e00700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e48003a00 0x7f4e48003e20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009b30 tx=0x7f4e48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9d5a20149b0eed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400e6c0 con 0x7f4e48003a00 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5400e820 con 0x7f4e48003a00 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e60a00700 1 -- 172.21.15.99:0/1529852005 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400eb90 con 0x7f4e48003a00 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e69000700 1 -- 172.21.15.99:0/1529852005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e48003a00 msgr2=0x7f4e48003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e69000700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e48003a00 0x7f4e48003e20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009b30 tx=0x7f4e48005880 comp rx=0 tx=0).stop 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e69000700 1 -- 172.21.15.99:0/1529852005 shutdown_connections 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e69000700 1 --2- 172.21.15.99:0/1529852005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e48003a00 0x7f4e48003e20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.644+0000 7f4e69000700 1 -- 172.21.15.99:0/1529852005 >> 172.21.15.99:0/1529852005 conn(0x7f4e6409c480 msgr2=0x7f4e640d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.645+0000 7f4e69000700 1 -- 172.21.15.99:0/1529852005 shutdown_connections 2024-03-29T09:10:26.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.645+0000 7f4e69000700 1 -- 172.21.15.99:0/1529852005 wait complete. 2024-03-29T09:10:26.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.646+0000 7f4e69000700 1 Processor -- start 2024-03-29T09:10:26.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.646+0000 7f4e69000700 1 -- start start 2024-03-29T09:10:26.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e69000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48003a00 0x7f4e6415fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e69000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e640cd020 0x7f4e641602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e69000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e641607e0 0x7f4e6415e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e69000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64160c90 con 0x7f4e640cd020 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e69000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640cd4a0 con 0x7f4e48003a00 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e69000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e640b68b0 con 0x7f4e641607e0 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e640cd020 0x7f4e641602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e640cd020 0x7f4e641602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35690/0 (socket says 172.21.15.99:35690) 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 -- 172.21.15.99:0/699807801 learned_addr learned my addr 172.21.15.99:0/699807801 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e62800700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e641607e0 0x7f4e6415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61e00700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48003a00 0x7f4e6415fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 -- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e641607e0 msgr2=0x7f4e6415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e641607e0 0x7f4e6415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 -- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48003a00 msgr2=0x7f4e6415fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48003a00 0x7f4e6415fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.648+0000 7f4e61400700 1 -- 172.21.15.99:0/699807801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e640b6b90 con 0x7f4e640cd020 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.649+0000 7f4e61400700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e640cd020 0x7f4e641602a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000e7b0 tx=0x7f4e5000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.649+0000 7f4e5a000700 1 -- 172.21.15.99:0/699807801 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500105d0 con 0x7f4e640cd020 2024-03-29T09:10:26.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.649+0000 7f4e5a000700 1 -- 172.21.15.99:0/699807801 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e50010730 con 0x7f4e640cd020 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.649+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e640b6e70 con 0x7f4e640cd020 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.649+0000 7f4e5a000700 1 -- 172.21.15.99:0/699807801 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50010ae0 con 0x7f4e640cd020 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.649+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e640b7430 con 0x7f4e640cd020 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.650+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e640b7650 con 0x7f4e640cd020 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.651+0000 7f4e5a000700 1 -- 172.21.15.99:0/699807801 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4e50010c40 con 0x7f4e640cd020 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.652+0000 7f4e5a000700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e4406ac90 0x7f4e4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.652+0000 7f4e5a000700 1 -- 172.21.15.99:0/699807801 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4e50088e60 con 0x7f4e640cd020 2024-03-29T09:10:26.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.652+0000 7f4e61e00700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e4406ac90 0x7f4e4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:26.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.653+0000 7f4e61e00700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e4406ac90 0x7f4e4406d150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e540090b0 tx=0x7f4e54008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:26.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.656+0000 7f4e5a000700 1 -- 172.21.15.99:0/699807801 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4e50053d80 con 0x7f4e640cd020 2024-03-29T09:10:26.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.860+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e640677a0 con 0x7f4e640cd020 2024-03-29T09:10:26.861 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.861+0000 7f4e5a000700 1 -- 172.21.15.99:0/699807801 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f4e50019070 con 0x7f4e640cd020 2024-03-29T09:10:26.861 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:26.862 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:26.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.863+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e4406ac90 msgr2=0x7f4e4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.863+0000 7f4e69000700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e4406ac90 0x7f4e4406d150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4e540090b0 tx=0x7f4e54008030 comp rx=0 tx=0).stop 2024-03-29T09:10:26.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.863+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e640cd020 msgr2=0x7f4e641602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:26.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.863+0000 7f4e69000700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e640cd020 0x7f4e641602a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000e7b0 tx=0x7f4e5000eb70 comp rx=0 tx=0).stop 2024-03-29T09:10:26.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.863+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 shutdown_connections 2024-03-29T09:10:26.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.864+0000 7f4e69000700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4e4406ac90 0x7f4e4406d150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.864+0000 7f4e69000700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e641607e0 0x7f4e6415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.864+0000 7f4e69000700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e48003a00 0x7f4e6415fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.864+0000 7f4e69000700 1 --2- 172.21.15.99:0/699807801 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e640cd020 0x7f4e641602a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:26.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.864+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 >> 172.21.15.99:0/699807801 conn(0x7f4e6409c480 msgr2=0x7f4e6406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:26.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.864+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 shutdown_connections 2024-03-29T09:10:26.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:26.864+0000 7f4e69000700 1 -- 172.21.15.99:0/699807801 wait complete. 2024-03-29T09:10:26.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:08:55.815767+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:08:56.358550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-29T09:08:59.440030+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-29T09:10:26.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.074+0000 7f13d7e00700 1 Processor -- start 2024-03-29T09:10:27.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.074+0000 7f13d7e00700 1 -- start start 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.075+0000 7f13d7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f13d0160770 0x7f13d00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.075+0000 7f13d7e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d00d1520 con 0x7f13d00d0d60 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.075+0000 7f13d7e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d00d1800 con 0x7f13d00d3550 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.075+0000 7f13d7e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d00d1ae0 con 0x7f13d0160770 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.075+0000 7f13d5400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f13d0160770 0x7f13d00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.075+0000 7f13d5400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f13d0160770 0x7f13d00b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35704/0 (socket says 172.21.15.99:35704) 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.075+0000 7f13d5400700 1 -- 172.21.15.99:0/2782677928 learned_addr learned my addr 172.21.15.99:0/2782677928 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13d5400700 1 -- 172.21.15.99:0/2782677928 >> v1:172.21.15.133:6789/0 conn(0x7f13d00d3550 legacy=0x7f13d00ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13d5400700 1 -- 172.21.15.99:0/2782677928 >> v1:172.21.15.99:6789/0 conn(0x7f13d00d0d60 legacy=0x7f13d00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13d5400700 1 -- 172.21.15.99:0/2782677928 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d00d1dc0 con 0x7f13d0160770 2024-03-29T09:10:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13cfe00700 1 -- 172.21.15.99:0/2782677928 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509509717 0 0) 0x7f13d00d1520 con 0x7f13d00d0d60 2024-03-29T09:10:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13d5400700 1 --2- 172.21.15.99:0/2782677928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00b3cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f13c40098f0 tx=0x7f13c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97a2dc7e2c20361 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13cfe00700 1 -- 172.21.15.99:0/2782677928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c4004230 con 0x7f13d0160770 2024-03-29T09:10:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13cfe00700 1 -- 172.21.15.99:0/2782677928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c4004390 con 0x7f13d0160770 2024-03-29T09:10:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13cfe00700 1 -- 172.21.15.99:0/2782677928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c40036a0 con 0x7f13d0160770 2024-03-29T09:10:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13d7e00700 1 -- 172.21.15.99:0/2782677928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 msgr2=0x7f13d00b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.076+0000 7f13d7e00700 1 --2- 172.21.15.99:0/2782677928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00b3cc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f13c40098f0 tx=0x7f13c4009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.077+0000 7f13d7e00700 1 -- 172.21.15.99:0/2782677928 shutdown_connections 2024-03-29T09:10:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.077+0000 7f13d7e00700 1 --2- 172.21.15.99:0/2782677928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00b3cc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.077+0000 7f13d7e00700 1 -- 172.21.15.99:0/2782677928 >> 172.21.15.99:0/2782677928 conn(0x7f13d008afd0 msgr2=0x7f13d009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.077+0000 7f13d7e00700 1 -- 172.21.15.99:0/2782677928 shutdown_connections 2024-03-29T09:10:27.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.077+0000 7f13d7e00700 1 -- 172.21.15.99:0/2782677928 wait complete. 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.078+0000 7f13d7e00700 1 Processor -- start 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.078+0000 7f13d7e00700 1 -- start start 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.079+0000 7f13d7e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13d00d0d60 0x7f13d00cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.079+0000 7f13d7e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13d00d3550 0x7f13d00ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.079+0000 7f13d7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.079+0000 7f13d7e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d009d270 con 0x7f13d0160770 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.079+0000 7f13d7e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d00cad40 con 0x7f13d00d3550 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.079+0000 7f13d7e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d00caf70 con 0x7f13d00d0d60 2024-03-29T09:10:27.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13d00d0d60 0x7f13d00cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35716/0 (socket says 172.21.15.99:35716) 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 -- 172.21.15.99:0/85446087 learned_addr learned my addr 172.21.15.99:0/85446087 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d4a00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13d00d3550 0x7f13d00ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 -- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13d00d0d60 msgr2=0x7f13d00cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13d00d0d60 0x7f13d00cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 -- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13d00d3550 msgr2=0x7f13d00ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13d00d3550 0x7f13d00ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 -- 172.21.15.99:0/85446087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c4009550 con 0x7f13d0160770 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d4a00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13d00d3550 0x7f13d00ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.080+0000 7f13d5e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00afdc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f13c800cba0 tx=0x7f13c800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.081+0000 7f13cd600700 1 -- 172.21.15.99:0/85446087 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c800aad0 con 0x7f13d0160770 2024-03-29T09:10:27.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.081+0000 7f13cd600700 1 -- 172.21.15.99:0/85446087 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c800ac30 con 0x7f13d0160770 2024-03-29T09:10:27.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.081+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d00cb250 con 0x7f13d0160770 2024-03-29T09:10:27.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.081+0000 7f13cd600700 1 -- 172.21.15.99:0/85446087 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8012420 con 0x7f13d0160770 2024-03-29T09:10:27.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.081+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d00cb5e0 con 0x7f13d0160770 2024-03-29T09:10:27.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.082+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d00678b0 con 0x7f13d0160770 2024-03-29T09:10:27.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.083+0000 7f13cd600700 1 -- 172.21.15.99:0/85446087 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f13c8012580 con 0x7f13d0160770 2024-03-29T09:10:27.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.084+0000 7f13cd600700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f13b006ac90 0x7f13b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.084+0000 7f13cd600700 1 -- 172.21.15.99:0/85446087 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f13c8089d30 con 0x7f13d0160770 2024-03-29T09:10:27.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.084+0000 7f13d5400700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f13b006ac90 0x7f13b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.085+0000 7f13d5400700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f13b006ac90 0x7f13b006d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009130 tx=0x7f13c4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.089+0000 7f13cd600700 1 -- 172.21.15.99:0/85446087 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f13c8054bb0 con 0x7f13d0160770 2024-03-29T09:10:27.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.291+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13d00d08a0 con 0x7f13d0160770 2024-03-29T09:10:27.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.291+0000 7f13cd600700 1 -- 172.21.15.99:0/85446087 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13c8016070 con 0x7f13d0160770 2024-03-29T09:10:27.291 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.293+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f13b006ac90 msgr2=0x7f13b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.293+0000 7f13d7e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f13b006ac90 0x7f13b006d150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009130 tx=0x7f13c4008030 comp rx=0 tx=0).stop 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.293+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 msgr2=0x7f13d00afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.293+0000 7f13d7e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00afdc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f13c800cba0 tx=0x7f13c800ceb0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 shutdown_connections 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f13b006ac90 0x7f13b006d150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f13d0160770 0x7f13d00afdc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f13d00d0d60 0x7f13d00cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.294 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 --2- 172.21.15.99:0/85446087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f13d00d3550 0x7f13d00ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 >> 172.21.15.99:0/85446087 conn(0x7f13d008afd0 msgr2=0x7f13d015f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:27.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 shutdown_connections 2024-03-29T09:10:27.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.294+0000 7f13d7e00700 1 -- 172.21.15.99:0/85446087 wait complete. 2024-03-29T09:10:27.312 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.504+0000 7f7fe2e00700 1 Processor -- start 2024-03-29T09:10:27.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.504+0000 7f7fe2e00700 1 -- start start 2024-03-29T09:10:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.505+0000 7f7fe2e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.505+0000 7f7fe2e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7fdc0d3550 0x7f7fdc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.505+0000 7f7fe2e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fdc0d1520 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.505+0000 7f7fe2e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fdc0d1800 con 0x7f7fdc0d3550 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.505+0000 7f7fe2e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0d1ae0 con 0x7f7fdc160770 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7fdc0d3550 0x7f7fdc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7fdc0d3550 0x7f7fdc0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35726/0 (socket says 172.21.15.99:35726) 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 -- 172.21.15.99:0/275920273 learned_addr learned my addr 172.21.15.99:0/275920273 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fdbe00700 1 --2- 172.21.15.99:0/275920273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 -- 172.21.15.99:0/275920273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 msgr2=0x7f7fdc0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 --2- 172.21.15.99:0/275920273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 -- 172.21.15.99:0/275920273 >> v1:172.21.15.133:6789/0 conn(0x7f7fdc0d0d60 legacy=0x7f7fdc0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 -- 172.21.15.99:0/275920273 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fdc0d1dc0 con 0x7f7fdc0d3550 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fe0e00700 1 --2- 172.21.15.99:0/275920273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d3550 0x7f7fdc0ce080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009e50 tx=0x7f7fd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2e16105aa0d86 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fdb400700 1 -- 172.21.15.99:0/275920273 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400e070 con 0x7f7fdc0d3550 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fdb400700 1 -- 172.21.15.99:0/275920273 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd400b9c0 con 0x7f7fdc0d3550 2024-03-29T09:10:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.506+0000 7f7fdb400700 1 -- 172.21.15.99:0/275920273 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400bd70 con 0x7f7fdc0d3550 2024-03-29T09:10:27.508 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 -- 172.21.15.99:0/275920273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d3550 msgr2=0x7f7fdc0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/275920273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d3550 0x7f7fdc0ce080 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009e50 tx=0x7f7fd400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 -- 172.21.15.99:0/275920273 shutdown_connections 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/275920273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d3550 0x7f7fdc0ce080 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/275920273 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 -- 172.21.15.99:0/275920273 >> 172.21.15.99:0/275920273 conn(0x7f7fdc08afd0 msgr2=0x7f7fdc09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 -- 172.21.15.99:0/275920273 shutdown_connections 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.507+0000 7f7fe2e00700 1 -- 172.21.15.99:0/275920273 wait complete. 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.509+0000 7f7fe2e00700 1 Processor -- start 2024-03-29T09:10:27.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.509+0000 7f7fe2e00700 1 -- start start 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fe2e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d0d60 0x7f7fdc1f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fe2e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fdc0d3550 0x7f7fdc0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fe2e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc14e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fe2e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc14c1d0 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fe2e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0cb710 con 0x7f7fdc0d3550 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fe2e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fdc0cb9f0 con 0x7f7fdc160770 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fdbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d0d60 0x7f7fdc1f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fdbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d0d60 0x7f7fdc1f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35730/0 (socket says 172.21.15.99:35730) 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fdbe00700 1 -- 172.21.15.99:0/53045440 learned_addr learned my addr 172.21.15.99:0/53045440 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fe0e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc14e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.510+0000 7f7fd3e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fdc0d3550 0x7f7fdc0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fdbe00700 1 -- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 msgr2=0x7f7fdc14e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fdbe00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc14e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fdbe00700 1 -- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fdc0d3550 msgr2=0x7f7fdc0cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fdbe00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fdc0d3550 0x7f7fdc0cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fdbe00700 1 -- 172.21.15.99:0/53045440 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4009b30 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fdbe00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d0d60 0x7f7fdc1f1fd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00d800 tx=0x7f7fcc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fd8c00700 1 -- 172.21.15.99:0/53045440 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0125f0 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fd8c00700 1 -- 172.21.15.99:0/53045440 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc012750 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc14c3e0 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fd8c00700 1 -- 172.21.15.99:0/53045440 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc012b00 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.511+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fdc14f000 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.512+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fdc0cbc10 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.513+0000 7f7fd8c00700 1 -- 172.21.15.99:0/53045440 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7fcc012c60 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.514+0000 7f7fd8c00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fb806ac90 0x7f7fb806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.514+0000 7f7fd8c00700 1 -- 172.21.15.99:0/53045440 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7fcc089ab0 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.514+0000 7f7fd3e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fb806ac90 0x7f7fb806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.515+0000 7f7fd3e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fb806ac90 0x7f7fb806d150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0b14f0 tx=0x7f7fc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.519+0000 7f7fd8c00700 1 -- 172.21.15.99:0/53045440 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7fcc0548b0 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.719+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7fdc0d20a0 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.719+0000 7f7fd8c00700 1 -- 172.21.15.99:0/53045440 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7fcc057ed0 con 0x7f7fdc0d0d60 2024-03-29T09:10:27.719 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fb806ac90 msgr2=0x7f7fb806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fb806ac90 0x7f7fb806d150 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0b14f0 tx=0x7f7fc4009040 comp rx=0 tx=0).stop 2024-03-29T09:10:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d0d60 msgr2=0x7f7fdc1f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d0d60 0x7f7fdc1f1fd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00d800 tx=0x7f7fcc00dbc0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 shutdown_connections 2024-03-29T09:10:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fb806ac90 0x7f7fb806d150 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fdc160770 0x7f7fdc14e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fdc0d0d60 0x7f7fdc1f1fd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 --2- 172.21.15.99:0/53045440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fdc0d3550 0x7f7fdc0cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 >> 172.21.15.99:0/53045440 conn(0x7f7fdc08afd0 msgr2=0x7f7fdc06cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:27.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 shutdown_connections 2024-03-29T09:10:27.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.722+0000 7f7fe2e00700 1 -- 172.21.15.99:0/53045440 wait complete. 2024-03-29T09:10:27.740 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.933+0000 7ff12b800700 1 Processor -- start 2024-03-29T09:10:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.933+0000 7ff12b800700 1 -- start start 2024-03-29T09:10:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff12b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240cdcb0 0x7ff1240d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff12b800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1240d12b0 0x7ff1240cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff12b800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff124161ae0 con 0x7ff1240d1760 2024-03-29T09:10:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff12b800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff124160f10 con 0x7ff1240d12b0 2024-03-29T09:10:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff12b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1241611b0 con 0x7ff1240cdcb0 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff129800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1240d12b0 0x7ff1240cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff123e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240cdcb0 0x7ff1240d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff129800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1240d12b0 0x7ff1240cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:38878/0 (socket says 172.21.15.99:38878) 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff128e00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff1240d1760 0x7ff1240d1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:50280/0 (socket says 172.21.15.99:50280) 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.934+0000 7ff129800700 1 -- 172.21.15.99:0/2245064475 learned_addr learned my addr 172.21.15.99:0/2245064475 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff123e00700 1 -- 172.21.15.99:0/2245064475 >> v2:172.21.15.133:3300/0 conn(0x7ff1240d12b0 msgr2=0x7ff1240cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff123e00700 1 --2- 172.21.15.99:0/2245064475 >> v2:172.21.15.133:3300/0 conn(0x7ff1240d12b0 0x7ff1240cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff123e00700 1 -- 172.21.15.99:0/2245064475 >> v1:172.21.15.133:6789/0 conn(0x7ff1240d1760 legacy=0x7ff1240d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff123e00700 1 -- 172.21.15.99:0/2245064475 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124161490 con 0x7ff1240cdcb0 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff129800700 1 --2- 172.21.15.99:0/2245064475 >> v2:172.21.15.133:3300/0 conn(0x7ff1240d12b0 0x7ff1240cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff123e00700 1 --2- 172.21.15.99:0/2245064475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240cdcb0 0x7ff1240d0d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff110009840 tx=0x7ff110009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5584d16baf902b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff123400700 1 -- 172.21.15.99:0/2245064475 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110011830 con 0x7ff1240cdcb0 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.935+0000 7ff123400700 1 -- 172.21.15.99:0/2245064475 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110011990 con 0x7ff1240cdcb0 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff123400700 1 -- 172.21.15.99:0/2245064475 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110011d20 con 0x7ff1240cdcb0 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 -- 172.21.15.99:0/2245064475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240cdcb0 msgr2=0x7ff1240d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 --2- 172.21.15.99:0/2245064475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240cdcb0 0x7ff1240d0d70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff110009840 tx=0x7ff110009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 -- 172.21.15.99:0/2245064475 shutdown_connections 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 --2- 172.21.15.99:0/2245064475 >> v2:172.21.15.133:3300/0 conn(0x7ff1240d12b0 0x7ff1240cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 --2- 172.21.15.99:0/2245064475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240cdcb0 0x7ff1240d0d70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 -- 172.21.15.99:0/2245064475 >> 172.21.15.99:0/2245064475 conn(0x7ff12408afd0 msgr2=0x7ff12409c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 -- 172.21.15.99:0/2245064475 shutdown_connections 2024-03-29T09:10:27.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.936+0000 7ff12b800700 1 -- 172.21.15.99:0/2245064475 wait complete. 2024-03-29T09:10:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.938+0000 7ff12b800700 1 Processor -- start 2024-03-29T09:10:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.938+0000 7ff12b800700 1 -- start start 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff12b800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1240cdcb0 0x7ff12415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff12b800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1240d12b0 0x7ff1240cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff12b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240d1760 0x7ff1240cf790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff12b800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12415de40 con 0x7ff1240cdcb0 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff12b800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12415f7b0 con 0x7ff1240d12b0 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff12b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12415fa90 con 0x7ff1240d1760 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff128e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1240cdcb0 0x7ff12415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff128e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1240cdcb0 0x7ff12415d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39832/0 (socket says 172.21.15.99:39832) 2024-03-29T09:10:27.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff128e00700 1 -- 172.21.15.99:0/4157862381 learned_addr learned my addr 172.21.15.99:0/4157862381 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff123e00700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1240d12b0 0x7ff1240cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.939+0000 7ff129800700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240d1760 0x7ff1240cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff128e00700 1 -- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240d1760 msgr2=0x7ff1240cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff128e00700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240d1760 0x7ff1240cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff128e00700 1 -- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1240d12b0 msgr2=0x7ff1240cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff128e00700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1240d12b0 0x7ff1240cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff128e00700 1 -- 172.21.15.99:0/4157862381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c009b30 con 0x7ff1240cdcb0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff128e00700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1240cdcb0 0x7ff12415d900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff11800c520 tx=0x7ff11800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff120c00700 1 -- 172.21.15.99:0/4157862381 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118005cb0 con 0x7ff1240cdcb0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff120c00700 1 -- 172.21.15.99:0/4157862381 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118005e10 con 0x7ff1240cdcb0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1100094a0 con 0x7ff1240cdcb0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff120c00700 1 -- 172.21.15.99:0/4157862381 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180072d0 con 0x7ff1240cdcb0 2024-03-29T09:10:27.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.940+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff124160050 con 0x7ff1240cdcb0 2024-03-29T09:10:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.941+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12415f8f0 con 0x7ff1240cdcb0 2024-03-29T09:10:27.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.942+0000 7ff120c00700 1 -- 172.21.15.99:0/4157862381 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff118007430 con 0x7ff1240cdcb0 2024-03-29T09:10:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.943+0000 7ff120c00700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff10406ac90 0x7ff10406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.943+0000 7ff120c00700 1 -- 172.21.15.99:0/4157862381 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff1180888f0 con 0x7ff1240cdcb0 2024-03-29T09:10:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.943+0000 7ff123e00700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff10406ac90 0x7ff10406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:27.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.944+0000 7ff123e00700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff10406ac90 0x7ff10406d150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff110000c00 tx=0x7ff11001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:27.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:27.948+0000 7ff120c00700 1 -- 172.21.15.99:0/4157862381 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff12415f8f0 con 0x7ff1240cdcb0 2024-03-29T09:10:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.151+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff12415f8f0 con 0x7ff1240cdcb0 2024-03-29T09:10:28.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.152+0000 7ff120c00700 1 -- 172.21.15.99:0/4157862381 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff12415f8f0 con 0x7ff1240cdcb0 2024-03-29T09:10:28.152 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff10406ac90 msgr2=0x7ff10406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff10406ac90 0x7ff10406d150 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff110000c00 tx=0x7ff11001c040 comp rx=0 tx=0).stop 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1240cdcb0 msgr2=0x7ff12415d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1240cdcb0 0x7ff12415d900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff11800c520 tx=0x7ff11800c830 comp rx=0 tx=0).stop 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 shutdown_connections 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff10406ac90 0x7ff10406d150 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1240d1760 0x7ff1240cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff1240cdcb0 0x7ff12415d900 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 --2- 172.21.15.99:0/4157862381 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1240d12b0 0x7ff1240cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 >> 172.21.15.99:0/4157862381 conn(0x7ff12408afd0 msgr2=0x7ff1240b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 shutdown_connections 2024-03-29T09:10:28.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.154+0000 7ff12b800700 1 -- 172.21.15.99:0/4157862381 wait complete. 2024-03-29T09:10:28.174 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-29T09:10:28.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.365+0000 7fb301200700 1 Processor -- start 2024-03-29T09:10:28.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.366+0000 7fb301200700 1 -- start start 2024-03-29T09:10:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.366+0000 7fb301200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2fc160770 0x7fb2fc0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb301200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb301200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc0d1520 con 0x7fb2fc0d0d60 2024-03-29T09:10:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb301200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0d1800 con 0x7fb2fc160770 2024-03-29T09:10:28.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb301200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0d1ae0 con 0x7fb2fc0d3550 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2f9e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2fc160770 0x7fb2fc0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2f9e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2fc160770 0x7fb2fc0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39842/0 (socket says 172.21.15.99:39842) 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2f9e00700 1 -- 172.21.15.99:0/1172327774 learned_addr learned my addr 172.21.15.99:0/1172327774 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2fa800700 1 --2- 172.21.15.99:0/1172327774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2f9e00700 1 -- 172.21.15.99:0/1172327774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 msgr2=0x7fb2fc0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2f9e00700 1 --2- 172.21.15.99:0/1172327774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2f9e00700 1 -- 172.21.15.99:0/1172327774 >> v1:172.21.15.133:6789/0 conn(0x7fb2fc0d0d60 legacy=0x7fb2fc0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.367+0000 7fb2f9e00700 1 -- 172.21.15.99:0/1172327774 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc0d1dc0 con 0x7fb2fc160770 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb2f9e00700 1 --2- 172.21.15.99:0/1172327774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0b3cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0098f0 tx=0x7fb2ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6882f445395905 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb2f8a00700 1 -- 172.21.15.99:0/1172327774 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec004230 con 0x7fb2fc160770 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb2f8a00700 1 -- 172.21.15.99:0/1172327774 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec004390 con 0x7fb2fc160770 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb2f8a00700 1 -- 172.21.15.99:0/1172327774 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0036a0 con 0x7fb2fc160770 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 -- 172.21.15.99:0/1172327774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 msgr2=0x7fb2fc0b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 --2- 172.21.15.99:0/1172327774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0b3cc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0098f0 tx=0x7fb2ec009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 -- 172.21.15.99:0/1172327774 shutdown_connections 2024-03-29T09:10:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 --2- 172.21.15.99:0/1172327774 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 --2- 172.21.15.99:0/1172327774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0b3cc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 -- 172.21.15.99:0/1172327774 >> 172.21.15.99:0/1172327774 conn(0x7fb2fc08afd0 msgr2=0x7fb2fc09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 -- 172.21.15.99:0/1172327774 shutdown_connections 2024-03-29T09:10:28.370 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.368+0000 7fb301200700 1 -- 172.21.15.99:0/1172327774 wait complete. 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.370+0000 7fb301200700 1 Processor -- start 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.370+0000 7fb301200700 1 -- start start 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb301200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0d0d60 0x7fb2fc1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb301200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb301200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb301200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc15b420 con 0x7fb2fc160770 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb301200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc15b0f0 con 0x7fb2fc0d0d60 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb301200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0b7040 con 0x7fb2fc0d3550 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb2fa800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb2fa800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39850/0 (socket says 172.21.15.99:39850) 2024-03-29T09:10:28.372 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.371+0000 7fb2fa800700 1 -- 172.21.15.99:0/171363949 learned_addr learned my addr 172.21.15.99:0/171363949 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2f9e00700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0d0d60 0x7fb2fc1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2f9400700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2fa800700 1 -- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 msgr2=0x7fb2fc0b5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2fa800700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0b5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2fa800700 1 -- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0d0d60 msgr2=0x7fb2fc1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2fa800700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0d0d60 0x7fb2fc1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2fa800700 1 -- 172.21.15.99:0/171363949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec009550 con 0x7fb2fc160770 2024-03-29T09:10:28.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2fa800700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0cfd90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4007e30 tx=0x7fb2f400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2f2000700 1 -- 172.21.15.99:0/171363949 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400ed30 con 0x7fb2fc160770 2024-03-29T09:10:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2f2000700 1 -- 172.21.15.99:0/171363949 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f400ee90 con 0x7fb2fc160770 2024-03-29T09:10:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb2f2000700 1 -- 172.21.15.99:0/171363949 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4014660 con 0x7fb2fc160770 2024-03-29T09:10:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.372+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc0b67a0 con 0x7fb2fc160770 2024-03-29T09:10:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.373+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc0b6d60 con 0x7fb2fc160770 2024-03-29T09:10:28.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.374+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc15b560 con 0x7fb2fc160770 2024-03-29T09:10:28.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.374+0000 7fb2f2000700 1 -- 172.21.15.99:0/171363949 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb2f40147c0 con 0x7fb2fc160770 2024-03-29T09:10:28.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.375+0000 7fb2f2000700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2dc06ac90 0x7fb2dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.375+0000 7fb2f2000700 1 -- 172.21.15.99:0/171363949 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb2f4089cc0 con 0x7fb2fc160770 2024-03-29T09:10:28.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.375+0000 7fb2f9e00700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2dc06ac90 0x7fb2dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.376+0000 7fb2f9e00700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2dc06ac90 0x7fb2dc06d150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009130 tx=0x7fb2ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:28.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.380+0000 7fb2f2000700 1 -- 172.21.15.99:0/171363949 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb2f4090080 con 0x7fb2fc160770 2024-03-29T09:10:28.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.581+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2fc0678b0 con 0x7fb2fc160770 2024-03-29T09:10:28.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.581+0000 7fb2f2000700 1 -- 172.21.15.99:0/171363949 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb2f4054b40 con 0x7fb2fc160770 2024-03-29T09:10:28.581 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2dc06ac90 msgr2=0x7fb2dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2dc06ac90 0x7fb2dc06d150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009130 tx=0x7fb2ec008030 comp rx=0 tx=0).stop 2024-03-29T09:10:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 msgr2=0x7fb2fc0cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0cfd90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4007e30 tx=0x7fb2f400e3c0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 shutdown_connections 2024-03-29T09:10:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb2dc06ac90 0x7fb2dc06d150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2fc160770 0x7fb2fc0cfd90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2fc0d0d60 0x7fb2fc1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 --2- 172.21.15.99:0/171363949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2fc0d3550 0x7fb2fc0b5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 >> 172.21.15.99:0/171363949 conn(0x7fb2fc08afd0 msgr2=0x7fb2fc0b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 shutdown_connections 2024-03-29T09:10:28.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.584+0000 7fb301200700 1 -- 172.21.15.99:0/171363949 wait complete. 2024-03-29T09:10:28.603 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:10:28.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.792+0000 7f1b72c00700 1 Processor -- start 2024-03-29T09:10:28.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.793+0000 7f1b72c00700 1 -- start start 2024-03-29T09:10:28.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b72c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b6c0d1b20 0x7f1b6c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b72c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c160cc0 0x7f1b6c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b72c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6c0cddc0 con 0x7f1b6c1608e0 2024-03-29T09:10:28.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b72c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0ce0a0 con 0x7f1b6c0d1b20 2024-03-29T09:10:28.794 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b72c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0ce380 con 0x7f1b6c160cc0 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b6b400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b6c0d1b20 0x7f1b6c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b70c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c160cc0 0x7f1b6c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b6b400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b6c0d1b20 0x7f1b6c0d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:38900/0 (socket says 172.21.15.99:38900) 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b70c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c160cc0 0x7f1b6c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:54624/0 (socket says 172.21.15.99:54624) 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b6b400700 1 -- 172.21.15.99:0/54140441 learned_addr learned my addr 172.21.15.99:0/54140441 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.794+0000 7f1b6be00700 1 --1- 172.21.15.99:0/54140441 >> v1:172.21.15.133:6789/0 conn(0x7f1b6c1608e0 0x7f1b6c0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:50292/0 (socket says 172.21.15.99:50292) 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6b400700 1 -- 172.21.15.99:0/54140441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c160cc0 msgr2=0x7f1b6c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:28.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6b400700 1 --2- 172.21.15.99:0/54140441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c160cc0 0x7f1b6c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6b400700 1 -- 172.21.15.99:0/54140441 >> v1:172.21.15.133:6789/0 conn(0x7f1b6c1608e0 legacy=0x7f1b6c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6b400700 1 -- 172.21.15.99:0/54140441 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c0ce660 con 0x7f1b6c0d1b20 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6b400700 1 --2- 172.21.15.99:0/54140441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c0d1b20 0x7f1b6c0d0150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1b580098f0 tx=0x7f1b58009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=166a290fcdb36745 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6aa00700 1 -- 172.21.15.99:0/54140441 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58011830 con 0x7f1b6c0d1b20 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6aa00700 1 -- 172.21.15.99:0/54140441 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b58011990 con 0x7f1b6c0d1b20 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.795+0000 7f1b6aa00700 1 -- 172.21.15.99:0/54140441 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58011d20 con 0x7f1b6c0d1b20 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 -- 172.21.15.99:0/54140441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c0d1b20 msgr2=0x7f1b6c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 --2- 172.21.15.99:0/54140441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c0d1b20 0x7f1b6c0d0150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1b580098f0 tx=0x7f1b58009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 -- 172.21.15.99:0/54140441 shutdown_connections 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 --2- 172.21.15.99:0/54140441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c160cc0 0x7f1b6c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 --2- 172.21.15.99:0/54140441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c0d1b20 0x7f1b6c0d0150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 -- 172.21.15.99:0/54140441 >> 172.21.15.99:0/54140441 conn(0x7f1b6c09c490 msgr2=0x7f1b6c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 -- 172.21.15.99:0/54140441 shutdown_connections 2024-03-29T09:10:28.797 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.796+0000 7f1b72c00700 1 -- 172.21.15.99:0/54140441 wait complete. 2024-03-29T09:10:28.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.798+0000 7f1b72c00700 1 Processor -- start 2024-03-29T09:10:28.798 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.798+0000 7f1b72c00700 1 -- start start 2024-03-29T09:10:28.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b72c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c0d1b20 0x7f1b6c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b72c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c1608e0 0x7f1b6c14e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b72c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6c160cc0 0x7f1b6c0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b72c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c14d510 con 0x7f1b6c160cc0 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b72c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c14d720 con 0x7f1b6c1608e0 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b72c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c14da00 con 0x7f1b6c0d1b20 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b70c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6c160cc0 0x7f1b6c0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b6be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c0d1b20 0x7f1b6c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b70c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6c160cc0 0x7f1b6c0b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39862/0 (socket says 172.21.15.99:39862) 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b70c00700 1 -- 172.21.15.99:0/1738458814 learned_addr learned my addr 172.21.15.99:0/1738458814 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.799+0000 7f1b6b400700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c1608e0 0x7f1b6c14e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b70c00700 1 -- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c0d1b20 msgr2=0x7f1b6c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b70c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c0d1b20 0x7f1b6c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b70c00700 1 -- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c1608e0 msgr2=0x7f1b6c14e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b70c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c1608e0 0x7f1b6c14e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:28.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b70c00700 1 -- 172.21.15.99:0/1738458814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58009550 con 0x7f1b6c160cc0 2024-03-29T09:10:28.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b6b400700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c1608e0 0x7f1b6c14e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:28.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b70c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6c160cc0 0x7f1b6c0b2830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400efc0 tx=0x7f1b64009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:28.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b63e00700 1 -- 172.21.15.99:0/1738458814 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64009b20 con 0x7f1b6c160cc0 2024-03-29T09:10:28.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b63e00700 1 -- 172.21.15.99:0/1738458814 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6400f040 con 0x7f1b6c160cc0 2024-03-29T09:10:28.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b63e00700 1 -- 172.21.15.99:0/1738458814 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6400cc90 con 0x7f1b6c160cc0 2024-03-29T09:10:28.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c14dce0 con 0x7f1b6c160cc0 2024-03-29T09:10:28.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.800+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c14e2a0 con 0x7f1b6c160cc0 2024-03-29T09:10:28.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.801+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c14e490 con 0x7f1b6c160cc0 2024-03-29T09:10:28.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.802+0000 7f1b63e00700 1 -- 172.21.15.99:0/1738458814 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b64014480 con 0x7f1b6c160cc0 2024-03-29T09:10:28.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.803+0000 7f1b63e00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b4c06ac90 0x7f1b4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:28.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.803+0000 7f1b63e00700 1 -- 172.21.15.99:0/1738458814 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1b6408a120 con 0x7f1b6c160cc0 2024-03-29T09:10:28.803 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.803+0000 7f1b6be00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b4c06ac90 0x7f1b4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:28.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.804+0000 7f1b6be00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b4c06ac90 0x7f1b4c06d150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000f60 tx=0x7f1b5c007ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:28.807 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:28.807+0000 7f1b63e00700 1 -- 172.21.15.99:0/1738458814 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b64054f10 con 0x7f1b6c160cc0 2024-03-29T09:10:29.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.051+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b6c0677a0 con 0x7f1b6c160cc0 2024-03-29T09:10:29.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.052+0000 7f1b63e00700 1 -- 172.21.15.99:0/1738458814 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f1b64058530 con 0x7f1b6c160cc0 2024-03-29T09:10:29.052 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:29.052 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b4c06ac90 msgr2=0x7f1b4c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b4c06ac90 0x7f1b4c06d150 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000f60 tx=0x7f1b5c007ed0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6c160cc0 msgr2=0x7f1b6c0b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6c160cc0 0x7f1b6c0b2830 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1b6400efc0 tx=0x7f1b64009f60 comp rx=0 tx=0).stop 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 shutdown_connections 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b4c06ac90 0x7f1b4c06d150 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b6c160cc0 0x7f1b6c0b2830 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b6c0d1b20 0x7f1b6c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.054+0000 7f1b72c00700 1 --2- 172.21.15.99:0/1738458814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6c1608e0 0x7f1b6c14e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.055+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 >> 172.21.15.99:0/1738458814 conn(0x7f1b6c09c490 msgr2=0x7f1b6c0cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.055+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 shutdown_connections 2024-03-29T09:10:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.055+0000 7f1b72c00700 1 -- 172.21.15.99:0/1738458814 wait complete. 2024-03-29T09:10:29.056 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:10:29.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.265+0000 7f3ff7000700 1 Processor -- start 2024-03-29T09:10:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.265+0000 7f3ff7000700 1 -- start start 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3ff7000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ff00d1b20 0x7f3ff00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3ff7000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ff0160cc0 0x7f3ff00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3ff7000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00cddc0 con 0x7f3ff01608e0 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3ff7000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00ce0a0 con 0x7f3ff00d1b20 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3ff7000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff00ce380 con 0x7f3ff0160cc0 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3fef400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ff00d1b20 0x7f3ff00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3fef400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3ff00d1b20 0x7f3ff00d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39878/0 (socket says 172.21.15.99:39878) 2024-03-29T09:10:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3fef400700 1 -- 172.21.15.99:0/40630808 learned_addr learned my addr 172.21.15.99:0/40630808 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:29.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.266+0000 7f3ff5000700 1 --2- 172.21.15.99:0/40630808 >> v2:172.21.15.133:3300/0 conn(0x7f3ff0160cc0 0x7f3ff00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3fef400700 1 -- 172.21.15.99:0/40630808 >> v2:172.21.15.133:3300/0 conn(0x7f3ff0160cc0 msgr2=0x7f3ff00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3fef400700 1 --2- 172.21.15.99:0/40630808 >> v2:172.21.15.133:3300/0 conn(0x7f3ff0160cc0 0x7f3ff00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3fef400700 1 -- 172.21.15.99:0/40630808 >> v1:172.21.15.133:6789/0 conn(0x7f3ff01608e0 legacy=0x7f3ff00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3fef400700 1 -- 172.21.15.99:0/40630808 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00ce660 con 0x7f3ff00d1b20 2024-03-29T09:10:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3ff5000700 1 --2- 172.21.15.99:0/40630808 >> v2:172.21.15.133:3300/0 conn(0x7f3ff0160cc0 0x7f3ff00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3fef400700 1 --2- 172.21.15.99:0/40630808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff00d0150 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00b91e0 tx=0x7f3fdc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a864332df51252 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3feea00700 1 -- 172.21.15.99:0/40630808 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00b870 con 0x7f3ff00d1b20 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3feea00700 1 -- 172.21.15.99:0/40630808 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc00b9d0 con 0x7f3ff00d1b20 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.267+0000 7f3feea00700 1 -- 172.21.15.99:0/40630808 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00bd60 con 0x7f3ff00d1b20 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 -- 172.21.15.99:0/40630808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 msgr2=0x7f3ff00d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 --2- 172.21.15.99:0/40630808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff00d0150 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00b91e0 tx=0x7f3fdc009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 -- 172.21.15.99:0/40630808 shutdown_connections 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 --2- 172.21.15.99:0/40630808 >> v2:172.21.15.133:3300/0 conn(0x7f3ff0160cc0 0x7f3ff00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 --2- 172.21.15.99:0/40630808 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff00d0150 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 -- 172.21.15.99:0/40630808 >> 172.21.15.99:0/40630808 conn(0x7f3ff009c490 msgr2=0x7f3ff00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 -- 172.21.15.99:0/40630808 shutdown_connections 2024-03-29T09:10:29.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.268+0000 7f3ff7000700 1 -- 172.21.15.99:0/40630808 wait complete. 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.269+0000 7f3ff7000700 1 Processor -- start 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.269+0000 7f3ff7000700 1 -- start start 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.270+0000 7f3ff7000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.270+0000 7f3ff7000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ff01608e0 0x7f3ff01589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.270+0000 7f3ff7000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff0160cc0 0x7f3ff0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.270+0000 7f3ff7000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00c97d0 con 0x7f3ff00d1b20 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.270+0000 7f3ff7000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00cc860 con 0x7f3ff0160cc0 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.270+0000 7f3ff7000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0158f30 con 0x7f3ff01608e0 2024-03-29T09:10:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff014c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39894/0 (socket says 172.21.15.99:39894) 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 -- 172.21.15.99:0/459199259 learned_addr learned my addr 172.21.15.99:0/459199259 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fef400700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ff01608e0 0x7f3ff01589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3ff5000700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff0160cc0 0x7f3ff0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 -- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ff01608e0 msgr2=0x7f3ff01589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ff01608e0 0x7f3ff01589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 -- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff0160cc0 msgr2=0x7f3ff0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff0160cc0 0x7f3ff0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 -- 172.21.15.99:0/459199259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc009550 con 0x7f3ff00d1b20 2024-03-29T09:10:29.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.271+0000 7f3fefe00700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff014c140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c510 tx=0x7f3fe000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:29.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.272+0000 7f3fe7e00700 1 -- 172.21.15.99:0/459199259 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00052d0 con 0x7f3ff00d1b20 2024-03-29T09:10:29.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.272+0000 7f3fe7e00700 1 -- 172.21.15.99:0/459199259 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe0005430 con 0x7f3ff00d1b20 2024-03-29T09:10:29.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.272+0000 7f3fe7e00700 1 -- 172.21.15.99:0/459199259 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00038c0 con 0x7f3ff00d1b20 2024-03-29T09:10:29.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.272+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff00ca960 con 0x7f3ff00d1b20 2024-03-29T09:10:29.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.272+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff00ccde0 con 0x7f3ff00d1b20 2024-03-29T09:10:29.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.273+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff00677a0 con 0x7f3ff00d1b20 2024-03-29T09:10:29.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.274+0000 7f3fe7e00700 1 -- 172.21.15.99:0/459199259 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3fe0003a20 con 0x7f3ff00d1b20 2024-03-29T09:10:29.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.274+0000 7f3fe7e00700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3fd006ac90 0x7f3fd006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.275+0000 7f3fe7e00700 1 -- 172.21.15.99:0/459199259 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3fe0088a70 con 0x7f3ff00d1b20 2024-03-29T09:10:29.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.275+0000 7f3fef400700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3fd006ac90 0x7f3fd006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.276+0000 7f3fef400700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3fd006ac90 0x7f3fd006d150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc000c00 tx=0x7f3fdc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:29.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.279+0000 7f3fe7e00700 1 -- 172.21.15.99:0/459199259 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3fe0053860 con 0x7f3ff00d1b20 2024-03-29T09:10:29.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.489+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff00d0700 con 0x7f3ff00d1b20 2024-03-29T09:10:29.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.490+0000 7f3fe7e00700 1 -- 172.21.15.99:0/459199259 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7f3fe0056e80 con 0x7f3ff00d1b20 2024-03-29T09:10:29.490 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:29.490 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3fd006ac90 msgr2=0x7f3fd006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3fd006ac90 0x7f3fd006d150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc000c00 tx=0x7f3fdc008030 comp rx=0 tx=0).stop 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 msgr2=0x7f3ff014c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff014c140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000c510 tx=0x7f3fe000c8d0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 shutdown_connections 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3fd006ac90 0x7f3fd006d150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff0160cc0 0x7f3ff0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ff00d1b20 0x7f3ff014c140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 --2- 172.21.15.99:0/459199259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ff01608e0 0x7f3ff01589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 >> 172.21.15.99:0/459199259 conn(0x7f3ff009c490 msgr2=0x7f3ff00cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:29.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 shutdown_connections 2024-03-29T09:10:29.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.493+0000 7f3ff7000700 1 -- 172.21.15.99:0/459199259 wait complete. 2024-03-29T09:10:29.511 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T09:10:29.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:10:29.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.702+0000 7f5460600700 1 Processor -- start 2024-03-29T09:10:29.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.702+0000 7f5460600700 1 -- start start 2024-03-29T09:10:29.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f5460600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54580d1b20 0x7f54580d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f5460600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54580cddc0 con 0x7f54581608e0 2024-03-29T09:10:29.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f5460600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54580ce0a0 con 0x7f5458160cc0 2024-03-29T09:10:29.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f5460600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54580ce380 con 0x7f54580d1b20 2024-03-29T09:10:29.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f545d200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54580d1b20 0x7f54580d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f545dc00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f54581608e0 0x7f54580d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54040/0 (socket says 172.21.15.99:54040) 2024-03-29T09:10:29.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f545d200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54580d1b20 0x7f54580d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39910/0 (socket says 172.21.15.99:39910) 2024-03-29T09:10:29.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.703+0000 7f545dc00700 1 -- 172.21.15.99:0/4272892351 learned_addr learned my addr 172.21.15.99:0/4272892351 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.704+0000 7f545d200700 1 -- 172.21.15.99:0/4272892351 >> v1:172.21.15.133:6789/0 conn(0x7f5458160cc0 legacy=0x7f54580afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.704+0000 7f545d200700 1 -- 172.21.15.99:0/4272892351 >> v1:172.21.15.99:6789/0 conn(0x7f54581608e0 legacy=0x7f54580d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.704+0000 7f545d200700 1 -- 172.21.15.99:0/4272892351 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54580ce660 con 0x7f54580d1b20 2024-03-29T09:10:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.704+0000 7f545d200700 1 --2- 172.21.15.99:0/4272892351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54580d1b20 0x7f54580d0150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5444009840 tx=0x7f5444009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b213f155ae6a762 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.704+0000 7f5457e00700 1 -- 172.21.15.99:0/4272892351 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444004230 con 0x7f54580d1b20 2024-03-29T09:10:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.704+0000 7f5457e00700 1 -- 172.21.15.99:0/4272892351 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5444004390 con 0x7f54580d1b20 2024-03-29T09:10:29.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.704+0000 7f5457e00700 1 -- 172.21.15.99:0/4272892351 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54440036a0 con 0x7f54580d1b20 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.705+0000 7f5460600700 1 -- 172.21.15.99:0/4272892351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54580d1b20 msgr2=0x7f54580d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.705+0000 7f5460600700 1 --2- 172.21.15.99:0/4272892351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54580d1b20 0x7f54580d0150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5444009840 tx=0x7f5444009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.705+0000 7f5460600700 1 -- 172.21.15.99:0/4272892351 shutdown_connections 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.705+0000 7f5460600700 1 --2- 172.21.15.99:0/4272892351 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54580d1b20 0x7f54580d0150 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.705+0000 7f5460600700 1 -- 172.21.15.99:0/4272892351 >> 172.21.15.99:0/4272892351 conn(0x7f545809c490 msgr2=0x7f54580d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.705+0000 7f5460600700 1 -- 172.21.15.99:0/4272892351 shutdown_connections 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.705+0000 7f5460600700 1 -- 172.21.15.99:0/4272892351 wait complete. 2024-03-29T09:10:29.706 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.707+0000 7f5460600700 1 Processor -- start 2024-03-29T09:10:29.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.707+0000 7f5460600700 1 -- start start 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f5460600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54580d1b20 0x7f545814fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f5460600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54581608e0 0x7f5458150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f5460600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5458160cc0 0x7f54580b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f5460600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545814d180 con 0x7f54581608e0 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f5460600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5458150130 con 0x7f54580d1b20 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f5460600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545814d3d0 con 0x7f5458160cc0 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54581608e0 0x7f5458150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54581608e0 0x7f5458150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39914/0 (socket says 172.21.15.99:39914) 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 -- 172.21.15.99:0/2679991252 learned_addr learned my addr 172.21.15.99:0/2679991252 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:29.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545dc00700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54580d1b20 0x7f545814fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545e600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5458160cc0 0x7f54580b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 -- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5458160cc0 msgr2=0x7f54580b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5458160cc0 0x7f54580b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 -- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54580d1b20 msgr2=0x7f545814fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54580d1b20 0x7f545814fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545d200700 1 -- 172.21.15.99:0/2679991252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440094a0 con 0x7f54581608e0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.708+0000 7f545e600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5458160cc0 0x7f54580b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.709+0000 7f545d200700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54581608e0 0x7f5458150670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f54440090b0 tx=0x7f5444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.709+0000 7f5455600700 1 -- 172.21.15.99:0/2679991252 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444011830 con 0x7f54581608e0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.709+0000 7f5455600700 1 -- 172.21.15.99:0/2679991252 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544401e070 con 0x7f54581608e0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.709+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545814d6b0 con 0x7f54581608e0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.709+0000 7f5455600700 1 -- 172.21.15.99:0/2679991252 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444011bf0 con 0x7f54581608e0 2024-03-29T09:10:29.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.709+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545814dc70 con 0x7f54581608e0 2024-03-29T09:10:29.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.710+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545806c5c0 con 0x7f54581608e0 2024-03-29T09:10:29.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.711+0000 7f5455600700 1 -- 172.21.15.99:0/2679991252 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5444011d50 con 0x7f54581608e0 2024-03-29T09:10:29.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.712+0000 7f5455600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f543806ac90 0x7f543806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:29.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.712+0000 7f5455600700 1 -- 172.21.15.99:0/2679991252 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f544408aae0 con 0x7f54581608e0 2024-03-29T09:10:29.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.712+0000 7f545dc00700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f543806ac90 0x7f543806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:29.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.713+0000 7f545dc00700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f543806ac90 0x7f543806d150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f544c005910 tx=0x7f544c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:29.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.717+0000 7f5455600700 1 -- 172.21.15.99:0/2679991252 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f54440558d0 con 0x7f54581608e0 2024-03-29T09:10:29.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.964+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54580677a0 con 0x7f54581608e0 2024-03-29T09:10:29.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.964+0000 7f5455600700 1 -- 172.21.15.99:0/2679991252 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f5444020020 con 0x7f54581608e0 2024-03-29T09:10:29.964 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:29.964 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.966+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f543806ac90 msgr2=0x7f543806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.966+0000 7f5460600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f543806ac90 0x7f543806d150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f544c005910 tx=0x7f544c009040 comp rx=0 tx=0).stop 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54581608e0 msgr2=0x7f5458150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54581608e0 0x7f5458150670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f54440090b0 tx=0x7f5444004750 comp rx=0 tx=0).stop 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 shutdown_connections 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f543806ac90 0x7f543806d150 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5458160cc0 0x7f54580b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54580d1b20 0x7f545814fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 --2- 172.21.15.99:0/2679991252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54581608e0 0x7f5458150670 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:29.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 >> 172.21.15.99:0/2679991252 conn(0x7f545809c490 msgr2=0x7f545815e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:29.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 shutdown_connections 2024-03-29T09:10:29.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:29.967+0000 7f5460600700 1 -- 172.21.15.99:0/2679991252 wait complete. 2024-03-29T09:10:29.968 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:10:29.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:10:30.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.176+0000 7f8a13c00700 1 Processor -- start 2024-03-29T09:10:30.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.176+0000 7f8a13c00700 1 -- start start 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a13c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15d690 0x7f8a0c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a13c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a0c15da70 0x7f8a0c0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a13c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0afde0 con 0x7f8a0c0d1740 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a13c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cf660 con 0x7f8a0c15da70 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a13c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0d01d0 con 0x7f8a0c15d690 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a0be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8a0c15da70 0x7f8a0c0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a11c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8a0c0d1740 0x7f8a0c0cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54050/0 (socket says 172.21.15.99:54050) 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.177+0000 7f8a11c00700 1 -- 172.21.15.99:0/1233508635 learned_addr learned my addr 172.21.15.99:0/1233508635 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a11200700 1 --2- 172.21.15.99:0/1233508635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15d690 0x7f8a0c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0be00700 1 -- 172.21.15.99:0/1233508635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15d690 msgr2=0x7f8a0c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:30.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0be00700 1 --2- 172.21.15.99:0/1233508635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15d690 0x7f8a0c0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0be00700 1 -- 172.21.15.99:0/1233508635 >> v1:172.21.15.99:6789/0 conn(0x7f8a0c0d1740 legacy=0x7f8a0c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0be00700 1 -- 172.21.15.99:0/1233508635 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0d04b0 con 0x7f8a0c15da70 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a11200700 1 --2- 172.21.15.99:0/1233508635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15d690 0x7f8a0c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0b400700 1 -- 172.21.15.99:0/1233508635 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498928063 0 0) 0x7f8a0c0afde0 con 0x7f8a0c0d1740 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0be00700 1 --2- 172.21.15.99:0/1233508635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c15da70 0x7f8a0c0cf120 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873aeb6e3d83ae0d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0b400700 1 -- 172.21.15.99:0/1233508635 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011830 con 0x7f8a0c15da70 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0b400700 1 -- 172.21.15.99:0/1233508635 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8011990 con 0x7f8a0c15da70 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.178+0000 7f8a0b400700 1 -- 172.21.15.99:0/1233508635 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011d20 con 0x7f8a0c15da70 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 -- 172.21.15.99:0/1233508635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c15da70 msgr2=0x7f8a0c0cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 --2- 172.21.15.99:0/1233508635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c15da70 0x7f8a0c0cf120 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f8009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 -- 172.21.15.99:0/1233508635 shutdown_connections 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 --2- 172.21.15.99:0/1233508635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15d690 0x7f8a0c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 --2- 172.21.15.99:0/1233508635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c15da70 0x7f8a0c0cf120 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 -- 172.21.15.99:0/1233508635 >> 172.21.15.99:0/1233508635 conn(0x7f8a0c09c480 msgr2=0x7f8a0c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:30.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 -- 172.21.15.99:0/1233508635 shutdown_connections 2024-03-29T09:10:30.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.179+0000 7f8a13c00700 1 -- 172.21.15.99:0/1233508635 wait complete. 2024-03-29T09:10:30.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.181+0000 7f8a13c00700 1 Processor -- start 2024-03-29T09:10:30.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.181+0000 7f8a13c00700 1 -- start start 2024-03-29T09:10:30.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a13c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c0d1740 0x7f8a0c15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a13c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0c15d690 0x7f8a0c06bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a13c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15da70 0x7f8a0c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a13c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c15af50 con 0x7f8a0c0d1740 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a13c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c1f6530 con 0x7f8a0c15d690 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a13c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c1f6810 con 0x7f8a0c15da70 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c0d1740 0x7f8a0c15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c0d1740 0x7f8a0c15aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39928/0 (socket says 172.21.15.99:39928) 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 -- 172.21.15.99:0/3772401755 learned_addr learned my addr 172.21.15.99:0/3772401755 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15da70 0x7f8a0c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a0be00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0c15d690 0x7f8a0c06bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 -- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15da70 msgr2=0x7f8a0c06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15da70 0x7f8a0c06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 -- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0c15d690 msgr2=0x7f8a0c06bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0c15d690 0x7f8a0c06bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.182+0000 7f8a11200700 1 -- 172.21.15.99:0/3772401755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80094a0 con 0x7f8a0c0d1740 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a0be00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0c15d690 0x7f8a0c06bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:30.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a11c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15da70 0x7f8a0c06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:30.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a11200700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c0d1740 0x7f8a0c15aa10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e810 tx=0x7f8a0000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:30.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a08c00700 1 -- 172.21.15.99:0/3772401755 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000c020 con 0x7f8a0c0d1740 2024-03-29T09:10:30.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a08c00700 1 -- 172.21.15.99:0/3772401755 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a00004250 con 0x7f8a0c0d1740 2024-03-29T09:10:30.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c1f6af0 con 0x7f8a0c0d1740 2024-03-29T09:10:30.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a08c00700 1 -- 172.21.15.99:0/3772401755 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a000036a0 con 0x7f8a0c0d1740 2024-03-29T09:10:30.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.183+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c1f70b0 con 0x7f8a0c0d1740 2024-03-29T09:10:30.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.184+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c1f7420 con 0x7f8a0c0d1740 2024-03-29T09:10:30.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.185+0000 7f8a08c00700 1 -- 172.21.15.99:0/3772401755 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8a00003800 con 0x7f8a0c0d1740 2024-03-29T09:10:30.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.186+0000 7f8a08c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f89ec06ac90 0x7f89ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.186+0000 7f8a08c00700 1 -- 172.21.15.99:0/3772401755 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8a000890e0 con 0x7f8a0c0d1740 2024-03-29T09:10:30.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.186+0000 7f8a0be00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f89ec06ac90 0x7f89ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.187+0000 7f8a0be00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f89ec06ac90 0x7f89ec06d150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:30.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.190+0000 7f8a08c00700 1 -- 172.21.15.99:0/3772401755 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8a00053f60 con 0x7f8a0c0d1740 2024-03-29T09:10:30.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.443+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a0c06d300 con 0x7f8a0c0d1740 2024-03-29T09:10:30.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.443+0000 7f8a08c00700 1 -- 172.21.15.99:0/3772401755 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4730 (secure 0 0 0) 0x7f8a00057580 con 0x7f8a0c0d1740 2024-03-29T09:10:30.443 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:30.444 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/2089183507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":2089183507},{"type":"v1","addr":"172.21.15.99:6836","nonce":2089183507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/1983243912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1983243912},{"type":"v1","addr":"172.21.15.133:6833","nonce":1983243912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/415185150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":415185150},{"type":"v1","addr":"172.21.15.99:6837","nonce":415185150}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:08:57.386923+0000","modified":"2024-03-29T09:09:24.917304+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/440339259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":440339259},{"type":"v1","addr":"172.21.15.133:6835","nonce":440339259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T09:10:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f89ec06ac90 msgr2=0x7f89ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f89ec06ac90 0x7f89ec06d150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f801c040 comp rx=0 tx=0).stop 2024-03-29T09:10:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c0d1740 msgr2=0x7f8a0c15aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c0d1740 0x7f8a0c15aa10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e810 tx=0x7f8a0000ebd0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 shutdown_connections 2024-03-29T09:10:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f89ec06ac90 0x7f89ec06d150 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a0c15da70 0x7f8a0c06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a0c0d1740 0x7f8a0c15aa10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 --2- 172.21.15.99:0/3772401755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a0c15d690 0x7f8a0c06bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 >> 172.21.15.99:0/3772401755 conn(0x7f8a0c09c480 msgr2=0x7f8a0c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 shutdown_connections 2024-03-29T09:10:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.446+0000 7f8a13c00700 1 -- 172.21.15.99:0/3772401755 wait complete. 2024-03-29T09:10:30.447 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 8 2024-03-29T09:10:30.464 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:30.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.654+0000 7fd3ff400700 1 Processor -- start 2024-03-29T09:10:30.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.654+0000 7fd3ff400700 1 -- start start 2024-03-29T09:10:30.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.655+0000 7fd3ff400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f80cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.655+0000 7fd3ff400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd3f815da70 0x7fd3f80cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.655+0000 7fd3ff400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f80afde0 con 0x7fd3f80d1740 2024-03-29T09:10:30.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.655+0000 7fd3ff400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f80cf660 con 0x7fd3f815da70 2024-03-29T09:10:30.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.655+0000 7fd3ff400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80d01d0 con 0x7fd3f815d690 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.656+0000 7fd3fca00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f80cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.656+0000 7fd3f7e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd3f815da70 0x7fd3f80cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.656+0000 7fd3fd400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd3f80d1740 0x7fd3f80cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:50316/0 (socket says 172.21.15.99:50316) 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.656+0000 7fd3fd400700 1 -- 172.21.15.99:0/3535373093 learned_addr learned my addr 172.21.15.99:0/3535373093 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7e00700 1 -- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 msgr2=0x7fd3f80cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7e00700 1 --2- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f80cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7e00700 1 -- 172.21.15.99:0/3535373093 >> v1:172.21.15.133:6789/0 conn(0x7fd3f80d1740 legacy=0x7fd3f80cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7e00700 1 -- 172.21.15.99:0/3535373093 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80d04b0 con 0x7fd3f815da70 2024-03-29T09:10:30.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3fca00700 1 --2- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f80cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7400700 1 -- 172.21.15.99:0/3535373093 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310558178 0 0) 0x7fd3f80afde0 con 0x7fd3f80d1740 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7e00700 1 --2- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 0x7fd3f80cf120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009840 tx=0x7fd3e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=424f76ee54d57569 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7400700 1 -- 172.21.15.99:0/3535373093 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4011830 con 0x7fd3f815da70 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7400700 1 -- 172.21.15.99:0/3535373093 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e4011990 con 0x7fd3f815da70 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.657+0000 7fd3f7400700 1 -- 172.21.15.99:0/3535373093 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4011d20 con 0x7fd3f815da70 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 -- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 msgr2=0x7fd3f80cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 --2- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 0x7fd3f80cf120 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009840 tx=0x7fd3e4009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 -- 172.21.15.99:0/3535373093 shutdown_connections 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 --2- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f80cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 --2- 172.21.15.99:0/3535373093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 0x7fd3f80cf120 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 -- 172.21.15.99:0/3535373093 >> 172.21.15.99:0/3535373093 conn(0x7fd3f809c480 msgr2=0x7fd3f80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:30.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 -- 172.21.15.99:0/3535373093 shutdown_connections 2024-03-29T09:10:30.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.658+0000 7fd3ff400700 1 -- 172.21.15.99:0/3535373093 wait complete. 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.660+0000 7fd3ff400700 1 Processor -- start 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.660+0000 7fd3ff400700 1 -- start start 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3ff400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f80d1740 0x7fd3f815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3ff400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3ff400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 0x7fd3f806c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3ff400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f815af50 con 0x7fd3f80d1740 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3ff400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f81f6530 con 0x7fd3f815da70 2024-03-29T09:10:30.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3ff400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f81f6810 con 0x7fd3f815d690 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3fca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f80d1740 0x7fd3f815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3fca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f80d1740 0x7fd3f815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39936/0 (socket says 172.21.15.99:39936) 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3fca00700 1 -- 172.21.15.99:0/2037215323 learned_addr learned my addr 172.21.15.99:0/2037215323 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3f7e00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3fca00700 1 -- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 msgr2=0x7fd3f806bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3fca00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f806bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3fca00700 1 -- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 msgr2=0x7fd3f806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.661+0000 7fd3fca00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 0x7fd3f806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3fca00700 1 -- 172.21.15.99:0/2037215323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0094a0 con 0x7fd3f80d1740 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3f7e00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:30.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3fca00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f80d1740 0x7fd3f815aa10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00ebe0 tx=0x7fd3ec009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3f4c00700 1 -- 172.21.15.99:0/2037215323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00c8b0 con 0x7fd3f80d1740 2024-03-29T09:10:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3f4c00700 1 -- 172.21.15.99:0/2037215323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec00ca10 con 0x7fd3f80d1740 2024-03-29T09:10:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e40094a0 con 0x7fd3f80d1740 2024-03-29T09:10:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3f4c00700 1 -- 172.21.15.99:0/2037215323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00cda0 con 0x7fd3f80d1740 2024-03-29T09:10:30.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.662+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f81f6dd0 con 0x7fd3f80d1740 2024-03-29T09:10:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.663+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f806d300 con 0x7fd3f80d1740 2024-03-29T09:10:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.664+0000 7fd3f4c00700 1 -- 172.21.15.99:0/2037215323 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd3ec013470 con 0x7fd3f80d1740 2024-03-29T09:10:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.665+0000 7fd3f4c00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd3d806ac90 0x7fd3d806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.665+0000 7fd3f4c00700 1 -- 172.21.15.99:0/2037215323 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd3ec0897d0 con 0x7fd3f80d1740 2024-03-29T09:10:30.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.665+0000 7fd3f7e00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd3d806ac90 0x7fd3d806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:30.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.666+0000 7fd3f7e00700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd3d806ac90 0x7fd3d806d150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009840 tx=0x7fd3e401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:30.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.669+0000 7fd3f4c00700 1 -- 172.21.15.99:0/2037215323 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd3ec054650 con 0x7fd3f80d1740 2024-03-29T09:10:30.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.873+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3f80677a0 con 0x7fd3f80d1740 2024-03-29T09:10:30.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.874+0000 7fd3f4c00700 1 -- 172.21.15.99:0/2037215323 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18024 (secure 0 0 0) 0x7fd3ec057c70 con 0x7fd3f80d1740 2024-03-29T09:10:30.874 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:30.874 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":26,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:09:41.997389+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:08:55.815767+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T09:08:56.358550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-29T09:08:59.440030+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2f","mappings":[{"from":0,"to":7}]},{"pgid":"4.4","mappings":[{"from":7,"to":4}]},{"pgid":"4.13","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd3d806ac90 msgr2=0x7fd3d806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd3d806ac90 0x7fd3d806d150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009840 tx=0x7fd3e401c040 comp rx=0 tx=0).stop 2024-03-29T09:10:30.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f80d1740 msgr2=0x7fd3f815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f80d1740 0x7fd3f815aa10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00ebe0 tx=0x7fd3ec009e20 comp rx=0 tx=0).stop 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 shutdown_connections 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd3d806ac90 0x7fd3d806d150 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3f815da70 0x7fd3f806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3f80d1740 0x7fd3f815aa10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 --2- 172.21.15.99:0/2037215323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3f815d690 0x7fd3f806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 >> 172.21.15.99:0/2037215323 conn(0x7fd3f809c480 msgr2=0x7fd3f80b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 shutdown_connections 2024-03-29T09:10:30.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:30.877+0000 7fd3ff400700 1 -- 172.21.15.99:0/2037215323 wait complete. 2024-03-29T09:10:30.895 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T09:10:31.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.085+0000 7f1b9be00700 1 Processor -- start 2024-03-29T09:10:31.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.086+0000 7f1b9be00700 1 -- start start 2024-03-29T09:10:31.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b9be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b940d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b9be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b94160cc0 0x7f1b940ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b9be00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b940cddc0 con 0x7f1b940d3a70 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b9be00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b940ce0a0 con 0x7f1b94160cc0 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b9be00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940ce380 con 0x7f1b941608e0 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b98a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b94160cc0 0x7f1b940ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b99e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1b940d3a70 0x7f1b940d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54066/0 (socket says 172.21.15.99:54066) 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b99e00700 1 -- 172.21.15.99:0/3709871218 learned_addr learned my addr 172.21.15.99:0/3709871218 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b99400700 1 --2- 172.21.15.99:0/3709871218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b940d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b98a00700 1 -- 172.21.15.99:0/3709871218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 msgr2=0x7f1b940d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b98a00700 1 --2- 172.21.15.99:0/3709871218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b940d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b98a00700 1 -- 172.21.15.99:0/3709871218 >> v1:172.21.15.99:6789/0 conn(0x7f1b940d3a70 legacy=0x7f1b940d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b98a00700 1 -- 172.21.15.99:0/3709871218 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940ce660 con 0x7f1b94160cc0 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.087+0000 7f1b99400700 1 --2- 172.21.15.99:0/3709871218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b940d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.088+0000 7f1b98a00700 1 --2- 172.21.15.99:0/3709871218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b94160cc0 0x7f1b940ceb80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009840 tx=0x7f1b80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a410ea2414b6cfe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:31.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.088+0000 7f1b93e00700 1 -- 172.21.15.99:0/3709871218 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011830 con 0x7f1b94160cc0 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.088+0000 7f1b93e00700 1 -- 172.21.15.99:0/3709871218 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80011990 con 0x7f1b94160cc0 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.088+0000 7f1b93e00700 1 -- 172.21.15.99:0/3709871218 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80011d20 con 0x7f1b94160cc0 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.088+0000 7f1b9be00700 1 -- 172.21.15.99:0/3709871218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b94160cc0 msgr2=0x7f1b940ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.088+0000 7f1b9be00700 1 --2- 172.21.15.99:0/3709871218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b94160cc0 0x7f1b940ceb80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009840 tx=0x7f1b80009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.089+0000 7f1b9be00700 1 -- 172.21.15.99:0/3709871218 shutdown_connections 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.089+0000 7f1b9be00700 1 --2- 172.21.15.99:0/3709871218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b940d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.089+0000 7f1b9be00700 1 --2- 172.21.15.99:0/3709871218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b94160cc0 0x7f1b940ceb80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.089+0000 7f1b9be00700 1 -- 172.21.15.99:0/3709871218 >> 172.21.15.99:0/3709871218 conn(0x7f1b9409c490 msgr2=0x7f1b940d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.089+0000 7f1b9be00700 1 -- 172.21.15.99:0/3709871218 shutdown_connections 2024-03-29T09:10:31.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.089+0000 7f1b9be00700 1 -- 172.21.15.99:0/3709871218 wait complete. 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.090+0000 7f1b9be00700 1 Processor -- start 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.090+0000 7f1b9be00700 1 -- start start 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.091+0000 7f1b9be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b940d3a70 0x7f1b940ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.091+0000 7f1b9be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b941574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.091+0000 7f1b9be00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94160cc0 0x7f1b940cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.091+0000 7f1b9be00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94158960 con 0x7f1b940d3a70 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.091+0000 7f1b9be00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94157a30 con 0x7f1b94160cc0 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.091+0000 7f1b9be00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b94157ce0 con 0x7f1b941608e0 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b940d3a70 0x7f1b940ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b940d3a70 0x7f1b940ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39962/0 (socket says 172.21.15.99:39962) 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 -- 172.21.15.99:0/1957874780 learned_addr learned my addr 172.21.15.99:0/1957874780 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b98a00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b941574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 -- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 msgr2=0x7f1b941574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99e00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94160cc0 0x7f1b940cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b941574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 -- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94160cc0 msgr2=0x7f1b940cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94160cc0 0x7f1b940cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 -- 172.21.15.99:0/1957874780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800094a0 con 0x7f1b940d3a70 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b98a00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b941574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99e00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94160cc0 0x7f1b940cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.092+0000 7f1b99400700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b940d3a70 0x7f1b940ca370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e810 tx=0x7f1b8800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.093+0000 7f1b91600700 1 -- 172.21.15.99:0/1957874780 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88013610 con 0x7f1b940d3a70 2024-03-29T09:10:31.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.093+0000 7f1b91600700 1 -- 172.21.15.99:0/1957874780 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88013770 con 0x7f1b940d3a70 2024-03-29T09:10:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.093+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b94157fc0 con 0x7f1b940d3a70 2024-03-29T09:10:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.093+0000 7f1b91600700 1 -- 172.21.15.99:0/1957874780 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88013b20 con 0x7f1b940d3a70 2024-03-29T09:10:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.093+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b94158580 con 0x7f1b940d3a70 2024-03-29T09:10:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.094+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b94158aa0 con 0x7f1b940d3a70 2024-03-29T09:10:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.095+0000 7f1b91600700 1 -- 172.21.15.99:0/1957874780 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b88013c80 con 0x7f1b940d3a70 2024-03-29T09:10:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.095+0000 7f1b91600700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b7406ac90 0x7f1b7406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.096+0000 7f1b91600700 1 -- 172.21.15.99:0/1957874780 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1b88089390 con 0x7f1b940d3a70 2024-03-29T09:10:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.096+0000 7f1b98a00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b7406ac90 0x7f1b7406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.096+0000 7f1b98a00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b7406ac90 0x7f1b7406d150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b940d3050 tx=0x7f1b80008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.100+0000 7f1b91600700 1 -- 172.21.15.99:0/1957874780 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b88054210 con 0x7f1b940d3a70 2024-03-29T09:10:31.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.346+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1b940677a0 con 0x7f1b940d3a70 2024-03-29T09:10:31.347 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-03-29T09:10:31.347+0000 7fe659e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T09:10:31.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.355+0000 7f1b91600700 1 -- 172.21.15.99:0/1957874780 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1b88057830 con 0x7f1b940d3a70 2024-03-29T09:10:31.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b7406ac90 msgr2=0x7f1b7406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b7406ac90 0x7f1b7406d150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b940d3050 tx=0x7f1b80008030 comp rx=0 tx=0).stop 2024-03-29T09:10:31.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b940d3a70 msgr2=0x7f1b940ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b940d3a70 0x7f1b940ca370 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e810 tx=0x7f1b8800ebd0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 shutdown_connections 2024-03-29T09:10:31.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b7406ac90 0x7f1b7406d150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b94160cc0 0x7f1b940cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b940d3a70 0x7f1b940ca370 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 --2- 172.21.15.99:0/1957874780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b941608e0 0x7f1b941574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 >> 172.21.15.99:0/1957874780 conn(0x7f1b9409c490 msgr2=0x7f1b940b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:31.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 shutdown_connections 2024-03-29T09:10:31.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.358+0000 7f1b9be00700 1 -- 172.21.15.99:0/1957874780 wait complete. 2024-03-29T09:10:31.359 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T09:10:31.367 INFO:tasks.ceph.mds.d.smithi133.stderr: -6212> 2024-03-29T09:08:58.876+0000 7fa564000700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T09:10:31.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T09:10:31.421 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-03-29T09:10:31.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.568+0000 7f2392600700 1 Processor -- start 2024-03-29T09:10:31.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.568+0000 7f2392600700 1 -- start start 2024-03-29T09:10:31.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f2392600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f238c0b3cc0 0x7f238c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f2392600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f2392600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238c0d3810 con 0x7f238c160770 2024-03-29T09:10:31.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f2392600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238c0b4140 con 0x7f238c0b3cc0 2024-03-29T09:10:31.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f2392600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238c0d1520 con 0x7f238c0d3380 2024-03-29T09:10:31.569 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f238aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f238c0b3cc0 0x7f238c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f238aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f238c0b3cc0 0x7f238c0d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39974/0 (socket says 172.21.15.99:39974) 2024-03-29T09:10:31.570 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.569+0000 7f238aa00700 1 -- 172.21.15.99:0/3268822076 learned_addr learned my addr 172.21.15.99:0/3268822076 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:31.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238be00700 1 --2- 172.21.15.99:0/3268822076 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238aa00700 1 -- 172.21.15.99:0/3268822076 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 msgr2=0x7f238c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:31.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238a000700 1 -- 172.21.15.99:0/3268822076 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698014642 0 0) 0x7f238c0d3810 con 0x7f238c160770 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238aa00700 1 --2- 172.21.15.99:0/3268822076 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238aa00700 1 -- 172.21.15.99:0/3268822076 >> v1:172.21.15.133:6789/0 conn(0x7f238c160770 legacy=0x7f238c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238aa00700 1 -- 172.21.15.99:0/3268822076 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c0d1800 con 0x7f238c0b3cc0 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238be00700 1 --2- 172.21.15.99:0/3268822076 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238aa00700 1 --2- 172.21.15.99:0/3268822076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c0b3cc0 0x7f238c0d2e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23780098f0 tx=0x7f2378009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3858c2a1b57bda8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238a000700 1 -- 172.21.15.99:0/3268822076 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800b8a0 con 0x7f238c0b3cc0 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238a000700 1 -- 172.21.15.99:0/3268822076 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237800ba00 con 0x7f238c0b3cc0 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.570+0000 7f238a000700 1 -- 172.21.15.99:0/3268822076 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237800bd90 con 0x7f238c0b3cc0 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 -- 172.21.15.99:0/3268822076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c0b3cc0 msgr2=0x7f238c0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 --2- 172.21.15.99:0/3268822076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c0b3cc0 0x7f238c0d2e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23780098f0 tx=0x7f2378009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 -- 172.21.15.99:0/3268822076 shutdown_connections 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 --2- 172.21.15.99:0/3268822076 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 --2- 172.21.15.99:0/3268822076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c0b3cc0 0x7f238c0d2e40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 -- 172.21.15.99:0/3268822076 >> 172.21.15.99:0/3268822076 conn(0x7f238c08afd0 msgr2=0x7f238c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:31.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 -- 172.21.15.99:0/3268822076 shutdown_connections 2024-03-29T09:10:31.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.571+0000 7f2392600700 1 -- 172.21.15.99:0/3268822076 wait complete. 2024-03-29T09:10:31.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.573+0000 7f2392600700 1 Processor -- start 2024-03-29T09:10:31.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.573+0000 7f2392600700 1 -- start start 2024-03-29T09:10:31.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f2392600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c0b3cc0 0x7f238c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f2392600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f2392600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c160770 0x7f238c1f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f2392600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1f35f0 con 0x7f238c160770 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f2392600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c0b03a0 con 0x7f238c0b3cc0 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f2392600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238c0b0680 con 0x7f238c0d3380 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f238be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c160770 0x7f238c1f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f238be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c160770 0x7f238c1f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39986/0 (socket says 172.21.15.99:39986) 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f238be00700 1 -- 172.21.15.99:0/4105550881 learned_addr learned my addr 172.21.15.99:0/4105550881 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f238b400700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c0b3cc0 0x7f238c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.574+0000 7f238aa00700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238be00700 1 -- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 msgr2=0x7f238c15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238be00700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238be00700 1 -- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c0b3cc0 msgr2=0x7f238c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238be00700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c0b3cc0 0x7f238c0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238be00700 1 -- 172.21.15.99:0/4105550881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2378009550 con 0x7f238c160770 2024-03-29T09:10:31.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238aa00700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:31.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238b400700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c0b3cc0 0x7f238c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:31.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f238be00700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c160770 0x7f238c1f30b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f238400cef0 tx=0x7f238400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:31.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f2383400700 1 -- 172.21.15.99:0/4105550881 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238400ad00 con 0x7f238c160770 2024-03-29T09:10:31.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f2383400700 1 -- 172.21.15.99:0/4105550881 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238400ae60 con 0x7f238c160770 2024-03-29T09:10:31.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c0b0960 con 0x7f238c160770 2024-03-29T09:10:31.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f2383400700 1 -- 172.21.15.99:0/4105550881 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384015680 con 0x7f238c160770 2024-03-29T09:10:31.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.575+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c0b7510 con 0x7f238c160770 2024-03-29T09:10:31.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.576+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238c0b5de0 con 0x7f238c160770 2024-03-29T09:10:31.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.577+0000 7f2383400700 1 -- 172.21.15.99:0/4105550881 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f238400d430 con 0x7f238c160770 2024-03-29T09:10:31.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.578+0000 7f2383400700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f236c06ac90 0x7f236c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:31.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.578+0000 7f2383400700 1 -- 172.21.15.99:0/4105550881 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6633+0+0 (secure 0 0 0) 0x7f2384089b20 con 0x7f238c160770 2024-03-29T09:10:31.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.578+0000 7f238b400700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f236c06ac90 0x7f236c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:31.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.579+0000 7f238b400700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f236c06ac90 0x7f236c06d150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f237c000f60 tx=0x7f237c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:31.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.583+0000 7f2383400700 1 -- 172.21.15.99:0/4105550881 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f238c0b5de0 con 0x7f238c160770 2024-03-29T09:10:31.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:31.829+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f238c0b5de0 con 0x7f238c160770 2024-03-29T09:10:32.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.357+0000 7f2383400700 1 -- 172.21.15.99:0/4105550881 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f238c0b5de0 con 0x7f238c160770 2024-03-29T09:10:32.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f236c06ac90 msgr2=0x7f236c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:32.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f236c06ac90 0x7f236c06d150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f237c000f60 tx=0x7f237c009040 comp rx=0 tx=0).stop 2024-03-29T09:10:32.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c160770 msgr2=0x7f238c1f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:32.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c160770 0x7f238c1f30b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f238400cef0 tx=0x7f238400cf20 comp rx=0 tx=0).stop 2024-03-29T09:10:32.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 shutdown_connections 2024-03-29T09:10:32.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f236c06ac90 0x7f236c06d150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f238c160770 0x7f238c1f30b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c0b3cc0 0x7f238c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 --2- 172.21.15.99:0/4105550881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f238c0d3380 0x7f238c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 >> 172.21.15.99:0/4105550881 conn(0x7f238c08afd0 msgr2=0x7f238c15e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:32.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 shutdown_connections 2024-03-29T09:10:32.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.360+0000 7f2392600700 1 -- 172.21.15.99:0/4105550881 wait complete. 2024-03-29T09:10:32.379 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T09:10:32.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.575+0000 7f3ef0000700 1 Processor -- start 2024-03-29T09:10:32.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.575+0000 7f3ef0000700 1 -- start start 2024-03-29T09:10:32.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3ef0000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee8072000 0x7f3ee80b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:32.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3ef0000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ee8072650 0x7f3ee80b2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:32.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3ef0000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80ad7b0 con 0x7f3ee815d450 2024-03-29T09:10:32.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3ef0000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee815d830 con 0x7f3ee8072650 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3ef0000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80b36c0 con 0x7f3ee8072000 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3eee000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ee8072650 0x7f3ee80b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3eecc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee8072000 0x7f3ee80b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3eecc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee8072000 0x7f3ee80b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:54728/0 (socket says 172.21.15.99:54728) 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.576+0000 7f3eecc00700 1 -- 172.21.15.99:0/1309964204 learned_addr learned my addr 172.21.15.99:0/1309964204 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.577+0000 7f3eee000700 1 -- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee8072000 msgr2=0x7f3ee80b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.577+0000 7f3eee000700 1 --2- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee8072000 0x7f3ee80b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.577+0000 7f3eee000700 1 -- 172.21.15.99:0/1309964204 >> v1:172.21.15.133:6789/0 conn(0x7f3ee815d450 legacy=0x7f3ee80b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.577+0000 7f3eee000700 1 -- 172.21.15.99:0/1309964204 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80ada50 con 0x7f3ee8072650 2024-03-29T09:10:32.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.577+0000 7f3eecc00700 1 --2- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee8072000 0x7f3ee80b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:32.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.577+0000 7f3eee000700 1 --2- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072650 0x7f3ee80b2400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009d90 tx=0x7f3ee000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0111a10a54bc2a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:32.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.577+0000 7f3ee7e00700 1 -- 172.21.15.99:0/1309964204 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000e070 con 0x7f3ee8072650 2024-03-29T09:10:32.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ee7e00700 1 -- 172.21.15.99:0/1309964204 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee000bb20 con 0x7f3ee8072650 2024-03-29T09:10:32.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ee7e00700 1 -- 172.21.15.99:0/1309964204 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0012430 con 0x7f3ee8072650 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 -- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072650 msgr2=0x7f3ee80b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072650 0x7f3ee80b2400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0009d90 tx=0x7f3ee000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 -- 172.21.15.99:0/1309964204 shutdown_connections 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072650 0x7f3ee80b2400 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1309964204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee8072000 0x7f3ee80b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 -- 172.21.15.99:0/1309964204 >> 172.21.15.99:0/1309964204 conn(0x7f3ee80b54d0 msgr2=0x7f3ee80b58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 -- 172.21.15.99:0/1309964204 shutdown_connections 2024-03-29T09:10:32.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.578+0000 7f3ef0000700 1 -- 172.21.15.99:0/1309964204 wait complete. 2024-03-29T09:10:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.580+0000 7f3ef0000700 1 Processor -- start 2024-03-29T09:10:32.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.581+0000 7f3ef0000700 1 -- start start 2024-03-29T09:10:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3ef0000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072000 0x7f3ee80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3ef0000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee8072650 0x7f3ee8151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3ef0000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee815d450 0x7f3ee81544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3ef0000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee815be30 con 0x7f3ee8072650 2024-03-29T09:10:32.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3ef0000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee815b870 con 0x7f3ee8072000 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3ef0000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee815bb50 con 0x7f3ee815d450 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3eecc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee8072650 0x7f3ee8151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3eecc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee8072650 0x7f3ee8151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:39996/0 (socket says 172.21.15.99:39996) 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3eecc00700 1 -- 172.21.15.99:0/1823011174 learned_addr learned my addr 172.21.15.99:0/1823011174 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3eee000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee815d450 0x7f3ee81544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.582+0000 7f3eed600700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072000 0x7f3ee80b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3eecc00700 1 -- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee815d450 msgr2=0x7f3ee81544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3eecc00700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee815d450 0x7f3ee81544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3eecc00700 1 -- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072000 msgr2=0x7f3ee80b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3eecc00700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072000 0x7f3ee80b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3eecc00700 1 -- 172.21.15.99:0/1823011174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed4009550 con 0x7f3ee8072650 2024-03-29T09:10:32.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3eee000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee815d450 0x7f3ee81544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3eecc00700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee8072650 0x7f3ee8151280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400ebe0 tx=0x7f3ed4009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3ee5600700 1 -- 172.21.15.99:0/1823011174 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed400c8b0 con 0x7f3ee8072650 2024-03-29T09:10:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3ee5600700 1 -- 172.21.15.99:0/1823011174 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed400ca10 con 0x7f3ee8072650 2024-03-29T09:10:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee0009a70 con 0x7f3ee8072650 2024-03-29T09:10:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3ee5600700 1 -- 172.21.15.99:0/1823011174 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed400cd80 con 0x7f3ee8072650 2024-03-29T09:10:32.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.583+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee8157d00 con 0x7f3ee8072650 2024-03-29T09:10:32.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.584+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8157f20 con 0x7f3ee8072650 2024-03-29T09:10:32.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.585+0000 7f3ee5600700 1 -- 172.21.15.99:0/1823011174 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3ed4013470 con 0x7f3ee8072650 2024-03-29T09:10:32.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.586+0000 7f3ee5600700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3ec806ac90 0x7f3ec806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:32.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.586+0000 7f3ee5600700 1 -- 172.21.15.99:0/1823011174 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6633+0+0 (secure 0 0 0) 0x7f3ed40899a0 con 0x7f3ee8072650 2024-03-29T09:10:32.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.586+0000 7f3eed600700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3ec806ac90 0x7f3ec806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:32.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.587+0000 7f3eed600700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3ec806ac90 0x7f3ec806d150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3edc000f60 tx=0x7f3edc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:32.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.590+0000 7f3ee5600700 1 -- 172.21.15.99:0/1823011174 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3ed40546c0 con 0x7f3ee8072650 2024-03-29T09:10:32.791 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:32.792+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3ee80678b0 con 0x7f3ee8072650 2024-03-29T09:10:33.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.362+0000 7f3ee5600700 1 -- 172.21.15.99:0/1823011174 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3ed4057ce0 con 0x7f3ee8072650 2024-03-29T09:10:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3ec806ac90 msgr2=0x7f3ec806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3ec806ac90 0x7f3ec806d150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3edc000f60 tx=0x7f3edc009040 comp rx=0 tx=0).stop 2024-03-29T09:10:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee8072650 msgr2=0x7f3ee8151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:33.365 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee8072650 0x7f3ee8151280 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400ebe0 tx=0x7f3ed4009e70 comp rx=0 tx=0).stop 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 shutdown_connections 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3ec806ac90 0x7f3ec806d150 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ee815d450 0x7f3ee81544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ee8072000 0x7f3ee80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 --2- 172.21.15.99:0/1823011174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ee8072650 0x7f3ee8151280 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 >> 172.21.15.99:0/1823011174 conn(0x7f3ee80b54d0 msgr2=0x7f3ee80af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 shutdown_connections 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.365+0000 7f3ef0000700 1 -- 172.21.15.99:0/1823011174 wait complete. 2024-03-29T09:10:33.366 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-03-29T09:10:33.384 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T09:10:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.591+0000 7fb05d200700 1 Processor -- start 2024-03-29T09:10:33.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.591+0000 7fb05d200700 1 -- start start 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb05d200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0581612c0 0x7fb0580b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb05d200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb0580ce2e0 0x7fb0580ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb05d200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0580d1920 con 0x7fb0580b2110 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb05d200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb058162250 con 0x7fb0581612c0 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb05d200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb05815ec70 con 0x7fb0580ce2e0 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb055e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0581612c0 0x7fb0580b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb055e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb0581612c0 0x7fb0580b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40000/0 (socket says 172.21.15.99:40000) 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.592+0000 7fb055e00700 1 -- 172.21.15.99:0/2609089127 learned_addr learned my addr 172.21.15.99:0/2609089127 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:33.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb056800700 1 --2- 172.21.15.99:0/2609089127 >> v2:172.21.15.133:3300/0 conn(0x7fb0580ce2e0 0x7fb0580ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055e00700 1 -- 172.21.15.99:0/2609089127 >> v2:172.21.15.133:3300/0 conn(0x7fb0580ce2e0 msgr2=0x7fb0580ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055e00700 1 --2- 172.21.15.99:0/2609089127 >> v2:172.21.15.133:3300/0 conn(0x7fb0580ce2e0 0x7fb0580ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055e00700 1 -- 172.21.15.99:0/2609089127 >> v1:172.21.15.133:6789/0 conn(0x7fb0580b2110 legacy=0x7fb0581616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055e00700 1 -- 172.21.15.99:0/2609089127 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058161f70 con 0x7fb0581612c0 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb056800700 1 --2- 172.21.15.99:0/2609089127 >> v2:172.21.15.133:3300/0 conn(0x7fb0580ce2e0 0x7fb0580ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055e00700 1 --2- 172.21.15.99:0/2609089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0581612c0 0x7fb0580b47f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb0480098f0 tx=0x7fb048009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899c595048e819d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055400700 1 -- 172.21.15.99:0/2609089127 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048004230 con 0x7fb0581612c0 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055400700 1 -- 172.21.15.99:0/2609089127 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb048004390 con 0x7fb0581612c0 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.593+0000 7fb055400700 1 -- 172.21.15.99:0/2609089127 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0480036a0 con 0x7fb0581612c0 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 -- 172.21.15.99:0/2609089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0581612c0 msgr2=0x7fb0580b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 --2- 172.21.15.99:0/2609089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0581612c0 0x7fb0580b47f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb0480098f0 tx=0x7fb048009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 -- 172.21.15.99:0/2609089127 shutdown_connections 2024-03-29T09:10:33.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 --2- 172.21.15.99:0/2609089127 >> v2:172.21.15.133:3300/0 conn(0x7fb0580ce2e0 0x7fb0580ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 --2- 172.21.15.99:0/2609089127 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0581612c0 0x7fb0580b47f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 -- 172.21.15.99:0/2609089127 >> 172.21.15.99:0/2609089127 conn(0x7fb05809c500 msgr2=0x7fb0580d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 -- 172.21.15.99:0/2609089127 shutdown_connections 2024-03-29T09:10:33.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.594+0000 7fb05d200700 1 -- 172.21.15.99:0/2609089127 wait complete. 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.595+0000 7fb05d200700 1 Processor -- start 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.596+0000 7fb05d200700 1 -- start start 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.596+0000 7fb05d200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0580b2110 0x7fb0580c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.596+0000 7fb05d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0580ce2e0 0x7fb05815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.596+0000 7fb05d200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0581612c0 0x7fb05815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb05d200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0580c99e0 con 0x7fb0580ce2e0 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb05d200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05815d100 con 0x7fb0580b2110 2024-03-29T09:10:33.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb05d200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05815d3e0 con 0x7fb0581612c0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0580ce2e0 0x7fb05815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0580ce2e0 0x7fb05815b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40006/0 (socket says 172.21.15.99:40006) 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 -- 172.21.15.99:0/232600688 learned_addr learned my addr 172.21.15.99:0/232600688 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb055e00700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0580b2110 0x7fb0580c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb056800700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0581612c0 0x7fb05815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 -- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0581612c0 msgr2=0x7fb05815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0581612c0 0x7fb05815b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 -- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0580b2110 msgr2=0x7fb0580c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0580b2110 0x7fb0580c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 -- 172.21.15.99:0/232600688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb048009550 con 0x7fb0580ce2e0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.597+0000 7fb04fe00700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0580ce2e0 0x7fb05815b000 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c4d0 tx=0x7fb04000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.598+0000 7fb04e000700 1 -- 172.21.15.99:0/232600688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011790 con 0x7fb0580ce2e0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.598+0000 7fb04e000700 1 -- 172.21.15.99:0/232600688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0400118f0 con 0x7fb0580ce2e0 2024-03-29T09:10:33.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.598+0000 7fb04e000700 1 -- 172.21.15.99:0/232600688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040011ca0 con 0x7fb0580ce2e0 2024-03-29T09:10:33.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.598+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05815d6c0 con 0x7fb0580ce2e0 2024-03-29T09:10:33.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.598+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05815dc80 con 0x7fb0580ce2e0 2024-03-29T09:10:33.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.599+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05808cfa0 con 0x7fb0580ce2e0 2024-03-29T09:10:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.600+0000 7fb04e000700 1 -- 172.21.15.99:0/232600688 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb04001c460 con 0x7fb0580ce2e0 2024-03-29T09:10:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.601+0000 7fb04e000700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb03406ac90 0x7fb03406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.601+0000 7fb04e000700 1 -- 172.21.15.99:0/232600688 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fb040089290 con 0x7fb0580ce2e0 2024-03-29T09:10:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.601+0000 7fb055e00700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb03406ac90 0x7fb03406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:33.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.601+0000 7fb055e00700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb03406ac90 0x7fb03406d150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb048009130 tx=0x7fb048008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:33.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.605+0000 7fb04e000700 1 -- 172.21.15.99:0/232600688 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb040054280 con 0x7fb0580ce2e0 2024-03-29T09:10:33.810 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:33.811+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb05808cc90 con 0x7fb0580ce2e0 2024-03-29T09:10:34.369 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.369+0000 7fb04e000700 1 -- 172.21.15.99:0/232600688 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb0400578a0 con 0x7fb0580ce2e0 2024-03-29T09:10:34.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb03406ac90 msgr2=0x7fb03406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:34.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb03406ac90 0x7fb03406d150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb048009130 tx=0x7fb048008030 comp rx=0 tx=0).stop 2024-03-29T09:10:34.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0580ce2e0 msgr2=0x7fb05815b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:34.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0580ce2e0 0x7fb05815b000 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb04000c4d0 tx=0x7fb04000c7e0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 shutdown_connections 2024-03-29T09:10:34.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb03406ac90 0x7fb03406d150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0581612c0 0x7fb05815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0580b2110 0x7fb0580c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 --2- 172.21.15.99:0/232600688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0580ce2e0 0x7fb05815b000 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 >> 172.21.15.99:0/232600688 conn(0x7fb05809c500 msgr2=0x7fb05806b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:34.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 shutdown_connections 2024-03-29T09:10:34.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.373+0000 7fb05d200700 1 -- 172.21.15.99:0/232600688 wait complete. 2024-03-29T09:10:34.374 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-03-29T09:10:34.394 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-29T09:10:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.590+0000 7f02ade00700 1 Processor -- start 2024-03-29T09:10:34.590 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.590+0000 7f02ade00700 1 -- start start 2024-03-29T09:10:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02ade00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02ade00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f02a80d0090 0x7f02a80ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02ade00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a80d1fa0 con 0x7f02a80b1f50 2024-03-29T09:10:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02ade00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a80cf440 con 0x7f02a80d0090 2024-03-29T09:10:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02ade00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80d0520 con 0x7f02a8160b30 2024-03-29T09:10:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02a7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f02a80d0090 0x7f02a80ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:34.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02a7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f02a80d0090 0x7f02a80ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40020/0 (socket says 172.21.15.99:40020) 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.591+0000 7f02a7400700 1 -- 172.21.15.99:0/635054038 learned_addr learned my addr 172.21.15.99:0/635054038 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a6a00700 1 --2- 172.21.15.99:0/635054038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a7400700 1 -- 172.21.15.99:0/635054038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 msgr2=0x7f02a80cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a7400700 1 --2- 172.21.15.99:0/635054038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a7400700 1 -- 172.21.15.99:0/635054038 >> v1:172.21.15.133:6789/0 conn(0x7f02a80b1f50 legacy=0x7f02a80b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a7400700 1 -- 172.21.15.99:0/635054038 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8160f10 con 0x7f02a80d0090 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a7400700 1 --2- 172.21.15.99:0/635054038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80d0090 0x7f02a80ceda0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009e50 tx=0x7f02a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bb4ee92b880e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a5600700 1 -- 172.21.15.99:0/635054038 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000e070 con 0x7f02a80d0090 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a5600700 1 -- 172.21.15.99:0/635054038 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0003a20 con 0x7f02a80d0090 2024-03-29T09:10:34.593 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.592+0000 7f02a5600700 1 -- 172.21.15.99:0/635054038 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0003dd0 con 0x7f02a80d0090 2024-03-29T09:10:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 -- 172.21.15.99:0/635054038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80d0090 msgr2=0x7f02a80ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 --2- 172.21.15.99:0/635054038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80d0090 0x7f02a80ceda0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009e50 tx=0x7f02a000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 -- 172.21.15.99:0/635054038 shutdown_connections 2024-03-29T09:10:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 --2- 172.21.15.99:0/635054038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80d0090 0x7f02a80ceda0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 --2- 172.21.15.99:0/635054038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 -- 172.21.15.99:0/635054038 >> 172.21.15.99:0/635054038 conn(0x7f02a809c4d0 msgr2=0x7f02a80d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:34.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 -- 172.21.15.99:0/635054038 shutdown_connections 2024-03-29T09:10:34.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.593+0000 7f02ade00700 1 -- 172.21.15.99:0/635054038 wait complete. 2024-03-29T09:10:34.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.595+0000 7f02ade00700 1 Processor -- start 2024-03-29T09:10:34.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.595+0000 7f02ade00700 1 -- start start 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02ade00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80b1f50 0x7f02a815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02ade00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02a80d0090 0x7f02a80b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02ade00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02ade00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80b6b00 con 0x7f02a80b1f50 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02ade00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80b5e80 con 0x7f02a80d0090 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02ade00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80b6160 con 0x7f02a8160b30 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80b1f50 0x7f02a815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80b1f50 0x7f02a815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40028/0 (socket says 172.21.15.99:40028) 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 -- 172.21.15.99:0/368030780 learned_addr learned my addr 172.21.15.99:0/368030780 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:34.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6000700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02a80d0090 0x7f02a80b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a7400700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 -- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 msgr2=0x7f02a80b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 -- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02a80d0090 msgr2=0x7f02a80b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02a80d0090 0x7f02a80b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.597+0000 7f02a6a00700 1 -- 172.21.15.99:0/368030780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a0009b30 con 0x7f02a80b1f50 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.598+0000 7f02a6a00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80b1f50 0x7f02a815b780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f029800d800 tx=0x7f029800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.598+0000 7f029f400700 1 -- 172.21.15.99:0/368030780 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02980125f0 con 0x7f02a80b1f50 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.598+0000 7f029f400700 1 -- 172.21.15.99:0/368030780 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0298012750 con 0x7f02a80b1f50 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.598+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a80b6440 con 0x7f02a80b1f50 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.598+0000 7f029f400700 1 -- 172.21.15.99:0/368030780 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298012b00 con 0x7f02a80b1f50 2024-03-29T09:10:34.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.598+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a815bcc0 con 0x7f02a80b1f50 2024-03-29T09:10:34.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.599+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a80b2970 con 0x7f02a80b1f50 2024-03-29T09:10:34.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.600+0000 7f029f400700 1 -- 172.21.15.99:0/368030780 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0298012c60 con 0x7f02a80b1f50 2024-03-29T09:10:34.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.601+0000 7f029f400700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f028806ac90 0x7f028806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:34.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.601+0000 7f029f400700 1 -- 172.21.15.99:0/368030780 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5752+0+0 (secure 0 0 0) 0x7f02980896e0 con 0x7f02a80b1f50 2024-03-29T09:10:34.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.601+0000 7f02a6000700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f028806ac90 0x7f028806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:34.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.602+0000 7f02a6000700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f028806ac90 0x7f028806d150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f02a80a02b0 tx=0x7f0294009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:34.607 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.607+0000 7f029f400700 1 -- 172.21.15.99:0/368030780 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0298054890 con 0x7f02a80b1f50 2024-03-29T09:10:34.811 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:34.811+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f02a80678b0 con 0x7f02a80b1f50 2024-03-29T09:10:35.379 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.379+0000 7f029f400700 1 -- 172.21.15.99:0/368030780 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0298057eb0 con 0x7f02a80b1f50 2024-03-29T09:10:35.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f028806ac90 msgr2=0x7f028806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:35.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f028806ac90 0x7f028806d150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f02a80a02b0 tx=0x7f0294009040 comp rx=0 tx=0).stop 2024-03-29T09:10:35.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80b1f50 msgr2=0x7f02a815b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:35.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80b1f50 0x7f02a815b780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f029800d800 tx=0x7f029800dbc0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 shutdown_connections 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f028806ac90 0x7f028806d150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02a8160b30 0x7f02a80b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f02a80b1f50 0x7f02a815b780 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 --2- 172.21.15.99:0/368030780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02a80d0090 0x7f02a80b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 >> 172.21.15.99:0/368030780 conn(0x7f02a809c4d0 msgr2=0x7f02a80b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 shutdown_connections 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.382+0000 7f02ade00700 1 -- 172.21.15.99:0/368030780 wait complete. 2024-03-29T09:10:35.383 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data_ec' removed 2024-03-29T09:10:35.402 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T09:10:35.402 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T09:10:35.402 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T09:10:35.402 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:35.402 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T09:10:35.403 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T09:10:35.403 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T09:10:35.403 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:35.403 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T09:10:35.403 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T09:10:35.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T09:10:35.404 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:35.404 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T09:10:35.404 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T09:10:35.404 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T09:10:35.404 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:10:35.419 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T09:10:35.419 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T09:10:35.420 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T09:10:35.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T09:10:35.421 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T09:10:35.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T09:10:35.422 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T09:10:35.422 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T09:10:35.464 INFO:tasks.ceph.mds.a:Started 2024-03-29T09:10:35.464 DEBUG:teuthology.parallel:result is None 2024-03-29T09:10:35.466 INFO:tasks.ceph.mds.b:Started 2024-03-29T09:10:35.466 DEBUG:teuthology.parallel:result is None 2024-03-29T09:10:35.467 INFO:tasks.ceph.mds.c:Started 2024-03-29T09:10:35.467 DEBUG:teuthology.parallel:result is None 2024-03-29T09:10:35.468 INFO:tasks.ceph.mds.d:Started 2024-03-29T09:10:35.469 DEBUG:teuthology.parallel:result is None 2024-03-29T09:10:35.469 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T09:10:35.610 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-03-29T09:10:35.611 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-03-29T09:10:35.612 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-03-29T09:10:35.614 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-03-29T09:10:35.624 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-03-29T09:10:35.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.680+0000 7f40a2200700 1 Processor -- start 2024-03-29T09:10:35.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.680+0000 7f40a2200700 1 -- start start 2024-03-29T09:10:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.681+0000 7f40a2200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f409c160cd0 0x7f409c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.681+0000 7f40a2200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f409c0b3640 0x7f409c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.681+0000 7f40a2200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c15efb0 con 0x7f409c099f80 2024-03-29T09:10:35.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.681+0000 7f40a2200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c09a3b0 con 0x7f409c0b3640 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.681+0000 7f40a2200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c06bf50 con 0x7f409c160cd0 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.681+0000 7f409aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f409c0b3640 0x7f409c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f409c0b3640 0x7f409c099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40056/0 (socket says 172.21.15.99:40056) 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409aa00700 1 -- 172.21.15.99:0/445019474 learned_addr learned my addr 172.21.15.99:0/445019474 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409b400700 1 --2- 172.21.15.99:0/445019474 >> v2:172.21.15.133:3300/0 conn(0x7f409c160cd0 0x7f409c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409aa00700 1 -- 172.21.15.99:0/445019474 >> v2:172.21.15.133:3300/0 conn(0x7f409c160cd0 msgr2=0x7f409c15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409aa00700 1 --2- 172.21.15.99:0/445019474 >> v2:172.21.15.133:3300/0 conn(0x7f409c160cd0 0x7f409c15dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409aa00700 1 -- 172.21.15.99:0/445019474 >> v1:172.21.15.133:6789/0 conn(0x7f409c099f80 legacy=0x7f409c161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409aa00700 1 -- 172.21.15.99:0/445019474 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c1610b0 con 0x7f409c0b3640 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409aa00700 1 --2- 172.21.15.99:0/445019474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0b3640 0x7f409c099a40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f409c071f30 tx=0x7f4088009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c526edaae8e24d80 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.682+0000 7f409a000700 1 -- 172.21.15.99:0/445019474 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088011830 con 0x7f409c0b3640 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f409a000700 1 -- 172.21.15.99:0/445019474 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4088011990 con 0x7f409c0b3640 2024-03-29T09:10:35.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f409a000700 1 -- 172.21.15.99:0/445019474 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088011d20 con 0x7f409c0b3640 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 -- 172.21.15.99:0/445019474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0b3640 msgr2=0x7f409c099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 --2- 172.21.15.99:0/445019474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0b3640 0x7f409c099a40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f409c071f30 tx=0x7f4088009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 -- 172.21.15.99:0/445019474 shutdown_connections 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 --2- 172.21.15.99:0/445019474 >> v2:172.21.15.133:3300/0 conn(0x7f409c160cd0 0x7f409c15dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 --2- 172.21.15.99:0/445019474 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c0b3640 0x7f409c099a40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 -- 172.21.15.99:0/445019474 >> 172.21.15.99:0/445019474 conn(0x7f409c0b2b20 msgr2=0x7f409c0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 -- 172.21.15.99:0/445019474 shutdown_connections 2024-03-29T09:10:35.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.683+0000 7f40a2200700 1 -- 172.21.15.99:0/445019474 wait complete. 2024-03-29T09:10:35.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.685+0000 7f40a2200700 1 Processor -- start 2024-03-29T09:10:35.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.685+0000 7f40a2200700 1 -- start start 2024-03-29T09:10:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f40a2200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f409c099f80 0x7f409c150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f40a2200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f409c0b3640 0x7f409c0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:35.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f40a2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c160cd0 0x7f409c0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f40a2200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c150990 con 0x7f409c160cd0 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f40a2200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b1700 con 0x7f409c0b3640 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f40a2200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c0b0af0 con 0x7f409c099f80 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f409be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c160cd0 0x7f409c0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f409b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f409c099f80 0x7f409c150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f409aa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f409c0b3640 0x7f409c0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f409be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c160cd0 0x7f409c0c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40064/0 (socket says 172.21.15.99:40064) 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.686+0000 7f409be00700 1 -- 172.21.15.99:0/3271956706 learned_addr learned my addr 172.21.15.99:0/3271956706 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f409be00700 1 -- 172.21.15.99:0/3271956706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f409c099f80 msgr2=0x7f409c150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f409be00700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f409c099f80 0x7f409c150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f409be00700 1 -- 172.21.15.99:0/3271956706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f409c0b3640 msgr2=0x7f409c0aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f409be00700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f409c0b3640 0x7f409c0aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:35.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f409be00700 1 -- 172.21.15.99:0/3271956706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088009550 con 0x7f409c160cd0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f409b400700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f409c099f80 0x7f409c150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f409be00700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c160cd0 0x7f409c0c6c50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f409400cb50 tx=0x7f409400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f4093400700 1 -- 172.21.15.99:0/3271956706 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094003c00 con 0x7f409c160cd0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f4093400700 1 -- 172.21.15.99:0/3271956706 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094003d60 con 0x7f409c160cd0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c0b0dd0 con 0x7f409c160cd0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f4093400700 1 -- 172.21.15.99:0/3271956706 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400aa20 con 0x7f409c160cd0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.687+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c0b1390 con 0x7f409c160cd0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.688+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c0677a0 con 0x7f409c160cd0 2024-03-29T09:10:35.689 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.689+0000 7f4093400700 1 -- 172.21.15.99:0/3271956706 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f409400ab80 con 0x7f409c160cd0 2024-03-29T09:10:35.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.690+0000 7f4093400700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f407c06ac90 0x7f407c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:35.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.690+0000 7f4093400700 1 -- 172.21.15.99:0/3271956706 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4094088ff0 con 0x7f409c160cd0 2024-03-29T09:10:35.690 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.690+0000 7f409b400700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f407c06ac90 0x7f407c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:35.691 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.691+0000 7f409b400700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f407c06ac90 0x7f407c06d150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f408c009fd0 tx=0x7f408c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:35.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.695+0000 7f4093400700 1 -- 172.21.15.99:0/3271956706 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4094054380 con 0x7f409c160cd0 2024-03-29T09:10:35.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:35.915+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f409c15efb0 con 0x7f409c160cd0 2024-03-29T09:10:36.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.656+0000 7f4093400700 1 -- 172.21.15.99:0/3271956706 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f409c15efb0 con 0x7f409c160cd0 2024-03-29T09:10:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f407c06ac90 msgr2=0x7f407c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f407c06ac90 0x7f407c06d150 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f408c009fd0 tx=0x7f408c009210 comp rx=0 tx=0).stop 2024-03-29T09:10:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c160cd0 msgr2=0x7f409c0c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c160cd0 0x7f409c0c6c50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f409400cb50 tx=0x7f409400ce60 comp rx=0 tx=0).stop 2024-03-29T09:10:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 shutdown_connections 2024-03-29T09:10:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f407c06ac90 0x7f407c06d150 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f409c160cd0 0x7f409c0c6c50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f409c099f80 0x7f409c150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 --2- 172.21.15.99:0/3271956706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f409c0b3640 0x7f409c0aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 >> 172.21.15.99:0/3271956706 conn(0x7f409c0b2b20 msgr2=0x7f409c0b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 shutdown_connections 2024-03-29T09:10:36.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.659+0000 7f40a2200700 1 -- 172.21.15.99:0/3271956706 wait complete. 2024-03-29T09:10:36.660 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-03-29T09:10:36.679 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T09:10:36.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.871+0000 7fe2e8000700 1 Processor -- start 2024-03-29T09:10:36.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.871+0000 7fe2e8000700 1 -- start start 2024-03-29T09:10:36.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.872+0000 7fe2e8000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 0x7fe2e00d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.872+0000 7fe2e8000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00cdf80 con 0x7fe2e00cfad0 2024-03-29T09:10:36.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.872+0000 7fe2e8000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e015eb80 con 0x7fe2e00d12f0 2024-03-29T09:10:36.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.872+0000 7fe2e8000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00d2140 con 0x7fe2e015e7a0 2024-03-29T09:10:36.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.872+0000 7fe2e6000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe2e00cfad0 0x7fe2e015efd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54086/0 (socket says 172.21.15.99:54086) 2024-03-29T09:10:36.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.872+0000 7fe2e6000700 1 -- 172.21.15.99:0/1559867402 learned_addr learned my addr 172.21.15.99:0/1559867402 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:36.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2e5600700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 0x7fe2e00d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457898421 0 0) 0x7fe2e00cdf80 con 0x7fe2e00cfad0 2024-03-29T09:10:36.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c80035b0 con 0x7fe2e00cfad0 2024-03-29T09:10:36.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391168363 0 0) 0x7fe2c80035b0 con 0x7fe2e00cfad0 2024-03-29T09:10:36.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 msgr2=0x7fe2e00d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:36.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 0x7fe2e00d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 >> v1:172.21.15.133:6789/0 conn(0x7fe2e00d12f0 legacy=0x7fe2e00cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00ce1f0 con 0x7fe2e00cfad0 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fe2d80019f0 con 0x7fe2e00cfad0 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 >> v1:172.21.15.99:6789/0 conn(0x7fe2e00cfad0 legacy=0x7fe2e015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 0x7fe2c8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2c8003f70 0x7fe2c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c8004d90 0x7fe2c8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00d2140 con 0x7fe2c8003f70 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c800ba70 con 0x7fe2c8004d90 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.873+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c800bc30 con 0x7fe2e015e7a0 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e4c00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2c8003f70 0x7fe2c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e5600700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 0x7fe2c8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e6000700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c8004d90 0x7fe2c8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e4c00700 1 -- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 msgr2=0x7fe2c8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e4c00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2e015e7a0 0x7fe2c8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e4c00700 1 -- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c8004d90 msgr2=0x7fe2c8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e4c00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c8004d90 0x7fe2c8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e4c00700 1 -- 172.21.15.99:0/1559867402 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c80035b0 con 0x7fe2c8003f70 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.874+0000 7fe2e4c00700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2c8003f70 0x7fe2c8004850 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc001980 tx=0x7fe2c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9b18b7cc7da9fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc016070 con 0x7fe2c8003f70 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc001cf0 con 0x7fe2c8003f70 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2dfe00700 1 -- 172.21.15.99:0/1559867402 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc001420 con 0x7fe2c8003f70 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2e8000700 1 -- 172.21.15.99:0/1559867402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2c8003f70 msgr2=0x7fe2c8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2e8000700 1 --2- 172.21.15.99:0/1559867402 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2c8003f70 0x7fe2c8004850 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc001980 tx=0x7fe2c8005880 comp rx=0 tx=0).stop 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2e6000700 1 -- 172.21.15.99:0/1559867402 reap_dead start 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2e8000700 1 -- 172.21.15.99:0/1559867402 shutdown_connections 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2e8000700 1 -- 172.21.15.99:0/1559867402 >> 172.21.15.99:0/1559867402 conn(0x7fe2e009c480 msgr2=0x7fe2e00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2e8000700 1 -- 172.21.15.99:0/1559867402 shutdown_connections 2024-03-29T09:10:36.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.875+0000 7fe2e8000700 1 -- 172.21.15.99:0/1559867402 wait complete. 2024-03-29T09:10:36.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.877+0000 7fe2e8000700 1 Processor -- start 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.877+0000 7fe2e8000700 1 -- start start 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e8000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c8003f70 0x7fe2e015c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e8000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2e00a0030 0x7fe2e015b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e8000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2e00a04b0 0x7fe2e015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e8000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00b5d00 con 0x7fe2e00a0030 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e8000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00b62c0 con 0x7fe2e00a04b0 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e8000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00a0a40 con 0x7fe2c8003f70 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e4c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2e00a0030 0x7fe2e015b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e4c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2e00a0030 0x7fe2e015b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40082/0 (socket says 172.21.15.99:40082) 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e4c00700 1 -- 172.21.15.99:0/383300744 learned_addr learned my addr 172.21.15.99:0/383300744 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e5600700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c8003f70 0x7fe2e015c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.878+0000 7fe2e6000700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2e00a04b0 0x7fe2e015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e4c00700 1 -- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c8003f70 msgr2=0x7fe2e015c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e4c00700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c8003f70 0x7fe2e015c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e4c00700 1 -- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2e00a04b0 msgr2=0x7fe2e015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e4c00700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2e00a04b0 0x7fe2e015ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e4c00700 1 -- 172.21.15.99:0/383300744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00a0d20 con 0x7fe2e00a0030 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e4c00700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2e00a0030 0x7fe2e015b4e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c3c0 tx=0x7fe2cc0012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2dd600700 1 -- 172.21.15.99:0/383300744 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc016040 con 0x7fe2e00a0030 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2dd600700 1 -- 172.21.15.99:0/383300744 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc0054e0 con 0x7fe2e00a0030 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2dd600700 1 -- 172.21.15.99:0/383300744 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc007260 con 0x7fe2e00a0030 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00b5d00 con 0x7fe2e00a0030 2024-03-29T09:10:36.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.879+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e015cd90 con 0x7fe2e00a0030 2024-03-29T09:10:36.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.880+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e00b6470 con 0x7fe2e00a0030 2024-03-29T09:10:36.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.881+0000 7fe2dd600700 1 -- 172.21.15.99:0/383300744 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe2cc0073c0 con 0x7fe2e00a0030 2024-03-29T09:10:36.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.882+0000 7fe2dd600700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2c006ac90 0x7fe2c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:36.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.882+0000 7fe2dd600700 1 -- 172.21.15.99:0/383300744 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe2cc089e80 con 0x7fe2e00a0030 2024-03-29T09:10:36.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.882+0000 7fe2e5600700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2c006ac90 0x7fe2c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:36.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.883+0000 7fe2e5600700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2c006ac90 0x7fe2c006d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009c60 tx=0x7fe2d4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:36.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:36.886+0000 7fe2dd600700 1 -- 172.21.15.99:0/383300744 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe2e00b6470 con 0x7fe2e00a0030 2024-03-29T09:10:37.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.167+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2e00b6430 con 0x7fe2e00a0030 2024-03-29T09:10:37.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.167+0000 7fe2dd600700 1 -- 172.21.15.99:0/383300744 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe2cc0552a0 con 0x7fe2e00a0030 2024-03-29T09:10:37.167 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZmrFiLOxAAKlHfn8PZHThkmZP7F+CTSA==", 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmQJsgFhAAXgodL3u1/qBUxO/ZydPMhQ==", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmfXkACxAAcCfih1bN4d00TqYlKIWjLg==", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.169 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmKnAzIRAA2psjQhAebm0ZQ+d833RbsA==", 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.170 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDwhAZmLMALHRAARJg5Ykd30R69SyDTaCBIFw==", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDzhAZmuL5pIBAA6jXsRZ/S6bp2Clf6SAQamg==", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQD2hAZmTOqEIxAAbl19sfN+WekZStEZ7F3WFQ==", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQD5hAZmjSHvJhAAWVvxpRqaTu+An+sB6KW0QA==", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.172 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQD/hAZm59SYOxAA6ZlyViyEh4T2LzLugojRlw==", 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.173 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQADhQZmX0pCAxAAJ3tggpnioWiMCdGoDitpwg==", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQAGhQZmA85NBhAAeK3A7HRf5p03ImHioYPkxw==", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQAJhQZmt9t7CRAAikERd7yLpKlPADqOxYTe4Q==", 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZm8QvmKhAAtjcMwjNBLPSYOuxJAXZEnw==", 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.175 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmKIVfORAAYXf8+I1h3ctIq9tQRkNJMQ==", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmvXccMhAALKLyc3Cfc3yjx3671P2yNw==", 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDthAZmlzsGBRAA3lYwuTrdEqSvk/f3Zx4NSA==", 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.177 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZm0EgVBxAA/dalJuZ4gfwlxIjb36vqZQ==", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.178 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-mds", 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQARhQZmb2pgBBAA2a6O0rnuRYDSPY9udpHHfg==", 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-mds" 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-mgr", 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQARhQZm0XtgBBAAkmXjd4GdkNnwavY2V7taRw==", 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-29T09:10:37.179 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-osd", 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQARhQZm74pgBBAAqAnX8FixXFCsly/ZUyygRg==", 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-osd" 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rbd", 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQARhQZm1JtgBBAAEVE1sbMhOEnrPOizdOHNIw==", 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQARhQZmcKhgBBAAZzszdeid8ynJvnETh9CepA==", 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.bootstrap-rgw", 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQARhQZmb7FgBBAAkoDVsez3Q5KtU/WU3gEpmg==", 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-03-29T09:10:37.181 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZm/LczNBAAakX4jUFzQMJdJNA14crh/A==", 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZmLEdRLBAAnVR8ae1uZ/ezeRT77pc7dw==", 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:37.182 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-29T09:10:37.183 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:10:37.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2c006ac90 msgr2=0x7fe2c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2c006ac90 0x7fe2c006d150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009c60 tx=0x7fe2d4009210 comp rx=0 tx=0).stop 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2e00a0030 msgr2=0x7fe2e015b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2e00a0030 0x7fe2e015b4e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c3c0 tx=0x7fe2cc0012b0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 shutdown_connections 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2c006ac90 0x7fe2c006d150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2e00a04b0 0x7fe2e015ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c8003f70 0x7fe2e015c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 --2- 172.21.15.99:0/383300744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2e00a0030 0x7fe2e015b4e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 >> 172.21.15.99:0/383300744 conn(0x7fe2e009c480 msgr2=0x7fe2e00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 shutdown_connections 2024-03-29T09:10:37.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.170+0000 7fe2e8000700 1 -- 172.21.15.99:0/383300744 wait complete. 2024-03-29T09:10:37.187 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-29T09:10:37.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.377+0000 7f9f10000700 1 Processor -- start 2024-03-29T09:10:37.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.377+0000 7f9f10000700 1 -- start start 2024-03-29T09:10:37.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.378+0000 7f9f10000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f080d1b20 0x7f9f080d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.378+0000 7f9f10000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9f08160cc0 0x7f9f080ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.378+0000 7f9f10000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f080cddc0 con 0x7f9f081608e0 2024-03-29T09:10:37.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f10000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f080ce0a0 con 0x7f9f080d1b20 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f10000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f080ce380 con 0x7f9f08160cc0 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0cc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f080d1b20 0x7f9f080d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0cc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9f080d1b20 0x7f9f080d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40094/0 (socket says 172.21.15.99:40094) 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0d600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9f081608e0 0x7f9f080d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:54094/0 (socket says 172.21.15.99:54094) 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0cc00700 1 -- 172.21.15.99:0/28208944 learned_addr learned my addr 172.21.15.99:0/28208944 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0e000700 1 --2- 172.21.15.99:0/28208944 >> v2:172.21.15.133:3300/0 conn(0x7f9f08160cc0 0x7f9f080ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0cc00700 1 -- 172.21.15.99:0/28208944 >> v2:172.21.15.133:3300/0 conn(0x7f9f08160cc0 msgr2=0x7f9f080ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0cc00700 1 --2- 172.21.15.99:0/28208944 >> v2:172.21.15.133:3300/0 conn(0x7f9f08160cc0 0x7f9f080ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0cc00700 1 -- 172.21.15.99:0/28208944 >> v1:172.21.15.99:6789/0 conn(0x7f9f081608e0 legacy=0x7f9f080d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f0cc00700 1 -- 172.21.15.99:0/28208944 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f080ce660 con 0x7f9f080d1b20 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.379+0000 7f9f07e00700 1 -- 172.21.15.99:0/28208944 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807711875 0 0) 0x7f9f080cddc0 con 0x7f9f081608e0 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f0cc00700 1 --2- 172.21.15.99:0/28208944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f080d1b20 0x7f9f080d0150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9f080b91e0 tx=0x7f9ef4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f630a2a78d4e8995 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f07e00700 1 -- 172.21.15.99:0/28208944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef400b870 con 0x7f9f080d1b20 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f07e00700 1 -- 172.21.15.99:0/28208944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef400b9d0 con 0x7f9f080d1b20 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f07e00700 1 -- 172.21.15.99:0/28208944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef400bd60 con 0x7f9f080d1b20 2024-03-29T09:10:37.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 -- 172.21.15.99:0/28208944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f080d1b20 msgr2=0x7f9f080d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 --2- 172.21.15.99:0/28208944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f080d1b20 0x7f9f080d0150 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9f080b91e0 tx=0x7f9ef4009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 -- 172.21.15.99:0/28208944 shutdown_connections 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 --2- 172.21.15.99:0/28208944 >> v2:172.21.15.133:3300/0 conn(0x7f9f08160cc0 0x7f9f080ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 --2- 172.21.15.99:0/28208944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f080d1b20 0x7f9f080d0150 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 -- 172.21.15.99:0/28208944 >> 172.21.15.99:0/28208944 conn(0x7f9f0809c490 msgr2=0x7f9f080d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 -- 172.21.15.99:0/28208944 shutdown_connections 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.380+0000 7f9f10000700 1 -- 172.21.15.99:0/28208944 wait complete. 2024-03-29T09:10:37.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.382+0000 7f9f10000700 1 Processor -- start 2024-03-29T09:10:37.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.383+0000 7f9f10000700 1 -- start start 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.383+0000 7f9f10000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f080d1b20 0x7f9f081596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.383+0000 7f9f10000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f081608e0 0x7f9f080ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f10000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f08160cc0 0x7f9f080b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f10000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08159bf0 con 0x7f9f08160cc0 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f10000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08158640 con 0x7f9f081608e0 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f10000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08156cf0 con 0x7f9f080d1b20 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f08160cc0 0x7f9f080b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0d600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f080d1b20 0x7f9f081596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f08160cc0 0x7f9f080b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40102/0 (socket says 172.21.15.99:40102) 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 -- 172.21.15.99:0/1473753003 learned_addr learned my addr 172.21.15.99:0/1473753003 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0cc00700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f081608e0 0x7f9f080ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 -- 172.21.15.99:0/1473753003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f080d1b20 msgr2=0x7f9f081596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f080d1b20 0x7f9f081596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 -- 172.21.15.99:0/1473753003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f081608e0 msgr2=0x7f9f080ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f081608e0 0x7f9f080ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.384+0000 7f9f0e000700 1 -- 172.21.15.99:0/1473753003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009550 con 0x7f9f08160cc0 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.385+0000 7f9f0e000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f08160cc0 0x7f9f080b0dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9f00007e60 tx=0x7f9f0000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.385+0000 7f9f05600700 1 -- 172.21.15.99:0/1473753003 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0000ed30 con 0x7f9f08160cc0 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.385+0000 7f9f05600700 1 -- 172.21.15.99:0/1473753003 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f00014420 con 0x7f9f08160cc0 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.385+0000 7f9f05600700 1 -- 172.21.15.99:0/1473753003 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000147d0 con 0x7f9f08160cc0 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.385+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f080cd510 con 0x7f9f08160cc0 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.385+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f08157d10 con 0x7f9f08160cc0 2024-03-29T09:10:37.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.386+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f080677a0 con 0x7f9f08160cc0 2024-03-29T09:10:37.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.388+0000 7f9f05600700 1 -- 172.21.15.99:0/1473753003 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9f00014930 con 0x7f9f08160cc0 2024-03-29T09:10:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.388+0000 7f9f05600700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee806ac90 0x7f9ee806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.389+0000 7f9f05600700 1 -- 172.21.15.99:0/1473753003 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9f000896d0 con 0x7f9f08160cc0 2024-03-29T09:10:37.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.389+0000 7f9f0d600700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee806ac90 0x7f9ee806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.389+0000 7f9f0d600700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee806ac90 0x7f9ee806d150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9efc007fd0 tx=0x7f9efc007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:37.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.394+0000 7f9f05600700 1 -- 172.21.15.99:0/1473753003 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9f00054ac0 con 0x7f9f08160cc0 2024-03-29T09:10:37.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.678+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f9f080d0700 con 0x7f9f08160cc0 2024-03-29T09:10:37.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.682+0000 7f9f05600700 1 -- 172.21.15.99:0/1473753003 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9f000580e0 con 0x7f9f08160cc0 2024-03-29T09:10:37.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee806ac90 msgr2=0x7f9ee806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee806ac90 0x7f9ee806d150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9efc007fd0 tx=0x7f9efc007de0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f08160cc0 msgr2=0x7f9f080b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f08160cc0 0x7f9f080b0dc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9f00007e60 tx=0x7f9f0000e3c0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 shutdown_connections 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ee806ac90 0x7f9ee806d150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f08160cc0 0x7f9f080b0dc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f080d1b20 0x7f9f081596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 --2- 172.21.15.99:0/1473753003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f081608e0 0x7f9f080ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 >> 172.21.15.99:0/1473753003 conn(0x7f9f0809c490 msgr2=0x7f9f0815c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 shutdown_connections 2024-03-29T09:10:37.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.684+0000 7f9f10000700 1 -- 172.21.15.99:0/1473753003 wait complete. 2024-03-29T09:10:37.702 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-29T09:10:37.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.891+0000 7fd866400700 1 Processor -- start 2024-03-29T09:10:37.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.892+0000 7fd866400700 1 -- start start 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd866400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 0x7fd8600d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd866400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd860160cc0 0x7fd8600ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd866400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8600cddc0 con 0x7fd8601608e0 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd866400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8600ce0a0 con 0x7fd860160cc0 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd866400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600ce380 con 0x7fd8600d1b20 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd85f400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd8601608e0 0x7fd8600d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47912/0 (socket says 172.21.15.99:47912) 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd85f400700 1 -- 172.21.15.99:0/119741514 learned_addr learned my addr 172.21.15.99:0/119741514 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd85ea00700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 0x7fd8600d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd85fe00700 1 --2- 172.21.15.99:0/119741514 >> v2:172.21.15.133:3300/0 conn(0x7fd860160cc0 0x7fd8600ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.893+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557356975 0 0) 0x7fd8600cddc0 con 0x7fd8601608e0 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8440035b0 con 0x7fd8601608e0 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 594337935 0 0) 0x7fd8440035b0 con 0x7fd8601608e0 2024-03-29T09:10:37.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 msgr2=0x7fd8600d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 0x7fd8600d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 >> v2:172.21.15.133:3300/0 conn(0x7fd860160cc0 msgr2=0x7fd8600ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 --2- 172.21.15.99:0/119741514 >> v2:172.21.15.133:3300/0 conn(0x7fd860160cc0 0x7fd8600ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600ce660 con 0x7fd8601608e0 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fd850001670 con 0x7fd8601608e0 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85ea00700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 0x7fd8600d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 >> v1:172.21.15.99:6789/0 conn(0x7fd8601608e0 legacy=0x7fd8600d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd844003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd844004850 0x7fd844004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 0x7fd844005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600ce0a0 con 0x7fd844003a00 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600ce380 con 0x7fd844004850 2024-03-29T09:10:37.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84400ba10 con 0x7fd8600d1b20 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85f400700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd844003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85ea00700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd844004850 0x7fd844004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.894+0000 7fd85fe00700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 0x7fd844005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85f400700 1 -- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 msgr2=0x7fd844005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85f400700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600d1b20 0x7fd844005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85f400700 1 -- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd844004850 msgr2=0x7fd844004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85f400700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd844004850 0x7fd844004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85f400700 1 -- 172.21.15.99:0/119741514 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0094a0 con 0x7fd844003a00 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85f400700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd844003e20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c320 tx=0x7fd844005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772b9a7ac2a95b0a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000ce00 con 0x7fd844003a00 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850005960 con 0x7fd844003a00 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.895+0000 7fd85e000700 1 -- 172.21.15.99:0/119741514 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850005d10 con 0x7fd844003a00 2024-03-29T09:10:37.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.896+0000 7fd866400700 1 -- 172.21.15.99:0/119741514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 msgr2=0x7fd844003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.896+0000 7fd866400700 1 --2- 172.21.15.99:0/119741514 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd844003e20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c320 tx=0x7fd844005880 comp rx=0 tx=0).stop 2024-03-29T09:10:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.896+0000 7fd85fe00700 1 -- 172.21.15.99:0/119741514 reap_dead start 2024-03-29T09:10:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.896+0000 7fd866400700 1 -- 172.21.15.99:0/119741514 shutdown_connections 2024-03-29T09:10:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.896+0000 7fd866400700 1 -- 172.21.15.99:0/119741514 >> 172.21.15.99:0/119741514 conn(0x7fd86009c490 msgr2=0x7fd8600d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.896+0000 7fd866400700 1 -- 172.21.15.99:0/119741514 shutdown_connections 2024-03-29T09:10:37.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.896+0000 7fd866400700 1 -- 172.21.15.99:0/119741514 wait complete. 2024-03-29T09:10:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.898+0000 7fd866400700 1 Processor -- start 2024-03-29T09:10:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.898+0000 7fd866400700 1 -- start start 2024-03-29T09:10:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd866400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd8600cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd866400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600ccb10 0x7fd8600ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd866400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd86014a5d0 0x7fd86014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd866400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860149250 con 0x7fd844003a00 2024-03-29T09:10:37.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd866400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860149530 con 0x7fd86014a5d0 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd866400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd860149810 con 0x7fd8600ccb10 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd8600cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd8600cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35636/0 (socket says 172.21.15.99:35636) 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 -- 172.21.15.99:0/999961122 learned_addr learned my addr 172.21.15.99:0/999961122 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85ea00700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600ccb10 0x7fd8600ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 -- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600ccb10 msgr2=0x7fd8600ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85fe00700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd86014a5d0 0x7fd86014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600ccb10 0x7fd8600ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 -- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd86014a5d0 msgr2=0x7fd86014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd86014a5d0 0x7fd86014aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85f400700 1 -- 172.21.15.99:0/999961122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860149af0 con 0x7fd844003a00 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.899+0000 7fd85ea00700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600ccb10 0x7fd8600ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.900+0000 7fd85f400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd8600cc5d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd850008fd0 tx=0x7fd850001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:37.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.900+0000 7fd857400700 1 -- 172.21.15.99:0/999961122 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000ce70 con 0x7fd844003a00 2024-03-29T09:10:37.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.900+0000 7fd857400700 1 -- 172.21.15.99:0/999961122 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850004a80 con 0x7fd844003a00 2024-03-29T09:10:37.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.900+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd860149dd0 con 0x7fd844003a00 2024-03-29T09:10:37.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.900+0000 7fd857400700 1 -- 172.21.15.99:0/999961122 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850004e10 con 0x7fd844003a00 2024-03-29T09:10:37.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.900+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd860149250 con 0x7fd844003a00 2024-03-29T09:10:37.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.901+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86014a420 con 0x7fd844003a00 2024-03-29T09:10:37.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.902+0000 7fd857400700 1 -- 172.21.15.99:0/999961122 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd8500070d0 con 0x7fd844003a00 2024-03-29T09:10:37.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.903+0000 7fd857400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd84006ac90 0x7fd84006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:37.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.903+0000 7fd857400700 1 -- 172.21.15.99:0/999961122 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd85008a550 con 0x7fd844003a00 2024-03-29T09:10:37.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.903+0000 7fd85ea00700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd84006ac90 0x7fd84006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:37.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.904+0000 7fd85ea00700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd84006ac90 0x7fd84006d150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006ba0 tx=0x7fd84c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:37.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:37.908+0000 7fd857400700 1 -- 172.21.15.99:0/999961122 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd850055a90 con 0x7fd844003a00 2024-03-29T09:10:38.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.192+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd8600677a0 con 0x7fd844003a00 2024-03-29T09:10:38.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.198+0000 7fd857400700 1 -- 172.21.15.99:0/999961122 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd850007da0 con 0x7fd844003a00 2024-03-29T09:10:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd84006ac90 msgr2=0x7fd84006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd84006ac90 0x7fd84006d150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006ba0 tx=0x7fd84c008040 comp rx=0 tx=0).stop 2024-03-29T09:10:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 msgr2=0x7fd8600cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd8600cc5d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd850008fd0 tx=0x7fd850001a00 comp rx=0 tx=0).stop 2024-03-29T09:10:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 shutdown_connections 2024-03-29T09:10:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd84006ac90 0x7fd84006d150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd86014a5d0 0x7fd86014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd844003a00 0x7fd8600cc5d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 --2- 172.21.15.99:0/999961122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8600ccb10 0x7fd8600ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 >> 172.21.15.99:0/999961122 conn(0x7fd86009c490 msgr2=0x7fd8600cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 shutdown_connections 2024-03-29T09:10:38.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.201+0000 7fd866400700 1 -- 172.21.15.99:0/999961122 wait complete. 2024-03-29T09:10:38.219 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-29T09:10:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.411+0000 7f9b30200700 1 Processor -- start 2024-03-29T09:10:38.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.411+0000 7f9b30200700 1 -- start start 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.412+0000 7f9b30200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b281608e0 0x7f9b280d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.412+0000 7f9b30200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b280d1b20 0x7f9b280ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.412+0000 7f9b30200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b280cddc0 con 0x7f9b28160cc0 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.412+0000 7f9b30200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b280ce0a0 con 0x7f9b281608e0 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.412+0000 7f9b30200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280ce380 con 0x7f9b280d1b20 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2ce00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9b28160cc0 0x7f9b280d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47926/0 (socket says 172.21.15.99:47926) 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2ce00700 1 -- 172.21.15.99:0/2374469264 learned_addr learned my addr 172.21.15.99:0/2374469264 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:38.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2d800700 1 --2- 172.21.15.99:0/2374469264 >> v2:172.21.15.133:3300/0 conn(0x7f9b281608e0 0x7f9b280d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2e200700 1 --2- 172.21.15.99:0/2374469264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b280d1b20 0x7f9b280ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b27e00700 1 -- 172.21.15.99:0/2374469264 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606738958 0 0) 0x7f9b280cddc0 con 0x7f9b28160cc0 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b27e00700 1 -- 172.21.15.99:0/2374469264 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b100035b0 con 0x7f9b28160cc0 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2e200700 1 -- 172.21.15.99:0/2374469264 >> v2:172.21.15.133:3300/0 conn(0x7f9b281608e0 msgr2=0x7f9b280d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2e200700 1 --2- 172.21.15.99:0/2374469264 >> v2:172.21.15.133:3300/0 conn(0x7f9b281608e0 0x7f9b280d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2e200700 1 -- 172.21.15.99:0/2374469264 >> v1:172.21.15.99:6789/0 conn(0x7f9b28160cc0 legacy=0x7f9b280d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2e200700 1 -- 172.21.15.99:0/2374469264 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b280ce660 con 0x7f9b280d1b20 2024-03-29T09:10:38.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.413+0000 7f9b2d800700 1 --2- 172.21.15.99:0/2374469264 >> v2:172.21.15.133:3300/0 conn(0x7f9b281608e0 0x7f9b280d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.414+0000 7f9b2e200700 1 --2- 172.21.15.99:0/2374469264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b280d1b20 0x7f9b280ceb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009d90 tx=0x7f9b2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a95fde5704831c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.414+0000 7f9b27e00700 1 -- 172.21.15.99:0/2374469264 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000e070 con 0x7f9b280d1b20 2024-03-29T09:10:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.414+0000 7f9b27e00700 1 -- 172.21.15.99:0/2374469264 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2000bb20 con 0x7f9b280d1b20 2024-03-29T09:10:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.414+0000 7f9b27e00700 1 -- 172.21.15.99:0/2374469264 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000be90 con 0x7f9b280d1b20 2024-03-29T09:10:38.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 -- 172.21.15.99:0/2374469264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b280d1b20 msgr2=0x7f9b280ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 --2- 172.21.15.99:0/2374469264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b280d1b20 0x7f9b280ceb80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009d90 tx=0x7f9b2000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 -- 172.21.15.99:0/2374469264 shutdown_connections 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 --2- 172.21.15.99:0/2374469264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b280d1b20 0x7f9b280ceb80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 --2- 172.21.15.99:0/2374469264 >> v2:172.21.15.133:3300/0 conn(0x7f9b281608e0 0x7f9b280d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 -- 172.21.15.99:0/2374469264 >> 172.21.15.99:0/2374469264 conn(0x7f9b2809c490 msgr2=0x7f9b280d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 -- 172.21.15.99:0/2374469264 shutdown_connections 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.415+0000 7f9b30200700 1 -- 172.21.15.99:0/2374469264 wait complete. 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.417+0000 7f9b30200700 1 Processor -- start 2024-03-29T09:10:38.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.417+0000 7f9b30200700 1 -- start start 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b30200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b280d1b20 0x7f9b28153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b30200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b281608e0 0x7f9b28156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b30200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b28160cc0 0x7f9b280b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b30200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280cb840 con 0x7f9b280d1b20 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b30200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b281f6950 con 0x7f9b28160cc0 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b30200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b281f6c30 con 0x7f9b281608e0 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b280d1b20 0x7f9b28153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b280d1b20 0x7f9b28153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35648/0 (socket says 172.21.15.99:35648) 2024-03-29T09:10:38.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 -- 172.21.15.99:0/1574731473 learned_addr learned my addr 172.21.15.99:0/1574731473 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2ce00700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b281608e0 0x7f9b28156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2e200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b28160cc0 0x7f9b280b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 -- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b281608e0 msgr2=0x7f9b28156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b281608e0 0x7f9b28156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 -- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b28160cc0 msgr2=0x7f9b280b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b28160cc0 0x7f9b280b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2d800700 1 -- 172.21.15.99:0/1574731473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0094a0 con 0x7f9b280d1b20 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2e200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b28160cc0 0x7f9b280b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.418+0000 7f9b2ce00700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b281608e0 0x7f9b28156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.419+0000 7f9b2d800700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b280d1b20 0x7f9b28153310 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00ebe0 tx=0x7f9b1c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.419+0000 7f9b25600700 1 -- 172.21.15.99:0/1574731473 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00c910 con 0x7f9b280d1b20 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.419+0000 7f9b25600700 1 -- 172.21.15.99:0/1574731473 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c00ca70 con 0x7f9b280d1b20 2024-03-29T09:10:38.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.419+0000 7f9b25600700 1 -- 172.21.15.99:0/1574731473 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c00cde0 con 0x7f9b280d1b20 2024-03-29T09:10:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.419+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b20009a70 con 0x7f9b280d1b20 2024-03-29T09:10:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.419+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b281f71f0 con 0x7f9b280d1b20 2024-03-29T09:10:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.420+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b280677a0 con 0x7f9b280d1b20 2024-03-29T09:10:38.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.421+0000 7f9b25600700 1 -- 172.21.15.99:0/1574731473 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9b1c013470 con 0x7f9b280d1b20 2024-03-29T09:10:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.422+0000 7f9b25600700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b0806ac90 0x7f9b0806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.422+0000 7f9b25600700 1 -- 172.21.15.99:0/1574731473 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9b1c0893a0 con 0x7f9b280d1b20 2024-03-29T09:10:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.422+0000 7f9b2ce00700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b0806ac90 0x7f9b0806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.423+0000 7f9b2ce00700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b0806ac90 0x7f9b0806d150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b280b4700 tx=0x7f9b14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:38.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.427+0000 7f9b25600700 1 -- 172.21.15.99:0/1574731473 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9b1c0546f0 con 0x7f9b280d1b20 2024-03-29T09:10:38.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.713+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9b2815ff80 con 0x7f9b280d1b20 2024-03-29T09:10:38.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.717+0000 7f9b25600700 1 -- 172.21.15.99:0/1574731473 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9b1c087e60 con 0x7f9b280d1b20 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b0806ac90 msgr2=0x7f9b0806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b0806ac90 0x7f9b0806d150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b280b4700 tx=0x7f9b14009040 comp rx=0 tx=0).stop 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b280d1b20 msgr2=0x7f9b28153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b280d1b20 0x7f9b28153310 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00ebe0 tx=0x7f9b1c009ec0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 shutdown_connections 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9b0806ac90 0x7f9b0806d150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b28160cc0 0x7f9b280b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b280d1b20 0x7f9b28153310 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 --2- 172.21.15.99:0/1574731473 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9b281608e0 0x7f9b28156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 >> 172.21.15.99:0/1574731473 conn(0x7f9b2809c490 msgr2=0x7f9b280d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 shutdown_connections 2024-03-29T09:10:38.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.720+0000 7f9b30200700 1 -- 172.21.15.99:0/1574731473 wait complete. 2024-03-29T09:10:38.738 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-29T09:10:38.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.927+0000 7f23e4200700 1 Processor -- start 2024-03-29T09:10:38.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.927+0000 7f23e4200700 1 -- start start 2024-03-29T09:10:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.928+0000 7f23e4200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc160cc0 0x7f23dc0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.928+0000 7f23e4200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0cddc0 con 0x7f23dc1608e0 2024-03-29T09:10:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.928+0000 7f23e4200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0ce0a0 con 0x7f23dc0d1b20 2024-03-29T09:10:38.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.928+0000 7f23e4200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc0ce380 con 0x7f23dc160cc0 2024-03-29T09:10:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.928+0000 7f23e1800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f23dc1608e0 0x7f23dc0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47934/0 (socket says 172.21.15.99:47934) 2024-03-29T09:10:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23e1800700 1 -- 172.21.15.99:0/1439955479 learned_addr learned my addr 172.21.15.99:0/1439955479 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:38.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23e2200700 1 --2- 172.21.15.99:0/1439955479 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc160cc0 0x7f23dc0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23dbe00700 1 -- 172.21.15.99:0/1439955479 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442427299 0 0) 0x7f23dc0cddc0 con 0x7f23dc1608e0 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23dbe00700 1 -- 172.21.15.99:0/1439955479 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c40035b0 con 0x7f23dc1608e0 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23dbe00700 1 -- 172.21.15.99:0/1439955479 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758628881 0 0) 0x7f23dc0ce0a0 con 0x7f23dc0d1b20 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23dbe00700 1 -- 172.21.15.99:0/1439955479 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23dc0cddc0 con 0x7f23dc0d1b20 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23e2200700 1 -- 172.21.15.99:0/1439955479 >> v1:172.21.15.133:6789/0 conn(0x7f23dc0d1b20 legacy=0x7f23dc0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23e2200700 1 -- 172.21.15.99:0/1439955479 >> v1:172.21.15.99:6789/0 conn(0x7f23dc1608e0 legacy=0x7f23dc0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.929+0000 7f23e2200700 1 -- 172.21.15.99:0/1439955479 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc0ce660 con 0x7f23dc160cc0 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23e2200700 1 --2- 172.21.15.99:0/1439955479 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc160cc0 0x7f23dc0d0150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009dc0 tx=0x7f23d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b1951794a052442 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23dbe00700 1 -- 172.21.15.99:0/1439955479 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d400e070 con 0x7f23dc160cc0 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23dbe00700 1 -- 172.21.15.99:0/1439955479 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d400bb20 con 0x7f23dc160cc0 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23dbe00700 1 -- 172.21.15.99:0/1439955479 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4012430 con 0x7f23dc160cc0 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23e4200700 1 -- 172.21.15.99:0/1439955479 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc160cc0 msgr2=0x7f23dc0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23e4200700 1 --2- 172.21.15.99:0/1439955479 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc160cc0 0x7f23dc0d0150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009dc0 tx=0x7f23d400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23e4200700 1 -- 172.21.15.99:0/1439955479 shutdown_connections 2024-03-29T09:10:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23e4200700 1 --2- 172.21.15.99:0/1439955479 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc160cc0 0x7f23dc0d0150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23e4200700 1 -- 172.21.15.99:0/1439955479 >> 172.21.15.99:0/1439955479 conn(0x7f23dc09c490 msgr2=0x7f23dc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.930+0000 7f23e4200700 1 -- 172.21.15.99:0/1439955479 shutdown_connections 2024-03-29T09:10:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.931+0000 7f23e4200700 1 -- 172.21.15.99:0/1439955479 wait complete. 2024-03-29T09:10:38.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.932+0000 7f23e4200700 1 Processor -- start 2024-03-29T09:10:38.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.932+0000 7f23e4200700 1 -- start start 2024-03-29T09:10:38.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e4200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc0d1b20 0x7f23dc1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e4200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc1608e0 0x7f23dc149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e4200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23dc160cc0 0x7f23dc0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e4200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc1f48b0 con 0x7f23dc160cc0 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e4200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc1f3ca0 con 0x7f23dc0d1b20 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e4200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc1f3f80 con 0x7f23dc1608e0 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23dc160cc0 0x7f23dc0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23dc160cc0 0x7f23dc0b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35650/0 (socket says 172.21.15.99:35650) 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.933+0000 7f23e2200700 1 -- 172.21.15.99:0/2983890823 learned_addr learned my addr 172.21.15.99:0/2983890823 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e2200700 1 -- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc1608e0 msgr2=0x7f23dc149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e0e00700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc1608e0 0x7f23dc149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e1800700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc0d1b20 0x7f23dc1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e2200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc1608e0 0x7f23dc149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e2200700 1 -- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc0d1b20 msgr2=0x7f23dc1548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e2200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc0d1b20 0x7f23dc1548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e2200700 1 -- 172.21.15.99:0/2983890823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d4009a70 con 0x7f23dc160cc0 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e0e00700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc1608e0 0x7f23dc149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e1800700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc0d1b20 0x7f23dc1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e2200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23dc160cc0 0x7f23dc0b3470 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f23d400b370 tx=0x7f23d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23d9600700 1 -- 172.21.15.99:0/2983890823 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d401b810 con 0x7f23dc160cc0 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23d9600700 1 -- 172.21.15.99:0/2983890823 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d4020070 con 0x7f23dc160cc0 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc1f4260 con 0x7f23dc160cc0 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.934+0000 7f23d9600700 1 -- 172.21.15.99:0/2983890823 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d401bbf0 con 0x7f23dc160cc0 2024-03-29T09:10:38.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.935+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc146f70 con 0x7f23dc160cc0 2024-03-29T09:10:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.936+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc0677a0 con 0x7f23dc160cc0 2024-03-29T09:10:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.936+0000 7f23d9600700 1 -- 172.21.15.99:0/2983890823 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f23d401bd50 con 0x7f23dc160cc0 2024-03-29T09:10:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.937+0000 7f23d9600700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23bc06acc0 0x7f23bc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.937+0000 7f23d9600700 1 -- 172.21.15.99:0/2983890823 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f23d4022070 con 0x7f23dc160cc0 2024-03-29T09:10:38.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.938+0000 7f23e1800700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23bc06acc0 0x7f23bc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:38.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.938+0000 7f23e1800700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23bc06acc0 0x7f23bc06d180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000f60 tx=0x7f23d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:38.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:38.942+0000 7f23d9600700 1 -- 172.21.15.99:0/2983890823 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f23d4056840 con 0x7f23dc160cc0 2024-03-29T09:10:39.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.226+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f23dc0d0700 con 0x7f23dc160cc0 2024-03-29T09:10:39.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.229+0000 7f23d9600700 1 -- 172.21.15.99:0/2983890823 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f23d402c0a0 con 0x7f23dc160cc0 2024-03-29T09:10:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23bc06acc0 msgr2=0x7f23bc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23bc06acc0 0x7f23bc06d180 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000f60 tx=0x7f23d0009040 comp rx=0 tx=0).stop 2024-03-29T09:10:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23dc160cc0 msgr2=0x7f23dc0b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23dc160cc0 0x7f23dc0b3470 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f23d400b370 tx=0x7f23d4004ad0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 shutdown_connections 2024-03-29T09:10:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23bc06acc0 0x7f23bc06d180 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23dc160cc0 0x7f23dc0b3470 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23dc0d1b20 0x7f23dc1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 --2- 172.21.15.99:0/2983890823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23dc1608e0 0x7f23dc149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 >> 172.21.15.99:0/2983890823 conn(0x7f23dc09c490 msgr2=0x7f23dc15e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 shutdown_connections 2024-03-29T09:10:39.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.232+0000 7f23e4200700 1 -- 172.21.15.99:0/2983890823 wait complete. 2024-03-29T09:10:39.250 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-29T09:10:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.443+0000 7f51b4000700 1 Processor -- start 2024-03-29T09:10:39.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.443+0000 7f51b4000700 1 -- start start 2024-03-29T09:10:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b4000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac15d690 0x7f51ac0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b4000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f51ac0d1740 0x7f51ac0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b4000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0afde0 con 0x7f51ac15da70 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b4000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac0d2130 con 0x7f51ac0d1740 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b4000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0d01d0 con 0x7f51ac15d690 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b0c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f51ac15da70 0x7f51ac0cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47938/0 (socket says 172.21.15.99:47938) 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b0c00700 1 -- 172.21.15.99:0/3854587482 learned_addr learned my addr 172.21.15.99:0/3854587482 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b1600700 1 --2- 172.21.15.99:0/3854587482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac15d690 0x7f51ac0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.444+0000 7f51b2000700 1 --2- 172.21.15.99:0/3854587482 >> v2:172.21.15.133:3300/0 conn(0x7f51ac0d1740 0x7f51ac0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.445+0000 7f51abe00700 1 -- 172.21.15.99:0/3854587482 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531032104 0 0) 0x7f51ac0afde0 con 0x7f51ac15da70 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.445+0000 7f51abe00700 1 -- 172.21.15.99:0/3854587482 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51940035b0 con 0x7f51ac15da70 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.445+0000 7f51b1600700 1 -- 172.21.15.99:0/3854587482 >> v2:172.21.15.133:3300/0 conn(0x7f51ac0d1740 msgr2=0x7f51ac0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.445+0000 7f51b1600700 1 --2- 172.21.15.99:0/3854587482 >> v2:172.21.15.133:3300/0 conn(0x7f51ac0d1740 0x7f51ac0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.445+0000 7f51b1600700 1 -- 172.21.15.99:0/3854587482 >> v1:172.21.15.99:6789/0 conn(0x7f51ac15da70 legacy=0x7f51ac0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.445+0000 7f51b1600700 1 -- 172.21.15.99:0/3854587482 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0d04b0 con 0x7f51ac15d690 2024-03-29T09:10:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.445+0000 7f51b1600700 1 --2- 172.21.15.99:0/3854587482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac15d690 0x7f51ac0cfb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009840 tx=0x7f51a0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4676bf1c221fe0f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51abe00700 1 -- 172.21.15.99:0/3854587482 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0011830 con 0x7f51ac15d690 2024-03-29T09:10:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51abe00700 1 -- 172.21.15.99:0/3854587482 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a0011990 con 0x7f51ac15d690 2024-03-29T09:10:39.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51abe00700 1 -- 172.21.15.99:0/3854587482 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0011d20 con 0x7f51ac15d690 2024-03-29T09:10:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 -- 172.21.15.99:0/3854587482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac15d690 msgr2=0x7f51ac0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 --2- 172.21.15.99:0/3854587482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac15d690 0x7f51ac0cfb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009840 tx=0x7f51a0009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 -- 172.21.15.99:0/3854587482 shutdown_connections 2024-03-29T09:10:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 --2- 172.21.15.99:0/3854587482 >> v2:172.21.15.133:3300/0 conn(0x7f51ac0d1740 0x7f51ac0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 --2- 172.21.15.99:0/3854587482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac15d690 0x7f51ac0cfb60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 -- 172.21.15.99:0/3854587482 >> 172.21.15.99:0/3854587482 conn(0x7f51ac09c480 msgr2=0x7f51ac0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 -- 172.21.15.99:0/3854587482 shutdown_connections 2024-03-29T09:10:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.446+0000 7f51b4000700 1 -- 172.21.15.99:0/3854587482 wait complete. 2024-03-29T09:10:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.448+0000 7f51b4000700 1 Processor -- start 2024-03-29T09:10:39.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.448+0000 7f51b4000700 1 -- start start 2024-03-29T09:10:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b4000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac0d1740 0x7f51ac1f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b4000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d690 0x7f51ac15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b4000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac15da70 0x7f51ac0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b4000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0b6110 con 0x7f51ac15d690 2024-03-29T09:10:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b4000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1f6fe0 con 0x7f51ac15da70 2024-03-29T09:10:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b4000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac1f6a20 con 0x7f51ac0d1740 2024-03-29T09:10:39.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d690 0x7f51ac15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d690 0x7f51ac15b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35652/0 (socket says 172.21.15.99:35652) 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b0c00700 1 -- 172.21.15.99:0/2573303843 learned_addr learned my addr 172.21.15.99:0/2573303843 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.449+0000 7f51b2000700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac15da70 0x7f51ac0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b0c00700 1 -- 172.21.15.99:0/2573303843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac0d1740 msgr2=0x7f51ac1f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b0c00700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac0d1740 0x7f51ac1f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b0c00700 1 -- 172.21.15.99:0/2573303843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac15da70 msgr2=0x7f51ac0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b0c00700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac15da70 0x7f51ac0cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b0c00700 1 -- 172.21.15.99:0/2573303843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00094a0 con 0x7f51ac15d690 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b0c00700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d690 0x7f51ac15b2b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5198001eb0 tx=0x7f519800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51a9600700 1 -- 172.21.15.99:0/2573303843 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198005d70 con 0x7f51ac15d690 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51a9600700 1 -- 172.21.15.99:0/2573303843 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51980070d0 con 0x7f51ac15d690 2024-03-29T09:10:39.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac1f6d00 con 0x7f51ac15d690 2024-03-29T09:10:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51a9600700 1 -- 172.21.15.99:0/2573303843 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198007460 con 0x7f51ac15d690 2024-03-29T09:10:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.450+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac1f6660 con 0x7f51ac15d690 2024-03-29T09:10:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.451+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac1f67f0 con 0x7f51ac15d690 2024-03-29T09:10:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.452+0000 7f51a9600700 1 -- 172.21.15.99:0/2573303843 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f51980075c0 con 0x7f51ac15d690 2024-03-29T09:10:39.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.453+0000 7f51a9600700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f518c06ac90 0x7f518c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.454+0000 7f51a9600700 1 -- 172.21.15.99:0/2573303843 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f51980886b0 con 0x7f51ac15d690 2024-03-29T09:10:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.454+0000 7f51b1600700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f518c06ac90 0x7f518c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.454+0000 7f51b1600700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f518c06ac90 0x7f518c06d150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0ca1d0 tx=0x7f51a0010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:39.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.458+0000 7f51a9600700 1 -- 172.21.15.99:0/2573303843 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5198053a00 con 0x7f51ac15d690 2024-03-29T09:10:39.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.745+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f51ac0b6250 con 0x7f51ac15d690 2024-03-29T09:10:39.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.749+0000 7f51a9600700 1 -- 172.21.15.99:0/2573303843 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f51ac0b6250 con 0x7f51ac15d690 2024-03-29T09:10:39.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f518c06ac90 msgr2=0x7f518c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f518c06ac90 0x7f518c06d150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0ca1d0 tx=0x7f51a0010000 comp rx=0 tx=0).stop 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d690 msgr2=0x7f51ac15b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d690 0x7f51ac15b2b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5198001eb0 tx=0x7f519800c7b0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 shutdown_connections 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f518c06ac90 0x7f518c06d150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ac15da70 0x7f51ac0cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ac0d1740 0x7f51ac1f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 --2- 172.21.15.99:0/2573303843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51ac15d690 0x7f51ac15b2b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 >> 172.21.15.99:0/2573303843 conn(0x7f51ac09c480 msgr2=0x7f51ac0cddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.751+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 shutdown_connections 2024-03-29T09:10:39.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.752+0000 7f51b4000700 1 -- 172.21.15.99:0/2573303843 wait complete. 2024-03-29T09:10:39.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-29T09:10:39.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.960+0000 7f7a70200700 1 Processor -- start 2024-03-29T09:10:39.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.960+0000 7f7a70200700 1 -- start start 2024-03-29T09:10:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.961+0000 7f7a70200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7a681608e0 0x7f7a680d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a70200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680cddc0 con 0x7f7a68160cc0 2024-03-29T09:10:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a70200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680ce0a0 con 0x7f7a680d1b20 2024-03-29T09:10:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a70200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680ce380 con 0x7f7a681608e0 2024-03-29T09:10:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a6d800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7a681608e0 0x7f7a680d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a6d800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7a681608e0 0x7f7a680d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35672/0 (socket says 172.21.15.99:35672) 2024-03-29T09:10:39.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a6d800700 1 -- 172.21.15.99:0/3809315977 learned_addr learned my addr 172.21.15.99:0/3809315977 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a6d800700 1 -- 172.21.15.99:0/3809315977 >> v1:172.21.15.133:6789/0 conn(0x7f7a680d1b20 legacy=0x7f7a680afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a6d800700 1 -- 172.21.15.99:0/3809315977 >> v1:172.21.15.99:6789/0 conn(0x7f7a68160cc0 legacy=0x7f7a680d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a6d800700 1 -- 172.21.15.99:0/3809315977 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680ce660 con 0x7f7a681608e0 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.962+0000 7f7a67e00700 1 -- 172.21.15.99:0/3809315977 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939065587 0 0) 0x7f7a680cddc0 con 0x7f7a68160cc0 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a6d800700 1 --2- 172.21.15.99:0/3809315977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680d0150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0098f0 tx=0x7f7a5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2ecbce97d23566 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a67e00700 1 -- 172.21.15.99:0/3809315977 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c004230 con 0x7f7a681608e0 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a67e00700 1 -- 172.21.15.99:0/3809315977 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c004390 con 0x7f7a681608e0 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a67e00700 1 -- 172.21.15.99:0/3809315977 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0036a0 con 0x7f7a681608e0 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a70200700 1 -- 172.21.15.99:0/3809315977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 msgr2=0x7f7a680d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a70200700 1 --2- 172.21.15.99:0/3809315977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680d0150 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c0098f0 tx=0x7f7a5c009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:39.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a70200700 1 -- 172.21.15.99:0/3809315977 shutdown_connections 2024-03-29T09:10:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a70200700 1 --2- 172.21.15.99:0/3809315977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680d0150 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a70200700 1 -- 172.21.15.99:0/3809315977 >> 172.21.15.99:0/3809315977 conn(0x7f7a6809c490 msgr2=0x7f7a680d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.963+0000 7f7a70200700 1 -- 172.21.15.99:0/3809315977 shutdown_connections 2024-03-29T09:10:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.964+0000 7f7a70200700 1 -- 172.21.15.99:0/3809315977 wait complete. 2024-03-29T09:10:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.965+0000 7f7a70200700 1 Processor -- start 2024-03-29T09:10:39.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.965+0000 7f7a70200700 1 -- start start 2024-03-29T09:10:39.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a70200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a680d1b20 0x7f7a68158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a70200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a70200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a68160cc0 0x7f7a681f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a70200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68158610 con 0x7f7a681608e0 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a70200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680ca760 con 0x7f7a680d1b20 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a70200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68157990 con 0x7f7a68160cc0 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a6ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a6ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35680/0 (socket says 172.21.15.99:35680) 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.966+0000 7f7a6ce00700 1 -- 172.21.15.99:0/212451007 learned_addr learned my addr 172.21.15.99:0/212451007 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6e200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a68160cc0 0x7f7a681f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6ce00700 1 -- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a68160cc0 msgr2=0x7f7a681f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6ce00700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a68160cc0 0x7f7a681f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6ce00700 1 -- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a680d1b20 msgr2=0x7f7a68158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:39.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6ce00700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a680d1b20 0x7f7a68158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6ce00700 1 -- 172.21.15.99:0/212451007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c009550 con 0x7f7a681608e0 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6e200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a68160cc0 0x7f7a681f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a6ce00700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680ca220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c450 tx=0x7f7a5400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a65600700 1 -- 172.21.15.99:0/212451007 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a540054e0 con 0x7f7a681608e0 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a65600700 1 -- 172.21.15.99:0/212451007 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54003680 con 0x7f7a681608e0 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a65600700 1 -- 172.21.15.99:0/212451007 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54003a10 con 0x7f7a681608e0 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.967+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68157c70 con 0x7f7a681608e0 2024-03-29T09:10:39.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.968+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68158230 con 0x7f7a681608e0 2024-03-29T09:10:39.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.969+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a681583f0 con 0x7f7a681608e0 2024-03-29T09:10:39.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.969+0000 7f7a65600700 1 -- 172.21.15.99:0/212451007 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a54003b70 con 0x7f7a681608e0 2024-03-29T09:10:39.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.970+0000 7f7a65600700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a4806ac90 0x7f7a4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:39.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.970+0000 7f7a65600700 1 -- 172.21.15.99:0/212451007 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7a540885c0 con 0x7f7a681608e0 2024-03-29T09:10:39.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.970+0000 7f7a6d800700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a4806ac90 0x7f7a4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:39.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.971+0000 7f7a6d800700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a4806ac90 0x7f7a4806d150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c000c00 tx=0x7f7a5c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:39.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:39.975+0000 7f7a65600700 1 -- 172.21.15.99:0/212451007 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a540539a0 con 0x7f7a681608e0 2024-03-29T09:10:40.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.257+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f7a6815cbf0 con 0x7f7a681608e0 2024-03-29T09:10:40.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.261+0000 7f7a65600700 1 -- 172.21.15.99:0/212451007 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7a540170a0 con 0x7f7a681608e0 2024-03-29T09:10:40.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a4806ac90 msgr2=0x7f7a4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a4806ac90 0x7f7a4806d150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c000c00 tx=0x7f7a5c010000 comp rx=0 tx=0).stop 2024-03-29T09:10:40.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 msgr2=0x7f7a680ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680ca220 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c450 tx=0x7f7a5400c810 comp rx=0 tx=0).stop 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 shutdown_connections 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a4806ac90 0x7f7a4806d150 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a68160cc0 0x7f7a681f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a680d1b20 0x7f7a68158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 --2- 172.21.15.99:0/212451007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a681608e0 0x7f7a680ca220 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 >> 172.21.15.99:0/212451007 conn(0x7f7a6809c490 msgr2=0x7f7a680ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 shutdown_connections 2024-03-29T09:10:40.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.264+0000 7f7a70200700 1 -- 172.21.15.99:0/212451007 wait complete. 2024-03-29T09:10:40.283 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.475+0000 7fb1eac00700 1 Processor -- start 2024-03-29T09:10:40.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.475+0000 7fb1eac00700 1 -- start start 2024-03-29T09:10:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1eac00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1e40d1ba0 0x7fb1e40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1eac00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb1e40cfd00 0x7fb1e40b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1eac00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40d0110 con 0x7fb1e415d140 2024-03-29T09:10:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1eac00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e40d38d0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1eac00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e40d0b00 con 0x7fb1e40cfd00 2024-03-29T09:10:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1e8e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1e40d1ba0 0x7fb1e40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1e8e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb1e40d1ba0 0x7fb1e40cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35696/0 (socket says 172.21.15.99:35696) 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.476+0000 7fb1e8e00700 1 -- 172.21.15.99:0/4251085304 learned_addr learned my addr 172.21.15.99:0/4251085304 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1ea200700 1 --2- 172.21.15.99:0/4251085304 >> v2:172.21.15.133:3300/0 conn(0x7fb1e40cfd00 0x7fb1e40b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e8e00700 1 -- 172.21.15.99:0/4251085304 >> v2:172.21.15.133:3300/0 conn(0x7fb1e40cfd00 msgr2=0x7fb1e40b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e8e00700 1 --2- 172.21.15.99:0/4251085304 >> v2:172.21.15.133:3300/0 conn(0x7fb1e40cfd00 0x7fb1e40b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e8e00700 1 -- 172.21.15.99:0/4251085304 >> v1:172.21.15.133:6789/0 conn(0x7fb1e415d140 legacy=0x7fb1e40ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e8e00700 1 -- 172.21.15.99:0/4251085304 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40d0de0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1ea200700 1 --2- 172.21.15.99:0/4251085304 >> v2:172.21.15.133:3300/0 conn(0x7fb1e40cfd00 0x7fb1e40b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e8e00700 1 --2- 172.21.15.99:0/4251085304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e40cf7c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40d2d00 tx=0x7fb1d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e554639679619e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e3e00700 1 -- 172.21.15.99:0/4251085304 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000b870 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e3e00700 1 -- 172.21.15.99:0/4251085304 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d000b9d0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.477+0000 7fb1e3e00700 1 -- 172.21.15.99:0/4251085304 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000bd60 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 -- 172.21.15.99:0/4251085304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 msgr2=0x7fb1e40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 --2- 172.21.15.99:0/4251085304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e40cf7c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40d2d00 tx=0x7fb1d0009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 -- 172.21.15.99:0/4251085304 shutdown_connections 2024-03-29T09:10:40.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 --2- 172.21.15.99:0/4251085304 >> v2:172.21.15.133:3300/0 conn(0x7fb1e40cfd00 0x7fb1e40b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 --2- 172.21.15.99:0/4251085304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e40cf7c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 -- 172.21.15.99:0/4251085304 >> 172.21.15.99:0/4251085304 conn(0x7fb1e409c2e0 msgr2=0x7fb1e40d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:40.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 -- 172.21.15.99:0/4251085304 shutdown_connections 2024-03-29T09:10:40.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.478+0000 7fb1eac00700 1 -- 172.21.15.99:0/4251085304 wait complete. 2024-03-29T09:10:40.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.480+0000 7fb1eac00700 1 Processor -- start 2024-03-29T09:10:40.480 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.481+0000 7fb1eac00700 1 -- start start 2024-03-29T09:10:40.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1eac00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e40cfd00 0x7fb1e41516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1eac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e409fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1eac00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e415d140 0x7fb1e415f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1eac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e409e0f0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1eac00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e409e3d0 con 0x7fb1e40cfd00 2024-03-29T09:10:40.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1eac00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e409e6b0 con 0x7fb1e415d140 2024-03-29T09:10:40.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e409fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e409fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35712/0 (socket says 172.21.15.99:35712) 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 -- 172.21.15.99:0/1112940965 learned_addr learned my addr 172.21.15.99:0/1112940965 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e9800700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e40cfd00 0x7fb1e41516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1ea200700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e415d140 0x7fb1e415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 -- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e415d140 msgr2=0x7fb1e415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e415d140 0x7fb1e415f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 -- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e40cfd00 msgr2=0x7fb1e41516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e40cfd00 0x7fb1e41516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e8e00700 1 -- 172.21.15.99:0/1112940965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d0009550 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.482+0000 7fb1e9800700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e40cfd00 0x7fb1e41516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.483+0000 7fb1e8e00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e409fa60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0000c00 tx=0x7fb1d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.483+0000 7fb1e1600700 1 -- 172.21.15.99:0/1112940965 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0004540 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.483+0000 7fb1e1600700 1 -- 172.21.15.99:0/1112940965 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d00036a0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.483+0000 7fb1e1600700 1 -- 172.21.15.99:0/1112940965 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0003a30 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.483+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e409e990 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.483+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e409f190 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.484+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e409eeb0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.485+0000 7fb1e1600700 1 -- 172.21.15.99:0/1112940965 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb1d0021070 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.486+0000 7fb1e1600700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb1c406ac90 0x7fb1c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.486+0000 7fb1e1600700 1 -- 172.21.15.99:0/1112940965 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb1d008a930 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.486+0000 7fb1e9800700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb1c406ac90 0x7fb1c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.487+0000 7fb1e9800700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb1c406ac90 0x7fb1c406d150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8000f60 tx=0x7fb1d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:40.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.491+0000 7fb1e1600700 1 -- 172.21.15.99:0/1112940965 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb1e409eeb0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.695+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1e409eeb0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.696+0000 7fb1e1600700 1 -- 172.21.15.99:0/1112940965 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+12594 (secure 0 0 0) 0x7fb1e409eeb0 con 0x7fb1e40d1ba0 2024-03-29T09:10:40.696 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:40.696 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":35,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:36.653263+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb1c406ac90 msgr2=0x7fb1c406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb1c406ac90 0x7fb1c406d150 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8000f60 tx=0x7fb1d8009040 comp rx=0 tx=0).stop 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 msgr2=0x7fb1e409fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e409fa60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0000c00 tx=0x7fb1d0004750 comp rx=0 tx=0).stop 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 shutdown_connections 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb1c406ac90 0x7fb1c406d150 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e415d140 0x7fb1e415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1e40cfd00 0x7fb1e41516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 --2- 172.21.15.99:0/1112940965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb1e40d1ba0 0x7fb1e409fa60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 >> 172.21.15.99:0/1112940965 conn(0x7fb1e409c2e0 msgr2=0x7fb1e40cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 shutdown_connections 2024-03-29T09:10:40.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.698+0000 7fb1eac00700 1 -- 172.21.15.99:0/1112940965 wait complete. 2024-03-29T09:10:40.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:10:40.718 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:40.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.910+0000 7f7fef800700 1 Processor -- start 2024-03-29T09:10:40.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.910+0000 7f7fef800700 1 -- start start 2024-03-29T09:10:40.912 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.911+0000 7f7fef800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7fe80b3cc0 0x7f7fe80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.911+0000 7f7fef800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.911+0000 7f7fef800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe80d3810 con 0x7f7fe8160770 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.911+0000 7f7fef800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe80b4140 con 0x7f7fe80b3cc0 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.911+0000 7f7fef800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80d1520 con 0x7f7fe80d3380 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fe7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7fe80b3cc0 0x7f7fe80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fece00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7fe8160770 0x7f7fe80d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47956/0 (socket says 172.21.15.99:47956) 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fece00700 1 -- 172.21.15.99:0/4129905659 learned_addr learned my addr 172.21.15.99:0/4129905659 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fed800700 1 --2- 172.21.15.99:0/4129905659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fe7e00700 1 -- 172.21.15.99:0/4129905659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 msgr2=0x7f7fe80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fe7e00700 1 --2- 172.21.15.99:0/4129905659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe80ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fe7e00700 1 -- 172.21.15.99:0/4129905659 >> v1:172.21.15.99:6789/0 conn(0x7f7fe8160770 legacy=0x7f7fe80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fe7e00700 1 -- 172.21.15.99:0/4129905659 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe80d1800 con 0x7f7fe80b3cc0 2024-03-29T09:10:40.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fed800700 1 --2- 172.21.15.99:0/4129905659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:40.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fe7400700 1 -- 172.21.15.99:0/4129905659 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053704787 0 0) 0x7f7fe80d3810 con 0x7f7fe8160770 2024-03-29T09:10:40.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.912+0000 7f7fe7e00700 1 --2- 172.21.15.99:0/4129905659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe80b3cc0 0x7f7fe80d2e40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009840 tx=0x7f7fd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c37d9727c89763 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:40.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fe7400700 1 -- 172.21.15.99:0/4129905659 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4011830 con 0x7f7fe80b3cc0 2024-03-29T09:10:40.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fe7400700 1 -- 172.21.15.99:0/4129905659 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd4011990 con 0x7f7fe80b3cc0 2024-03-29T09:10:40.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fe7400700 1 -- 172.21.15.99:0/4129905659 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4011d20 con 0x7f7fe80b3cc0 2024-03-29T09:10:40.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 -- 172.21.15.99:0/4129905659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe80b3cc0 msgr2=0x7f7fe80d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 --2- 172.21.15.99:0/4129905659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe80b3cc0 0x7f7fe80d2e40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009840 tx=0x7f7fd4009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 -- 172.21.15.99:0/4129905659 shutdown_connections 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 --2- 172.21.15.99:0/4129905659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 --2- 172.21.15.99:0/4129905659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe80b3cc0 0x7f7fe80d2e40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 -- 172.21.15.99:0/4129905659 >> 172.21.15.99:0/4129905659 conn(0x7f7fe808afd0 msgr2=0x7f7fe809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 -- 172.21.15.99:0/4129905659 shutdown_connections 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.913+0000 7f7fef800700 1 -- 172.21.15.99:0/4129905659 wait complete. 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.915+0000 7f7fef800700 1 Processor -- start 2024-03-29T09:10:40.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.915+0000 7f7fef800700 1 -- start start 2024-03-29T09:10:40.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fef800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe80b3cc0 0x7f7fe80afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fef800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe815c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fef800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe8160770 0x7f7fe81f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fef800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81f35f0 con 0x7f7fe8160770 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fef800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80b03a0 con 0x7f7fe80b3cc0 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fef800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80b0680 con 0x7f7fe80d3380 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fed800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe8160770 0x7f7fe81f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fed800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe8160770 0x7f7fe81f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35724/0 (socket says 172.21.15.99:35724) 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fed800700 1 -- 172.21.15.99:0/2322213106 learned_addr learned my addr 172.21.15.99:0/2322213106 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:40.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.916+0000 7f7fece00700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe80b3cc0 0x7f7fe80afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fe7e00700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe815c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fed800700 1 -- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 msgr2=0x7f7fe815c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fed800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe815c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fed800700 1 -- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe80b3cc0 msgr2=0x7f7fe80afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fed800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe80b3cc0 0x7f7fe80afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fed800700 1 -- 172.21.15.99:0/2322213106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd40094a0 con 0x7f7fe8160770 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fed800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe8160770 0x7f7fe81f30b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cef0 tx=0x7f7fe000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fe4c00700 1 -- 172.21.15.99:0/2322213106 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000ad00 con 0x7f7fe8160770 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fe4c00700 1 -- 172.21.15.99:0/2322213106 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe000ae60 con 0x7f7fe8160770 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fe4c00700 1 -- 172.21.15.99:0/2322213106 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0015680 con 0x7f7fe8160770 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe80b0960 con 0x7f7fe8160770 2024-03-29T09:10:40.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.917+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe80b7510 con 0x7f7fe8160770 2024-03-29T09:10:40.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.918+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe80b5de0 con 0x7f7fe8160770 2024-03-29T09:10:40.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.919+0000 7f7fe4c00700 1 -- 172.21.15.99:0/2322213106 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7fe000d430 con 0x7f7fe8160770 2024-03-29T09:10:40.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.920+0000 7f7fe4c00700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fc806ac90 0x7f7fc806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.920+0000 7f7fe4c00700 1 -- 172.21.15.99:0/2322213106 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7fe0089610 con 0x7f7fe8160770 2024-03-29T09:10:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.920+0000 7f7fece00700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fc806ac90 0x7f7fc806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:40.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.921+0000 7f7fece00700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fc806ac90 0x7f7fc806d150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000f60 tx=0x7f7fdc007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:40.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:40.925+0000 7f7fe4c00700 1 -- 172.21.15.99:0/2322213106 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7fe80b5de0 con 0x7f7fe8160770 2024-03-29T09:10:41.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.128+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fe80b5de0 con 0x7f7fe8160770 2024-03-29T09:10:41.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.128+0000 7f7fe4c00700 1 -- 172.21.15.99:0/2322213106 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fe80b5de0 con 0x7f7fe8160770 2024-03-29T09:10:41.128 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:41.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fc806ac90 msgr2=0x7f7fc806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fc806ac90 0x7f7fc806d150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc000f60 tx=0x7f7fdc007e70 comp rx=0 tx=0).stop 2024-03-29T09:10:41.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe8160770 msgr2=0x7f7fe81f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe8160770 0x7f7fe81f30b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cef0 tx=0x7f7fe000cf20 comp rx=0 tx=0).stop 2024-03-29T09:10:41.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 shutdown_connections 2024-03-29T09:10:41.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7fc806ac90 0x7f7fc806d150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7fe8160770 0x7f7fe81f30b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe80b3cc0 0x7f7fe80afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 --2- 172.21.15.99:0/2322213106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fe80d3380 0x7f7fe815c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 >> 172.21.15.99:0/2322213106 conn(0x7f7fe808afd0 msgr2=0x7f7fe815e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:41.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 shutdown_connections 2024-03-29T09:10:41.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.131+0000 7f7fef800700 1 -- 172.21.15.99:0/2322213106 wait complete. 2024-03-29T09:10:41.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T09:10:41.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.338+0000 7f92e9000700 1 Processor -- start 2024-03-29T09:10:41.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.338+0000 7f92e9000700 1 -- start start 2024-03-29T09:10:41.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e9000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92e415e7a0 0x7f92e40d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e9000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 0x7f92e40cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e9000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f92e40cfad0 0x7f92e40cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e9000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40d0460 con 0x7f92e415e7a0 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e9000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40cdf80 con 0x7f92e40cfad0 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e9000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e415eb80 con 0x7f92e40d12f0 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e1e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92e415e7a0 0x7f92e40d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e1e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f92e415e7a0 0x7f92e40d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35728/0 (socket says 172.21.15.99:35728) 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e1e00700 1 -- 172.21.15.99:0/2607537814 learned_addr learned my addr 172.21.15.99:0/2607537814 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92dbe00700 1 --2- 172.21.15.99:0/2607537814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 0x7f92e40cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.339+0000 7f92e2800700 1 --2- 172.21.15.99:0/2607537814 >> v2:172.21.15.133:3300/0 conn(0x7f92e40cfad0 0x7f92e40cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1e00700 1 -- 172.21.15.99:0/2607537814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 msgr2=0x7f92e40cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1e00700 1 --2- 172.21.15.99:0/2607537814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 0x7f92e40cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1e00700 1 -- 172.21.15.99:0/2607537814 >> v2:172.21.15.133:3300/0 conn(0x7f92e40cfad0 msgr2=0x7f92e40cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1e00700 1 --2- 172.21.15.99:0/2607537814 >> v2:172.21.15.133:3300/0 conn(0x7f92e40cfad0 0x7f92e40cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1e00700 1 -- 172.21.15.99:0/2607537814 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40d2140 con 0x7f92e415e7a0 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e2800700 1 --2- 172.21.15.99:0/2607537814 >> v2:172.21.15.133:3300/0 conn(0x7f92e40cfad0 0x7f92e40cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1e00700 1 --2- 172.21.15.99:0/2607537814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e415e7a0 0x7f92e40d1a40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f92e40cd910 tx=0x7f92d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a275fb1ad59bd8fe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1400700 1 -- 172.21.15.99:0/2607537814 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4011830 con 0x7f92e415e7a0 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1400700 1 -- 172.21.15.99:0/2607537814 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4011990 con 0x7f92e415e7a0 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.340+0000 7f92e1400700 1 -- 172.21.15.99:0/2607537814 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4011d20 con 0x7f92e415e7a0 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 -- 172.21.15.99:0/2607537814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e415e7a0 msgr2=0x7f92e40d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 --2- 172.21.15.99:0/2607537814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e415e7a0 0x7f92e40d1a40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f92e40cd910 tx=0x7f92d4009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 -- 172.21.15.99:0/2607537814 shutdown_connections 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 --2- 172.21.15.99:0/2607537814 >> v2:172.21.15.133:3300/0 conn(0x7f92e40cfad0 0x7f92e40cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 --2- 172.21.15.99:0/2607537814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e415e7a0 0x7f92e40d1a40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 --2- 172.21.15.99:0/2607537814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 0x7f92e40cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 -- 172.21.15.99:0/2607537814 >> 172.21.15.99:0/2607537814 conn(0x7f92e409c480 msgr2=0x7f92e40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:41.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 -- 172.21.15.99:0/2607537814 shutdown_connections 2024-03-29T09:10:41.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.341+0000 7f92e9000700 1 -- 172.21.15.99:0/2607537814 wait complete. 2024-03-29T09:10:41.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.343+0000 7f92e9000700 1 Processor -- start 2024-03-29T09:10:41.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.343+0000 7f92e9000700 1 -- start start 2024-03-29T09:10:41.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e40cfad0 0x7f92e41576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e9000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 0x7f92e40c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e9000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92e415e7a0 0x7f92e414da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e9000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c95a0 con 0x7f92e40cfad0 2024-03-29T09:10:41.346 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e9000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e41562c0 con 0x7f92e415e7a0 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e9000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c99c0 con 0x7f92e40d12f0 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e40cfad0 0x7f92e41576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e40cfad0 0x7f92e41576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35732/0 (socket says 172.21.15.99:35732) 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 -- 172.21.15.99:0/3900660644 learned_addr learned my addr 172.21.15.99:0/3900660644 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 -- 172.21.15.99:0/3900660644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 msgr2=0x7f92e40c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 0x7f92e40c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 -- 172.21.15.99:0/3900660644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92e415e7a0 msgr2=0x7f92e414da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92e415e7a0 0x7f92e414da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.345+0000 7f92e1e00700 1 -- 172.21.15.99:0/3900660644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4009550 con 0x7f92e40cfad0 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.346+0000 7f92e1e00700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e40cfad0 0x7f92e41576c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f92e408d930 tx=0x7f92d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.346+0000 7f92da000700 1 -- 172.21.15.99:0/3900660644 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40036a0 con 0x7f92e40cfad0 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.346+0000 7f92da000700 1 -- 172.21.15.99:0/3900660644 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d401e070 con 0x7f92e40cfad0 2024-03-29T09:10:41.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.346+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e40c8920 con 0x7f92e40cfad0 2024-03-29T09:10:41.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.346+0000 7f92da000700 1 -- 172.21.15.99:0/3900660644 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4003a60 con 0x7f92e40cfad0 2024-03-29T09:10:41.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.346+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e40c8ee0 con 0x7f92e40cfad0 2024-03-29T09:10:41.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.347+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e40677a0 con 0x7f92e40cfad0 2024-03-29T09:10:41.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.348+0000 7f92da000700 1 -- 172.21.15.99:0/3900660644 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f92d4003bc0 con 0x7f92e40cfad0 2024-03-29T09:10:41.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.348+0000 7f92da000700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92c006ac90 0x7f92c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.348+0000 7f92da000700 1 -- 172.21.15.99:0/3900660644 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f92d408a080 con 0x7f92e40cfad0 2024-03-29T09:10:41.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.349+0000 7f92dbe00700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92c006ac90 0x7f92c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.349+0000 7f92dbe00700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92c006ac90 0x7f92c006d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f92e40cc9c0 tx=0x7f92cc0095a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:41.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.353+0000 7f92da000700 1 -- 172.21.15.99:0/3900660644 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f92d40bd580 con 0x7f92e40cfad0 2024-03-29T09:10:41.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.564+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f92e408cf20 con 0x7f92e40cfad0 2024-03-29T09:10:41.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.564+0000 7f92da000700 1 -- 172.21.15.99:0/3900660644 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f92d4020090 con 0x7f92e40cfad0 2024-03-29T09:10:41.565 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:41.565 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-03-29T09:10:41.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:10:41.565 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-03-29T09:10:41.565 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-03-29T09:10:41.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:41.565 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92c006ac90 msgr2=0x7f92c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92c006ac90 0x7f92c006d150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f92e40cc9c0 tx=0x7f92cc0095a0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e40cfad0 msgr2=0x7f92e41576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e40cfad0 0x7f92e41576c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f92e408d930 tx=0x7f92d4004750 comp rx=0 tx=0).stop 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 shutdown_connections 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f92c006ac90 0x7f92c006d150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92e415e7a0 0x7f92e414da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f92e40cfad0 0x7f92e41576c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 --2- 172.21.15.99:0/3900660644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f92e40d12f0 0x7f92e40c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 >> 172.21.15.99:0/3900660644 conn(0x7f92e409c480 msgr2=0x7f92e406cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:41.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 shutdown_connections 2024-03-29T09:10:41.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.567+0000 7f92e9000700 1 -- 172.21.15.99:0/3900660644 wait complete. 2024-03-29T09:10:41.585 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T09:10:41.585 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T09:10:41.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.779+0000 7fc1c1a00700 1 Processor -- start 2024-03-29T09:10:41.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.779+0000 7fc1c1a00700 1 -- start start 2024-03-29T09:10:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.780+0000 7fc1c1a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc160b30 0x7fc1bc0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.780+0000 7fc1c1a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc1bc0d0090 0x7fc1bc0ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.780+0000 7fc1c1a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1bc0d1fa0 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.780+0000 7fc1c1a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1bc0cf440 con 0x7fc1bc0d0090 2024-03-29T09:10:41.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.780+0000 7fc1c1a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc0d0520 con 0x7fc1bc160b30 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.780+0000 7fc1bb400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc1bc0d0090 0x7fc1bc0ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1baa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc160b30 0x7fc1bc0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1baa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc160b30 0x7fc1bc0cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:37546/0 (socket says 172.21.15.99:37546) 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1baa00700 1 -- 172.21.15.99:0/3119024689 learned_addr learned my addr 172.21.15.99:0/3119024689 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1bb400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc1bc0d0090 0x7fc1bc0ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:38544/0 (socket says 172.21.15.99:38544) 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1ba000700 1 --1- 172.21.15.99:0/3119024689 >> v1:172.21.15.133:6789/0 conn(0x7fc1bc0b1f50 0x7fc1bc0b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:44250/0 (socket says 172.21.15.99:44250) 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1bb400700 1 -- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc160b30 msgr2=0x7fc1bc0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:41.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1bb400700 1 --2- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc160b30 0x7fc1bc0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1bb400700 1 -- 172.21.15.99:0/3119024689 >> v1:172.21.15.133:6789/0 conn(0x7fc1bc0b1f50 legacy=0x7fc1bc0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1bb400700 1 -- 172.21.15.99:0/3119024689 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc160f10 con 0x7fc1bc0d0090 2024-03-29T09:10:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1baa00700 1 --2- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc160b30 0x7fc1bc0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:41.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.781+0000 7fc1bb400700 1 --2- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc0d0090 0x7fc1bc0ceda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009d90 tx=0x7fc1b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0b8b1d2d897032 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1b9600700 1 -- 172.21.15.99:0/3119024689 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400e070 con 0x7fc1bc0d0090 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1b9600700 1 -- 172.21.15.99:0/3119024689 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b400bb20 con 0x7fc1bc0d0090 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1b9600700 1 -- 172.21.15.99:0/3119024689 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4012430 con 0x7fc1bc0d0090 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 -- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc0d0090 msgr2=0x7fc1bc0ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc0d0090 0x7fc1bc0ceda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009d90 tx=0x7fc1b400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 -- 172.21.15.99:0/3119024689 shutdown_connections 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc0d0090 0x7fc1bc0ceda0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/3119024689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc160b30 0x7fc1bc0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 -- 172.21.15.99:0/3119024689 >> 172.21.15.99:0/3119024689 conn(0x7fc1bc09c4d0 msgr2=0x7fc1bc0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:41.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 -- 172.21.15.99:0/3119024689 shutdown_connections 2024-03-29T09:10:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.782+0000 7fc1c1a00700 1 -- 172.21.15.99:0/3119024689 wait complete. 2024-03-29T09:10:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.784+0000 7fc1c1a00700 1 Processor -- start 2024-03-29T09:10:41.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.784+0000 7fc1c1a00700 1 -- start start 2024-03-29T09:10:41.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1c1a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0b1f50 0x7fc1bc14fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1c1a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc0d0090 0x7fc1bc1f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1c1a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc160b30 0x7fc1bc0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1c1a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1502b0 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1c1a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc14d600 con 0x7fc1bc160b30 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1c1a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc14d8e0 con 0x7fc1bc0d0090 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1baa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0b1f50 0x7fc1bc14fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.785+0000 7fc1baa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0b1f50 0x7fc1bc14fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35738/0 (socket says 172.21.15.99:35738) 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1baa00700 1 -- 172.21.15.99:0/1572019192 learned_addr learned my addr 172.21.15.99:0/1572019192 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1bb400700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc160b30 0x7fc1bc0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1ba000700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc0d0090 0x7fc1bc1f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1baa00700 1 -- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc0d0090 msgr2=0x7fc1bc1f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1baa00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc0d0090 0x7fc1bc1f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1baa00700 1 -- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc160b30 msgr2=0x7fc1bc0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1baa00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc160b30 0x7fc1bc0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:41.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1baa00700 1 -- 172.21.15.99:0/1572019192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac009550 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1bb400700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc160b30 0x7fc1bc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1baa00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0b1f50 0x7fc1bc14fd70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00e9b0 tx=0x7fc1ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1b2a00700 1 -- 172.21.15.99:0/1572019192 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00c850 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1b2a00700 1 -- 172.21.15.99:0/1572019192 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac00c9b0 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1b2a00700 1 -- 172.21.15.99:0/1572019192 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00cd20 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.786+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b4009a70 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.787+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc14dea0 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.788+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc0b08b0 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.788+0000 7fc1b2a00700 1 -- 172.21.15.99:0/1572019192 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc1ac013470 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.789+0000 7fc1b2a00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc19c06ac90 0x7fc19c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:41.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.789+0000 7fc1b2a00700 1 -- 172.21.15.99:0/1572019192 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc1ac089360 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.790+0000 7fc1ba000700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc19c06ac90 0x7fc19c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:41.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.790+0000 7fc1ba000700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc19c06ac90 0x7fc19c06d150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc1f6350 tx=0x7fc1a8007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:41.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.793+0000 7fc1b2a00700 1 -- 172.21.15.99:0/1572019192 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc1ac0546b0 con 0x7fc1bc0b1f50 2024-03-29T09:10:41.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:41.997+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc1bc0678b0 con 0x7fc1bc0b1f50 2024-03-29T09:10:42.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-03-29T09:10:42.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.411+0000 7fc1b2a00700 1 -- 172.21.15.99:0/1572019192 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc1ac057cd0 con 0x7fc1bc0b1f50 2024-03-29T09:10:42.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc19c06ac90 msgr2=0x7fc19c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:42.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc19c06ac90 0x7fc19c06d150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc1f6350 tx=0x7fc1a8007d10 comp rx=0 tx=0).stop 2024-03-29T09:10:42.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0b1f50 msgr2=0x7fc1bc14fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:42.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0b1f50 0x7fc1bc14fd70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00e9b0 tx=0x7fc1ac00ed70 comp rx=0 tx=0).stop 2024-03-29T09:10:42.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 shutdown_connections 2024-03-29T09:10:42.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc19c06ac90 0x7fc19c06d150 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:42.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1bc160b30 0x7fc1bc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:42.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1bc0b1f50 0x7fc1bc14fd70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:42.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 --2- 172.21.15.99:0/1572019192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1bc0d0090 0x7fc1bc1f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:42.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 >> 172.21.15.99:0/1572019192 conn(0x7fc1bc09c4d0 msgr2=0x7fc1bc0b01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:42.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 shutdown_connections 2024-03-29T09:10:42.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.414+0000 7fc1c1a00700 1 -- 172.21.15.99:0/1572019192 wait complete. 2024-03-29T09:10:42.418 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-03-29T09:10:42.437 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T09:10:42.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.633+0000 7f0687a00700 1 Processor -- start 2024-03-29T09:10:42.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.633+0000 7f0687a00700 1 -- start start 2024-03-29T09:10:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.634+0000 7f0687a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f06800cf090 0x7f0680167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f0687a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06800d1520 con 0x7f06800b3cc0 2024-03-29T09:10:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f0687a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06800d1800 con 0x7f0680160770 2024-03-29T09:10:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f0687a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06800b4140 con 0x7f06800cf090 2024-03-29T09:10:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f067fe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f06800b3cc0 0x7f06800d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47964/0 (socket says 172.21.15.99:47964) 2024-03-29T09:10:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f067fe00700 1 -- 172.21.15.99:0/422118226 learned_addr learned my addr 172.21.15.99:0/422118226 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:42.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f0685a00700 1 --2- 172.21.15.99:0/422118226 >> v2:172.21.15.133:3300/0 conn(0x7f06800cf090 0x7f0680167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f067f400700 1 -- 172.21.15.99:0/422118226 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278602779 0 0) 0x7f06800d1520 con 0x7f06800b3cc0 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f067f400700 1 -- 172.21.15.99:0/422118226 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06680035b0 con 0x7f06800b3cc0 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.635+0000 7f0685a00700 1 -- 172.21.15.99:0/422118226 >> v1:172.21.15.133:6789/0 conn(0x7f0680160770 legacy=0x7f06800d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.636+0000 7f0685a00700 1 -- 172.21.15.99:0/422118226 >> v1:172.21.15.99:6789/0 conn(0x7f06800b3cc0 legacy=0x7f06800d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.636+0000 7f0685a00700 1 -- 172.21.15.99:0/422118226 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06800d1ae0 con 0x7f06800cf090 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.636+0000 7f0685a00700 1 --2- 172.21.15.99:0/422118226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800cf090 0x7f0680167f40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0678009d90 tx=0x7f067800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82fda235a7854f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.636+0000 7f067f400700 1 -- 172.21.15.99:0/422118226 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800e070 con 0x7f06800cf090 2024-03-29T09:10:42.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.636+0000 7f067f400700 1 -- 172.21.15.99:0/422118226 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067800bb20 con 0x7f06800cf090 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.636+0000 7f067f400700 1 -- 172.21.15.99:0/422118226 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678012430 con 0x7f06800cf090 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.637+0000 7f0687a00700 1 -- 172.21.15.99:0/422118226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800cf090 msgr2=0x7f0680167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.637+0000 7f0687a00700 1 --2- 172.21.15.99:0/422118226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800cf090 0x7f0680167f40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0678009d90 tx=0x7f067800b330 comp rx=0 tx=0).stop 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.637+0000 7f0687a00700 1 -- 172.21.15.99:0/422118226 shutdown_connections 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.637+0000 7f0687a00700 1 --2- 172.21.15.99:0/422118226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800cf090 0x7f0680167f40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.637+0000 7f0687a00700 1 -- 172.21.15.99:0/422118226 >> 172.21.15.99:0/422118226 conn(0x7f068008afd0 msgr2=0x7f068009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.637+0000 7f0687a00700 1 -- 172.21.15.99:0/422118226 shutdown_connections 2024-03-29T09:10:42.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.637+0000 7f0687a00700 1 -- 172.21.15.99:0/422118226 wait complete. 2024-03-29T09:10:42.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.638+0000 7f0687a00700 1 Processor -- start 2024-03-29T09:10:42.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.638+0000 7f0687a00700 1 -- start start 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.639+0000 7f0687a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800b3cc0 0x7f06801547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.639+0000 7f0687a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06800cf090 0x7f06800a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.639+0000 7f0687a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680160770 0x7f06801f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.639+0000 7f0687a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06800a0ee0 con 0x7f0680160770 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.639+0000 7f0687a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068009fbe0 con 0x7f06800b3cc0 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.639+0000 7f0687a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068009fec0 con 0x7f06800cf090 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680160770 0x7f06801f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680160770 0x7f06801f3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35746/0 (socket says 172.21.15.99:35746) 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 -- 172.21.15.99:0/1299383593 learned_addr learned my addr 172.21.15.99:0/1299383593 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:42.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685000700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800b3cc0 0x7f06801547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f067fe00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06800cf090 0x7f06800a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 -- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06800cf090 msgr2=0x7f06800a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06800cf090 0x7f06800a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 -- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800b3cc0 msgr2=0x7f06801547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800b3cc0 0x7f06801547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 -- 172.21.15.99:0/1299383593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678009a70 con 0x7f0680160770 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f067fe00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06800cf090 0x7f06800a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.640+0000 7f0685a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680160770 0x7f06801f3150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0678013000 tx=0x7f0678004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.641+0000 7f067cc00700 1 -- 172.21.15.99:0/1299383593 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678012430 con 0x7f0680160770 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.641+0000 7f067cc00700 1 -- 172.21.15.99:0/1299383593 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678020070 con 0x7f0680160770 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.641+0000 7f067cc00700 1 -- 172.21.15.99:0/1299383593 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678004280 con 0x7f0680160770 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.641+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06800a01a0 con 0x7f0680160770 2024-03-29T09:10:42.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.641+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068009f890 con 0x7f0680160770 2024-03-29T09:10:42.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.642+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06800a06c0 con 0x7f0680160770 2024-03-29T09:10:42.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.643+0000 7f067cc00700 1 -- 172.21.15.99:0/1299383593 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f06780043e0 con 0x7f0680160770 2024-03-29T09:10:42.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.644+0000 7f067cc00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f066006ac90 0x7f066006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:42.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.644+0000 7f067cc00700 1 -- 172.21.15.99:0/1299383593 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f0678022070 con 0x7f0680160770 2024-03-29T09:10:42.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.644+0000 7f0685000700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f066006ac90 0x7f066006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:42.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.645+0000 7f0685000700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f066006ac90 0x7f066006d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0674000f60 tx=0x7f0674009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:42.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.649+0000 7f067cc00700 1 -- 172.21.15.99:0/1299383593 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f06800a06c0 con 0x7f0680160770 2024-03-29T09:10:42.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:42.852+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f06800a06c0 con 0x7f0680160770 2024-03-29T09:10:43.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.421+0000 7f067cc00700 1 -- 172.21.15.99:0/1299383593 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f06800a06c0 con 0x7f0680160770 2024-03-29T09:10:43.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.426+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f066006ac90 msgr2=0x7f066006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.426+0000 7f0687a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f066006ac90 0x7f066006d150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0674000f60 tx=0x7f0674009040 comp rx=0 tx=0).stop 2024-03-29T09:10:43.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.426+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680160770 msgr2=0x7f06801f3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.426+0000 7f0687a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680160770 0x7f06801f3150 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0678013000 tx=0x7f0678004ad0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.429+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 shutdown_connections 2024-03-29T09:10:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.429+0000 7f0687a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f066006ac90 0x7f066006d150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.429+0000 7f0687a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0680160770 0x7f06801f3150 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.429+0000 7f0687a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06800b3cc0 0x7f06801547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.429+0000 7f0687a00700 1 --2- 172.21.15.99:0/1299383593 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06800cf090 0x7f06800a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.429+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 >> 172.21.15.99:0/1299383593 conn(0x7f068008afd0 msgr2=0x7f068006b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:43.432 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.432+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 shutdown_connections 2024-03-29T09:10:43.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.434+0000 7f0687a00700 1 -- 172.21.15.99:0/1299383593 wait complete. 2024-03-29T09:10:43.441 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-03-29T09:10:43.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T09:10:43.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.659+0000 7f56f5e00700 1 Processor -- start 2024-03-29T09:10:43.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.659+0000 7f56f5e00700 1 -- start start 2024-03-29T09:10:43.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56f5e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56f00d1760 0x7f56f00d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56f5e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 0x7f56f00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56f5e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0161ae0 con 0x7f56f00cdcb0 2024-03-29T09:10:43.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56f5e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f0160f10 con 0x7f56f00d1760 2024-03-29T09:10:43.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56f5e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00d2080 con 0x7f56f00cfb00 2024-03-29T09:10:43.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56ee000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f56f00cdcb0 0x7f56f00d0d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47972/0 (socket says 172.21.15.99:47972) 2024-03-29T09:10:43.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56ee000700 1 -- 172.21.15.99:0/3183353708 learned_addr learned my addr 172.21.15.99:0/3183353708 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:43.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56eea00700 1 --2- 172.21.15.99:0/3183353708 >> v2:172.21.15.133:3300/0 conn(0x7f56f00d1760 0x7f56f00d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56ef400700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 0x7f56f00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.660+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727389089 0 0) 0x7f56f0161ae0 con 0x7f56f00cdcb0 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d40035b0 con 0x7f56f00cdcb0 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2252072179 0 0) 0x7f56d40035b0 con 0x7f56f00cdcb0 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 msgr2=0x7f56f00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 0x7f56f00cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 >> v2:172.21.15.133:3300/0 conn(0x7f56f00d1760 msgr2=0x7f56f00d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 --2- 172.21.15.99:0/3183353708 >> v2:172.21.15.133:3300/0 conn(0x7f56f00d1760 0x7f56f00d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f01611b0 con 0x7f56f00cdcb0 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f56dc001670 con 0x7f56f00cdcb0 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 >> v1:172.21.15.99:6789/0 conn(0x7f56f00cdcb0 legacy=0x7f56f00d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 0x7f56d4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d4004d90 0x7f56d4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0160f10 con 0x7f56d4003f80 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00d2080 con 0x7f56d4004d90 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d400ba10 con 0x7f56f00cfb00 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56eea00700 1 --2- 172.21.15.99:0/3183353708 >> v2:172.21.15.133:3300/0 conn(0x7f56f00d1760 0x7f56f00d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56ee000700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ef400700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d4004d90 0x7f56d4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.661+0000 7f56eea00700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 0x7f56d4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ee000700 1 -- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 msgr2=0x7f56d4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ee000700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00cfb00 0x7f56d4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ee000700 1 -- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d4004d90 msgr2=0x7f56d4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ee000700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56d4004d90 0x7f56d4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ee000700 1 -- 172.21.15.99:0/3183353708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40035b0 con 0x7f56d4003f80 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ee000700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56d4004850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00c320 tx=0x7f56d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1125a125368e7cb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00ce00 con 0x7f56d4003f80 2024-03-29T09:10:43.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc005960 con 0x7f56d4003f80 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.662+0000 7f56ed600700 1 -- 172.21.15.99:0/3183353708 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc005d10 con 0x7f56d4003f80 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.663+0000 7f56f5e00700 1 -- 172.21.15.99:0/3183353708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 msgr2=0x7f56d4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.663+0000 7f56f5e00700 1 --2- 172.21.15.99:0/3183353708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56d4004850 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00c320 tx=0x7f56d4005880 comp rx=0 tx=0).stop 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.663+0000 7f56ef400700 1 -- 172.21.15.99:0/3183353708 reap_dead start 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.663+0000 7f56f5e00700 1 -- 172.21.15.99:0/3183353708 shutdown_connections 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.663+0000 7f56f5e00700 1 -- 172.21.15.99:0/3183353708 >> 172.21.15.99:0/3183353708 conn(0x7f56f008afd0 msgr2=0x7f56f009c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.663+0000 7f56f5e00700 1 -- 172.21.15.99:0/3183353708 shutdown_connections 2024-03-29T09:10:43.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.663+0000 7f56f5e00700 1 -- 172.21.15.99:0/3183353708 wait complete. 2024-03-29T09:10:43.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.664+0000 7f56f5e00700 1 Processor -- start 2024-03-29T09:10:43.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.665+0000 7f56f5e00700 1 -- start start 2024-03-29T09:10:43.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.665+0000 7f56f5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56f01f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56f5e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00ca270 0x7f56f00ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56f5e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f00b0080 0x7f56f00b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56f5e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00cac30 con 0x7f56d4003f80 2024-03-29T09:10:43.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56f5e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f01f3c00 con 0x7f56f00b0080 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56f5e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00b0db0 con 0x7f56f00ca270 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56f01f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56f01f36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35764/0 (socket says 172.21.15.99:35764) 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 -- 172.21.15.99:0/4042498766 learned_addr learned my addr 172.21.15.99:0/4042498766 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56ee000700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00ca270 0x7f56f00ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56ef400700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f00b0080 0x7f56f00b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 -- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00ca270 msgr2=0x7f56f00ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00ca270 0x7f56f00ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 -- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f00b0080 msgr2=0x7f56f00b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f00b0080 0x7f56f00b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.666+0000 7f56eea00700 1 -- 172.21.15.99:0/4042498766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f00b1090 con 0x7f56d4003f80 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.667+0000 7f56eea00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56f01f36c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f56e0010a50 tx=0x7f56e0010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.667+0000 7f56e7400700 1 -- 172.21.15.99:0/4042498766 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00097a0 con 0x7f56d4003f80 2024-03-29T09:10:43.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.667+0000 7f56e7400700 1 -- 172.21.15.99:0/4042498766 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e000e8d0 con 0x7f56d4003f80 2024-03-29T09:10:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.667+0000 7f56e7400700 1 -- 172.21.15.99:0/4042498766 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e000eca0 con 0x7f56d4003f80 2024-03-29T09:10:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.667+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f00b1370 con 0x7f56d4003f80 2024-03-29T09:10:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.667+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f01f3dd0 con 0x7f56d4003f80 2024-03-29T09:10:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.668+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f01f4140 con 0x7f56d4003f80 2024-03-29T09:10:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.670+0000 7f56e7400700 1 -- 172.21.15.99:0/4042498766 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f56e0016460 con 0x7f56d4003f80 2024-03-29T09:10:43.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.671+0000 7f56e7400700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f56d006ac90 0x7f56d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:43.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.671+0000 7f56e7400700 1 -- 172.21.15.99:0/4042498766 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f56e0089f80 con 0x7f56d4003f80 2024-03-29T09:10:43.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.671+0000 7f56ee000700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f56d006ac90 0x7f56d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:43.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.672+0000 7f56ee000700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f56d006ac90 0x7f56d006d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56dc000c00 tx=0x7f56dc00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:43.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.675+0000 7f56e7400700 1 -- 172.21.15.99:0/4042498766 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f56e0055160 con 0x7f56d4003f80 2024-03-29T09:10:43.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.921+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f56f00678b0 con 0x7f56d4003f80 2024-03-29T09:10:43.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.928+0000 7f56e7400700 1 -- 172.21.15.99:0/4042498766 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f56e0058780 con 0x7f56d4003f80 2024-03-29T09:10:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.940+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f56d006ac90 msgr2=0x7f56d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.940+0000 7f56f5e00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f56d006ac90 0x7f56d006d150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56dc000c00 tx=0x7f56dc00a000 comp rx=0 tx=0).stop 2024-03-29T09:10:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.940+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 msgr2=0x7f56f01f36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:43.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.940+0000 7f56f5e00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56f01f36c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f56e0010a50 tx=0x7f56e0010d60 comp rx=0 tx=0).stop 2024-03-29T09:10:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.941+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 shutdown_connections 2024-03-29T09:10:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.941+0000 7f56f5e00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f56d006ac90 0x7f56d006d150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.941+0000 7f56f5e00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56f00b0080 0x7f56f00b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.941+0000 7f56f5e00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56d4003f80 0x7f56f01f36c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.941+0000 7f56f5e00700 1 --2- 172.21.15.99:0/4042498766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56f00ca270 0x7f56f00ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.941+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 >> 172.21.15.99:0/4042498766 conn(0x7f56f008afd0 msgr2=0x7f56f006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.942+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 shutdown_connections 2024-03-29T09:10:43.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:43.942+0000 7f56f5e00700 1 -- 172.21.15.99:0/4042498766 wait complete. 2024-03-29T09:10:43.949 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 5 and data pool 6 2024-03-29T09:10:43.974 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:44.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.163+0000 7f95a7200700 1 Processor -- start 2024-03-29T09:10:44.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.164+0000 7f95a7200700 1 -- start start 2024-03-29T09:10:44.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f95a7200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 0x7f95a00cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f95a7200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a00afde0 con 0x7f95a015da70 2024-03-29T09:10:44.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f95a7200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a00d01d0 con 0x7f95a015d690 2024-03-29T09:10:44.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f95a7200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00d04b0 con 0x7f95a00d1e50 2024-03-29T09:10:44.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f959f400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f95a015da70 0x7f95a00d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47988/0 (socket says 172.21.15.99:47988) 2024-03-29T09:10:44.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f959f400700 1 -- 172.21.15.99:0/1336708975 learned_addr learned my addr 172.21.15.99:0/1336708975 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747704514 0 0) 0x7f95a00afde0 con 0x7f95a015da70 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f95a5200700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 0x7f95a00cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.165+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95840035b0 con 0x7f95a015da70 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775827094 0 0) 0x7f95840035b0 con 0x7f95a015da70 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 msgr2=0x7f95a00cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 0x7f95a00cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 >> v1:172.21.15.133:6789/0 conn(0x7f95a015d690 legacy=0x7f95a00cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00b0030 con 0x7f95a015da70 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f958c001670 con 0x7f95a015da70 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 >> v1:172.21.15.99:6789/0 conn(0x7f95a015da70 legacy=0x7f95a00d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 0x7f9584003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f9584004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584004d90 0x7f9584005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00d04b0 con 0x7f9584003f70 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958400bc00 con 0x7f9584004d90 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958400bdc0 con 0x7f95a00d1e50 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959f400700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f9584004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f959fe00700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 0x7f9584003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.166+0000 7f95a5200700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584004d90 0x7f9584005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959f400700 1 -- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 msgr2=0x7f9584003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959f400700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00d1e50 0x7f9584003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959f400700 1 -- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584004d90 msgr2=0x7f9584005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959f400700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584004d90 0x7f9584005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959f400700 1 -- 172.21.15.99:0/1336708975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840035b0 con 0x7f9584003f70 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959f400700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f9584004850 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c210 tx=0x7f9584005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527265de5f33a8fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:44.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c007b70 con 0x7f9584003f70 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c00cdb0 con 0x7f9584003f70 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f959ea00700 1 -- 172.21.15.99:0/1336708975 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c005ba0 con 0x7f9584003f70 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f95a7200700 1 -- 172.21.15.99:0/1336708975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 msgr2=0x7f9584004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.167+0000 7f95a7200700 1 --2- 172.21.15.99:0/1336708975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f9584004850 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f958c00c210 tx=0x7f9584005880 comp rx=0 tx=0).stop 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.168+0000 7f95a5200700 1 -- 172.21.15.99:0/1336708975 reap_dead start 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.168+0000 7f95a7200700 1 -- 172.21.15.99:0/1336708975 shutdown_connections 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.168+0000 7f95a7200700 1 -- 172.21.15.99:0/1336708975 >> 172.21.15.99:0/1336708975 conn(0x7f95a009c480 msgr2=0x7f95a00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.168+0000 7f95a7200700 1 -- 172.21.15.99:0/1336708975 shutdown_connections 2024-03-29T09:10:44.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.168+0000 7f95a7200700 1 -- 172.21.15.99:0/1336708975 wait complete. 2024-03-29T09:10:44.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.169+0000 7f95a7200700 1 Processor -- start 2024-03-29T09:10:44.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.170+0000 7f95a7200700 1 -- start start 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.170+0000 7f95a7200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f95a00b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.170+0000 7f95a7200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00c9c80 0x7f95a00b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f95a7200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00c8b70 0x7f95a00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f95a7200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00b8c60 con 0x7f9584003f70 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f95a7200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00b73f0 con 0x7f95a00c9c80 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f95a7200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a00c7df0 con 0x7f95a00c8b70 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f95a00b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f95a00b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35776/0 (socket says 172.21.15.99:35776) 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 -- 172.21.15.99:0/1370521816 learned_addr learned my addr 172.21.15.99:0/1370521816 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959f400700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00c9c80 0x7f95a00b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f95a5200700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00c8b70 0x7f95a00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 -- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00c8b70 msgr2=0x7f95a00b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00c8b70 0x7f95a00b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 -- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00c9c80 msgr2=0x7f95a00b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00c9c80 0x7f95a00b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 -- 172.21.15.99:0/1370521816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a00b8980 con 0x7f9584003f70 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.171+0000 7f959fe00700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f95a00b6eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f959000f820 tx=0x7f959000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.172+0000 7f9597e00700 1 -- 172.21.15.99:0/1370521816 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95900052e0 con 0x7f9584003f70 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.172+0000 7f9597e00700 1 -- 172.21.15.99:0/1370521816 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9590005460 con 0x7f9584003f70 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.172+0000 7f9597e00700 1 -- 172.21.15.99:0/1370521816 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95900038c0 con 0x7f9584003f70 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.172+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a00c8090 con 0x7f9584003f70 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.172+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a00c8650 con 0x7f9584003f70 2024-03-29T09:10:44.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.173+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a00677a0 con 0x7f9584003f70 2024-03-29T09:10:44.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.174+0000 7f9597e00700 1 -- 172.21.15.99:0/1370521816 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9590003a20 con 0x7f9584003f70 2024-03-29T09:10:44.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.174+0000 7f9597e00700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958006acc0 0x7f958006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.175+0000 7f9597e00700 1 -- 172.21.15.99:0/1370521816 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9590089440 con 0x7f9584003f70 2024-03-29T09:10:44.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.175+0000 7f959f400700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958006acc0 0x7f958006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.175+0000 7f959f400700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958006acc0 0x7f958006d180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f958c015040 tx=0x7f958c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:44.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.179+0000 7f9597e00700 1 -- 172.21.15.99:0/1370521816 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9590054450 con 0x7f9584003f70 2024-03-29T09:10:44.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.382+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95a00b8ee0 con 0x7f9584003f70 2024-03-29T09:10:44.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.383+0000 7f9597e00700 1 -- 172.21.15.99:0/1370521816 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16319 (secure 0 0 0) 0x7f9590057a70 con 0x7f9584003f70 2024-03-29T09:10:44.384 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:44.384 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":38,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:43.923325+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:44.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958006acc0 msgr2=0x7f958006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958006acc0 0x7f958006d180 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f958c015040 tx=0x7f958c007000 comp rx=0 tx=0).stop 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 msgr2=0x7f95a00b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f95a00b6eb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f959000f820 tx=0x7f959000fbe0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 shutdown_connections 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f958006acc0 0x7f958006d180 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95a00c8b70 0x7f95a00b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9584003f70 0x7f95a00b6eb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 --2- 172.21.15.99:0/1370521816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95a00c9c80 0x7f95a00b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 >> 172.21.15.99:0/1370521816 conn(0x7f95a009c480 msgr2=0x7f95a00ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 shutdown_connections 2024-03-29T09:10:44.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.386+0000 7f95a7200700 1 -- 172.21.15.99:0/1370521816 wait complete. 2024-03-29T09:10:44.405 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:44.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.594+0000 7ffbcd200700 1 Processor -- start 2024-03-29T09:10:44.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.594+0000 7ffbcd200700 1 -- start start 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.595+0000 7ffbcd200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffbc81608e0 0x7ffbc80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.595+0000 7ffbcd200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.595+0000 7ffbcd200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc80cddc0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.595+0000 7ffbcd200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc80ce0a0 con 0x7ffbc81608e0 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbcd200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80ce380 con 0x7ffbc8160cc0 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc6800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7ffbc80d3a70 0x7ffbc80d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48002/0 (socket says 172.21.15.99:48002) 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc6800700 1 -- 172.21.15.99:0/938792357 learned_addr learned my addr 172.21.15.99:0/938792357 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc5400700 1 --2- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.596 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc5e00700 1 --2- 172.21.15.99:0/938792357 >> v2:172.21.15.133:3300/0 conn(0x7ffbc81608e0 0x7ffbc80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc4a00700 1 -- 172.21.15.99:0/938792357 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866904577 0 0) 0x7ffbc80cddc0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc4a00700 1 -- 172.21.15.99:0/938792357 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbac0035b0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc4a00700 1 -- 172.21.15.99:0/938792357 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2060428293 0 0) 0x7ffbac0035b0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.596+0000 7ffbc5e00700 1 -- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 msgr2=0x7ffbc80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:44.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.597+0000 7ffbc5e00700 1 --2- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.597+0000 7ffbc5e00700 1 -- 172.21.15.99:0/938792357 >> v1:172.21.15.99:6789/0 conn(0x7ffbc80d3a70 legacy=0x7ffbc80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.597+0000 7ffbc5e00700 1 -- 172.21.15.99:0/938792357 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc80ce660 con 0x7ffbc81608e0 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.597+0000 7ffbc5e00700 1 --2- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 0x7ffbc80d0150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009840 tx=0x7ffbb8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9200cb889c641043 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.597+0000 7ffbc4a00700 1 -- 172.21.15.99:0/938792357 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00019f0 con 0x7ffbc81608e0 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.597+0000 7ffbc4a00700 1 -- 172.21.15.99:0/938792357 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb8004030 con 0x7ffbc81608e0 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.597+0000 7ffbc4a00700 1 -- 172.21.15.99:0/938792357 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80043c0 con 0x7ffbc81608e0 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 -- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 msgr2=0x7ffbc80d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 --2- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 0x7ffbc80d0150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009840 tx=0x7ffbb8009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 -- 172.21.15.99:0/938792357 shutdown_connections 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 --2- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 0x7ffbc80d0150 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 --2- 172.21.15.99:0/938792357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 -- 172.21.15.99:0/938792357 >> 172.21.15.99:0/938792357 conn(0x7ffbc809c490 msgr2=0x7ffbc80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 -- 172.21.15.99:0/938792357 shutdown_connections 2024-03-29T09:10:44.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.598+0000 7ffbcd200700 1 -- 172.21.15.99:0/938792357 wait complete. 2024-03-29T09:10:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.600+0000 7ffbcd200700 1 Processor -- start 2024-03-29T09:10:44.600 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.600+0000 7ffbcd200700 1 -- start start 2024-03-29T09:10:44.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.601+0000 7ffbcd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc80d3a70 0x7ffbc815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.601+0000 7ffbcd200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 0x7ffbc80cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbcd200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbcd200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc8159950 con 0x7ffbc80d3a70 2024-03-29T09:10:44.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbcd200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc8159c30 con 0x7ffbc81608e0 2024-03-29T09:10:44.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbcd200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80ccbc0 con 0x7ffbc8160cc0 2024-03-29T09:10:44.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc80d3a70 0x7ffbc815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc80d3a70 0x7ffbc815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35792/0 (socket says 172.21.15.99:35792) 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 -- 172.21.15.99:0/1732472016 learned_addr learned my addr 172.21.15.99:0/1732472016 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc6800700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5400700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 0x7ffbc80cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 -- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 msgr2=0x7ffbc80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 -- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 msgr2=0x7ffbc80cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 0x7ffbc80cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 -- 172.21.15.99:0/1732472016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80094a0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc6800700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.602+0000 7ffbc5e00700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc80d3a70 0x7ffbc815ba50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80090b0 tx=0x7ffbb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.603+0000 7ffbbe000700 1 -- 172.21.15.99:0/1732472016 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8003d20 con 0x7ffbc80d3a70 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.603+0000 7ffbbe000700 1 -- 172.21.15.99:0/1732472016 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb801f070 con 0x7ffbc80d3a70 2024-03-29T09:10:44.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.603+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc80ccea0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.603+0000 7ffbbe000700 1 -- 172.21.15.99:0/1732472016 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8003ea0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.603+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc815c4a0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.604+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc80677a0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.605+0000 7ffbbe000700 1 -- 172.21.15.99:0/1732472016 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffbb8011870 con 0x7ffbc80d3a70 2024-03-29T09:10:44.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.606+0000 7ffbbe000700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffba806ac90 0x7ffba806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:44.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.606+0000 7ffbbe000700 1 -- 172.21.15.99:0/1732472016 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffbb8021070 con 0x7ffbc80d3a70 2024-03-29T09:10:44.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.606+0000 7ffbc5400700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffba806ac90 0x7ffba806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:44.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.607+0000 7ffbc5400700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffba806ac90 0x7ffba806d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80d10d0 tx=0x7ffbb4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:44.610 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.610+0000 7ffbbe000700 1 -- 172.21.15.99:0/1732472016 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffbb8055680 con 0x7ffbc80d3a70 2024-03-29T09:10:44.812 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.812+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbc80d0700 con 0x7ffbc80d3a70 2024-03-29T09:10:44.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.813+0000 7ffbbe000700 1 -- 172.21.15.99:0/1732472016 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16319 (secure 0 0 0) 0x7ffbb8058ca0 con 0x7ffbc80d3a70 2024-03-29T09:10:44.814 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:44.814 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":38,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:43.923325+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffba806ac90 msgr2=0x7ffba806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffba806ac90 0x7ffba806d150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80d10d0 tx=0x7ffbb4009210 comp rx=0 tx=0).stop 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc80d3a70 msgr2=0x7ffbc815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc80d3a70 0x7ffbc815ba50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80090b0 tx=0x7ffbb8004750 comp rx=0 tx=0).stop 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 shutdown_connections 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffba806ac90 0x7ffba806d150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbc8160cc0 0x7ffbc80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbc80d3a70 0x7ffbc815ba50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 --2- 172.21.15.99:0/1732472016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbc81608e0 0x7ffbc80cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:44.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 >> 172.21.15.99:0/1732472016 conn(0x7ffbc809c490 msgr2=0x7ffbc80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 shutdown_connections 2024-03-29T09:10:44.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:44.816+0000 7ffbcd200700 1 -- 172.21.15.99:0/1732472016 wait complete. 2024-03-29T09:10:44.834 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T09:10:45.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.028+0000 7fab8c000700 1 Processor -- start 2024-03-29T09:10:45.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.029+0000 7fab8c000700 1 -- start start 2024-03-29T09:10:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab8c000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab840d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab8c000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab840d3380 0x7fab840ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab8c000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab840d3810 con 0x7fab84160770 2024-03-29T09:10:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab8c000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab840b4140 con 0x7fab840d3380 2024-03-29T09:10:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab8c000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab840d1520 con 0x7fab840b3cc0 2024-03-29T09:10:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab89600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fab84160770 0x7fab840d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48004/0 (socket says 172.21.15.99:48004) 2024-03-29T09:10:45.031 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab89600700 1 -- 172.21.15.99:0/2337765904 learned_addr learned my addr 172.21.15.99:0/2337765904 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab88c00700 1 --2- 172.21.15.99:0/2337765904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab840d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab8a000700 1 --2- 172.21.15.99:0/2337765904 >> v2:172.21.15.133:3300/0 conn(0x7fab840d3380 0x7fab840ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.030+0000 7fab83e00700 1 -- 172.21.15.99:0/2337765904 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283748176 0 0) 0x7fab840d3810 con 0x7fab84160770 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab83e00700 1 -- 172.21.15.99:0/2337765904 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab6c0035b0 con 0x7fab84160770 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab88c00700 1 -- 172.21.15.99:0/2337765904 >> v2:172.21.15.133:3300/0 conn(0x7fab840d3380 msgr2=0x7fab840ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab88c00700 1 --2- 172.21.15.99:0/2337765904 >> v2:172.21.15.133:3300/0 conn(0x7fab840d3380 0x7fab840ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab88c00700 1 -- 172.21.15.99:0/2337765904 >> v1:172.21.15.99:6789/0 conn(0x7fab84160770 legacy=0x7fab840d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab88c00700 1 -- 172.21.15.99:0/2337765904 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840d1800 con 0x7fab840b3cc0 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab88c00700 1 --2- 172.21.15.99:0/2337765904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab840d2e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab70009840 tx=0x7fab70009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72dd20225b9696f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab83e00700 1 -- 172.21.15.99:0/2337765904 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70011830 con 0x7fab840b3cc0 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab83e00700 1 -- 172.21.15.99:0/2337765904 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab70011990 con 0x7fab840b3cc0 2024-03-29T09:10:45.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.031+0000 7fab83e00700 1 -- 172.21.15.99:0/2337765904 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab70011d20 con 0x7fab840b3cc0 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 -- 172.21.15.99:0/2337765904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 msgr2=0x7fab840d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 --2- 172.21.15.99:0/2337765904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab840d2e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab70009840 tx=0x7fab70009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 -- 172.21.15.99:0/2337765904 shutdown_connections 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 --2- 172.21.15.99:0/2337765904 >> v2:172.21.15.133:3300/0 conn(0x7fab840d3380 0x7fab840ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 --2- 172.21.15.99:0/2337765904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab840d2e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 -- 172.21.15.99:0/2337765904 >> 172.21.15.99:0/2337765904 conn(0x7fab8408afd0 msgr2=0x7fab8409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 -- 172.21.15.99:0/2337765904 shutdown_connections 2024-03-29T09:10:45.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.032+0000 7fab8c000700 1 -- 172.21.15.99:0/2337765904 wait complete. 2024-03-29T09:10:45.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.034+0000 7fab8c000700 1 Processor -- start 2024-03-29T09:10:45.034 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.034+0000 7fab8c000700 1 -- start start 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8c000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab8409dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8c000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab840d3380 0x7fab840b7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8c000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab84160770 0x7fab84154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8c000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab841f5d40 con 0x7fab84160770 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8c000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab841f5f90 con 0x7fab840d3380 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8c000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab841f6270 con 0x7fab840b3cc0 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab84160770 0x7fab84154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab84160770 0x7fab84154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35806/0 (socket says 172.21.15.99:35806) 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab8a000700 1 -- 172.21.15.99:0/2653216992 learned_addr learned my addr 172.21.15.99:0/2653216992 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:45.036 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab88c00700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab840d3380 0x7fab840b7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.035+0000 7fab89600700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab8409dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8a000700 1 -- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 msgr2=0x7fab8409dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8a000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab8409dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8a000700 1 -- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab840d3380 msgr2=0x7fab840b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8a000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab840d3380 0x7fab840b7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8a000700 1 -- 172.21.15.99:0/2653216992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab700094a0 con 0x7fab84160770 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8a000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab84160770 0x7fab84154180 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00efc0 tx=0x7fab7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab81600700 1 -- 172.21.15.99:0/2653216992 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c009ea0 con 0x7fab84160770 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab81600700 1 -- 172.21.15.99:0/2653216992 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c00f040 con 0x7fab84160770 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab841f6550 con 0x7fab84160770 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab81600700 1 -- 172.21.15.99:0/2653216992 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00ced0 con 0x7fab84160770 2024-03-29T09:10:45.037 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.036+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8409e2c0 con 0x7fab84160770 2024-03-29T09:10:45.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.037+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab840678b0 con 0x7fab84160770 2024-03-29T09:10:45.038 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.038+0000 7fab81600700 1 -- 172.21.15.99:0/2653216992 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fab7c014480 con 0x7fab84160770 2024-03-29T09:10:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.039+0000 7fab81600700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab6406ac90 0x7fab6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.039+0000 7fab81600700 1 -- 172.21.15.99:0/2653216992 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fab7c089c70 con 0x7fab84160770 2024-03-29T09:10:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.039+0000 7fab89600700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab6406ac90 0x7fab6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:45.039 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.040+0000 7fab89600700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab6406ac90 0x7fab6406d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab78007fd0 tx=0x7fab78007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:45.043 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.043+0000 7fab81600700 1 -- 172.21.15.99:0/2653216992 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fab7c054d00 con 0x7fab84160770 2024-03-29T09:10:45.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.297+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fab840cf3c0 con 0x7fab84160770 2024-03-29T09:10:45.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.935+0000 7fab81600700 1 -- 172.21.15.99:0/2653216992 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fab7c018020 con 0x7fab84160770 2024-03-29T09:10:45.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab6406ac90 msgr2=0x7fab6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:45.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab6406ac90 0x7fab6406d150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fab78007fd0 tx=0x7fab78007dc0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab84160770 msgr2=0x7fab84154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:45.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab84160770 0x7fab84154180 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00efc0 tx=0x7fab7c00c3a0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 shutdown_connections 2024-03-29T09:10:45.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab6406ac90 0x7fab6406d150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab84160770 0x7fab84154180 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab840b3cc0 0x7fab8409dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 --2- 172.21.15.99:0/2653216992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab840d3380 0x7fab840b7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:45.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 >> 172.21.15.99:0/2653216992 conn(0x7fab8408afd0 msgr2=0x7fab8415f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:45.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 shutdown_connections 2024-03-29T09:10:45.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:45.938+0000 7fab8c000700 1 -- 172.21.15.99:0/2653216992 wait complete. 2024-03-29T09:10:45.958 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T09:10:45.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T09:10:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.154+0000 7f7c0c800700 1 Processor -- start 2024-03-29T09:10:46.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.154+0000 7f7c0c800700 1 -- start start 2024-03-29T09:10:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.155+0000 7f7c0c800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7c04160770 0x7f7c040b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.155+0000 7f7c0c800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c040d3550 0x7f7c040ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.155+0000 7f7c0c800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c040d1520 con 0x7f7c040d0d60 2024-03-29T09:10:46.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.155+0000 7f7c0c800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c040d1800 con 0x7f7c04160770 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.155+0000 7f7c0c800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c040d1ae0 con 0x7f7c040d3550 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.155+0000 7f7c09e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7c04160770 0x7f7c040b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c09400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7c040d0d60 0x7f7c040d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48018/0 (socket says 172.21.15.99:48018) 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c09400700 1 -- 172.21.15.99:0/2621972464 learned_addr learned my addr 172.21.15.99:0/2621972464 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c0a800700 1 --2- 172.21.15.99:0/2621972464 >> v2:172.21.15.133:3300/0 conn(0x7f7c040d3550 0x7f7c040ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c09e00700 1 -- 172.21.15.99:0/2621972464 >> v2:172.21.15.133:3300/0 conn(0x7f7c040d3550 msgr2=0x7f7c040ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c09e00700 1 --2- 172.21.15.99:0/2621972464 >> v2:172.21.15.133:3300/0 conn(0x7f7c040d3550 0x7f7c040ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c09e00700 1 -- 172.21.15.99:0/2621972464 >> v1:172.21.15.99:6789/0 conn(0x7f7c040d0d60 legacy=0x7f7c040d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c09e00700 1 -- 172.21.15.99:0/2621972464 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c040d1dc0 con 0x7f7c04160770 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c08a00700 1 -- 172.21.15.99:0/2621972464 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929464190 0 0) 0x7f7c040d1520 con 0x7f7c040d0d60 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c09e00700 1 --2- 172.21.15.99:0/2621972464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b3cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c040b3230 tx=0x7f7bf8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee19af05f800701a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c08a00700 1 -- 172.21.15.99:0/2621972464 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8011830 con 0x7f7c04160770 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c08a00700 1 -- 172.21.15.99:0/2621972464 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf8011990 con 0x7f7c04160770 2024-03-29T09:10:46.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.156+0000 7f7c08a00700 1 -- 172.21.15.99:0/2621972464 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8011d20 con 0x7f7c04160770 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 -- 172.21.15.99:0/2621972464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 msgr2=0x7f7c040b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 --2- 172.21.15.99:0/2621972464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b3cc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c040b3230 tx=0x7f7bf8009b20 comp rx=0 tx=0).stop 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 -- 172.21.15.99:0/2621972464 shutdown_connections 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 --2- 172.21.15.99:0/2621972464 >> v2:172.21.15.133:3300/0 conn(0x7f7c040d3550 0x7f7c040ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 --2- 172.21.15.99:0/2621972464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b3cc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 -- 172.21.15.99:0/2621972464 >> 172.21.15.99:0/2621972464 conn(0x7f7c0408afd0 msgr2=0x7f7c0409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 -- 172.21.15.99:0/2621972464 shutdown_connections 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.157+0000 7f7c0c800700 1 -- 172.21.15.99:0/2621972464 wait complete. 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.159+0000 7f7c0c800700 1 Processor -- start 2024-03-29T09:10:46.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.159+0000 7f7c0c800700 1 -- start start 2024-03-29T09:10:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0c800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c040d0d60 0x7f7c040afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0c800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c040d3550 0x7f7c041f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0c800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0c800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040b16b0 con 0x7f7c04160770 2024-03-29T09:10:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0c800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040b0610 con 0x7f7c040d0d60 2024-03-29T09:10:46.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0c800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c040b08f0 con 0x7f7c040d3550 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35830/0 (socket says 172.21.15.99:35830) 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c0a800700 1 -- 172.21.15.99:0/4013111465 learned_addr learned my addr 172.21.15.99:0/4013111465 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c09400700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c040d3550 0x7f7c041f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.160+0000 7f7c09e00700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c040d0d60 0x7f7c040afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0a800700 1 -- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c040d3550 msgr2=0x7f7c041f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0a800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c040d3550 0x7f7c041f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0a800700 1 -- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c040d0d60 msgr2=0x7f7c040afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0a800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c040d0d60 0x7f7c040afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0a800700 1 -- 172.21.15.99:0/4013111465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf80094a0 con 0x7f7c04160770 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0a800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b8200 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7c00007e30 tx=0x7f7c0000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7bfe000700 1 -- 172.21.15.99:0/4013111465 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0000ed30 con 0x7f7c04160770 2024-03-29T09:10:46.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7bfe000700 1 -- 172.21.15.99:0/4013111465 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0000ee90 con 0x7f7c04160770 2024-03-29T09:10:46.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c040b0bd0 con 0x7f7c04160770 2024-03-29T09:10:46.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7bfe000700 1 -- 172.21.15.99:0/4013111465 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c00014660 con 0x7f7c04160770 2024-03-29T09:10:46.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.161+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c040b1190 con 0x7f7c04160770 2024-03-29T09:10:46.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.162+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c040678b0 con 0x7f7c04160770 2024-03-29T09:10:46.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.163+0000 7f7bfe000700 1 -- 172.21.15.99:0/4013111465 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7c000147c0 con 0x7f7c04160770 2024-03-29T09:10:46.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.164+0000 7f7bfe000700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7be806ac90 0x7f7be806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:46.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.164+0000 7f7bfe000700 1 -- 172.21.15.99:0/4013111465 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7c00089a40 con 0x7f7c04160770 2024-03-29T09:10:46.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.164+0000 7f7c09e00700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7be806ac90 0x7f7be806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:46.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.165+0000 7f7c09e00700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7be806ac90 0x7f7be806d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80090b0 tx=0x7f7bf801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:46.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.168+0000 7f7bfe000700 1 -- 172.21.15.99:0/4013111465 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7c0008be60 con 0x7f7c04160770 2024-03-29T09:10:46.388 INFO:tasks.ceph.mds.a.smithi099.stderr:2024-03-29T09:10:46.388+0000 7f1109600700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T09:10:46.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.419+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7c04073230 con 0x7f7c04160770 2024-03-29T09:10:46.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.937+0000 7f7bfe000700 1 -- 172.21.15.99:0/4013111465 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7c00054b70 con 0x7f7c04160770 2024-03-29T09:10:46.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7be806ac90 msgr2=0x7f7be806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:46.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7be806ac90 0x7f7be806d150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80090b0 tx=0x7f7bf801c040 comp rx=0 tx=0).stop 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 msgr2=0x7f7c040b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b8200 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7c00007e30 tx=0x7f7c0000e3c0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 shutdown_connections 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7be806ac90 0x7f7be806d150 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c04160770 0x7f7c040b8200 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c040d0d60 0x7f7c040afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 --2- 172.21.15.99:0/4013111465 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c040d3550 0x7f7c041f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 >> 172.21.15.99:0/4013111465 conn(0x7f7c0408afd0 msgr2=0x7f7c0415f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 shutdown_connections 2024-03-29T09:10:46.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:46.940+0000 7f7c0c800700 1 -- 172.21.15.99:0/4013111465 wait complete. 2024-03-29T09:10:46.960 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T09:10:46.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:10:47.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.150+0000 7fb285000700 1 Processor -- start 2024-03-29T09:10:47.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.150+0000 7fb285000700 1 -- start start 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.151+0000 7fb285000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb280143940 0x7fb280070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.151+0000 7fb285000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb2800ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.151+0000 7fb285000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2800cec50 con 0x7fb280070ca0 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.151+0000 7fb285000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb280143d20 con 0x7fb280143940 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.151+0000 7fb285000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28015eaf0 con 0x7fb2800ce2c0 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.151+0000 7fb27de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb280143940 0x7fb280070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb280143940 0x7fb280070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35834/0 (socket says 172.21.15.99:35834) 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27de00700 1 -- 172.21.15.99:0/3111789002 learned_addr learned my addr 172.21.15.99:0/3111789002 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27d400700 1 --1- 172.21.15.99:0/3111789002 >> v1:172.21.15.99:6789/0 conn(0x7fb280070ca0 0x7fb28015db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48030/0 (socket says 172.21.15.99:48030) 2024-03-29T09:10:47.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27e800700 1 --2- 172.21.15.99:0/3111789002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb2800ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27de00700 1 -- 172.21.15.99:0/3111789002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 msgr2=0x7fb2800ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27de00700 1 --2- 172.21.15.99:0/3111789002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb2800ce710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27de00700 1 -- 172.21.15.99:0/3111789002 >> v1:172.21.15.99:6789/0 conn(0x7fb280070ca0 legacy=0x7fb28015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27de00700 1 -- 172.21.15.99:0/3111789002 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800cd820 con 0x7fb280143940 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27e800700 1 --2- 172.21.15.99:0/3111789002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb2800ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27ca00700 1 -- 172.21.15.99:0/3111789002 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596481513 0 0) 0x7fb2800cec50 con 0x7fb280070ca0 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27de00700 1 --2- 172.21.15.99:0/3111789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb280070760 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb2700098f0 tx=0x7fb270009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8130f66d666a0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.152+0000 7fb27ca00700 1 -- 172.21.15.99:0/3111789002 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000b8a0 con 0x7fb280143940 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb27ca00700 1 -- 172.21.15.99:0/3111789002 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27000ba00 con 0x7fb280143940 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb27ca00700 1 -- 172.21.15.99:0/3111789002 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27000bd90 con 0x7fb280143940 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 -- 172.21.15.99:0/3111789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 msgr2=0x7fb280070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 --2- 172.21.15.99:0/3111789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb280070760 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb2700098f0 tx=0x7fb270009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 -- 172.21.15.99:0/3111789002 shutdown_connections 2024-03-29T09:10:47.154 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 --2- 172.21.15.99:0/3111789002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb2800ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 --2- 172.21.15.99:0/3111789002 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb280070760 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 -- 172.21.15.99:0/3111789002 >> 172.21.15.99:0/3111789002 conn(0x7fb280066540 msgr2=0x7fb280066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 -- 172.21.15.99:0/3111789002 shutdown_connections 2024-03-29T09:10:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.153+0000 7fb285000700 1 -- 172.21.15.99:0/3111789002 wait complete. 2024-03-29T09:10:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.155+0000 7fb285000700 1 Processor -- start 2024-03-29T09:10:47.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.155+0000 7fb285000700 1 -- start start 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb285000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb280070ca0 0x7fb2800a9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb285000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb280139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb285000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb28013e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb285000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800abb70 con 0x7fb280143940 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb285000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800aa6c0 con 0x7fb280070ca0 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb285000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800aa100 con 0x7fb2800ce2c0 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb28013e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb28013e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35846/0 (socket says 172.21.15.99:35846) 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 -- 172.21.15.99:0/1891281095 learned_addr learned my addr 172.21.15.99:0/1891281095 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27d400700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb280139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 -- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 msgr2=0x7fb280139710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27de00700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb280070ca0 0x7fb2800a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb280139710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 -- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb280070ca0 msgr2=0x7fb2800a9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb280070ca0 0x7fb2800a9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27e800700 1 -- 172.21.15.99:0/1891281095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270009550 con 0x7fb280143940 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27d400700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb280139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.156+0000 7fb27de00700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb280070ca0 0x7fb2800a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.157+0000 7fb27e800700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb28013e840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb27800cef0 tx=0x7fb27800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.157+0000 7fb276000700 1 -- 172.21.15.99:0/1891281095 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27800ae50 con 0x7fb280143940 2024-03-29T09:10:47.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.157+0000 7fb276000700 1 -- 172.21.15.99:0/1891281095 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278015440 con 0x7fb280143940 2024-03-29T09:10:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.157+0000 7fb276000700 1 -- 172.21.15.99:0/1891281095 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780157f0 con 0x7fb280143940 2024-03-29T09:10:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.157+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2800aa3e0 con 0x7fb280143940 2024-03-29T09:10:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.157+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2800aefb0 con 0x7fb280143940 2024-03-29T09:10:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.158+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2800cd1d0 con 0x7fb280143940 2024-03-29T09:10:47.159 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.159+0000 7fb276000700 1 -- 172.21.15.99:0/1891281095 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb27800d430 con 0x7fb280143940 2024-03-29T09:10:47.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.160+0000 7fb276000700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb26006ac90 0x7fb26006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.160+0000 7fb276000700 1 -- 172.21.15.99:0/1891281095 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb278089ac0 con 0x7fb280143940 2024-03-29T09:10:47.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.160+0000 7fb27de00700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb26006ac90 0x7fb26006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.161+0000 7fb27de00700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb26006ac90 0x7fb26006d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2800ae4d0 tx=0x7fb270008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:47.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.165+0000 7fb276000700 1 -- 172.21.15.99:0/1891281095 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb278054b50 con 0x7fb280143940 2024-03-29T09:10:47.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.410+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2800af1d0 con 0x7fb280143940 2024-03-29T09:10:47.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.410+0000 7fb276000700 1 -- 172.21.15.99:0/1891281095 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4740 (secure 0 0 0) 0x7fb27801a020 con 0x7fb280143940 2024-03-29T09:10:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:47.411 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:10:43.923687+0000","modified":"2024-03-29T09:10:46.934239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/1841077010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1841077010},{"type":"v1","addr":"172.21.15.99:6835","nonce":1841077010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T09:10:47.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb26006ac90 msgr2=0x7fb26006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb26006ac90 0x7fb26006d150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2800ae4d0 tx=0x7fb270008030 comp rx=0 tx=0).stop 2024-03-29T09:10:47.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 msgr2=0x7fb28013e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb28013e840 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb27800cef0 tx=0x7fb27800a340 comp rx=0 tx=0).stop 2024-03-29T09:10:47.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 shutdown_connections 2024-03-29T09:10:47.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb26006ac90 0x7fb26006d150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb280143940 0x7fb28013e840 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb280070ca0 0x7fb2800a9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 --2- 172.21.15.99:0/1891281095 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2800ce2c0 0x7fb280139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 >> 172.21.15.99:0/1891281095 conn(0x7fb280066540 msgr2=0x7fb28015ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:47.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 shutdown_connections 2024-03-29T09:10:47.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.413+0000 7fb285000700 1 -- 172.21.15.99:0/1891281095 wait complete. 2024-03-29T09:10:47.417 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 16 2024-03-29T09:10:47.432 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:47.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.620+0000 7f1e88200700 1 Processor -- start 2024-03-29T09:10:47.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.620+0000 7f1e88200700 1 -- start start 2024-03-29T09:10:47.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.621+0000 7f1e88200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e88200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e800d1b20 0x7f1e800ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e88200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e800cddc0 con 0x7f1e80160cc0 2024-03-29T09:10:47.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e88200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e800ce0a0 con 0x7f1e800d1b20 2024-03-29T09:10:47.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e88200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e800ce380 con 0x7f1e801608e0 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e86200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e800d1b20 0x7f1e800ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e85800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e85800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:37682/0 (socket says 172.21.15.99:37682) 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e85800700 1 -- 172.21.15.99:0/3272220096 learned_addr learned my addr 172.21.15.99:0/3272220096 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.622+0000 7f1e84e00700 1 --1- 172.21.15.99:0/3272220096 >> v1:172.21.15.133:6789/0 conn(0x7f1e80160cc0 0x7f1e800d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:44274/0 (socket says 172.21.15.99:44274) 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e85800700 1 -- 172.21.15.99:0/3272220096 >> v2:172.21.15.133:3300/0 conn(0x7f1e800d1b20 msgr2=0x7f1e800ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e85800700 1 --2- 172.21.15.99:0/3272220096 >> v2:172.21.15.133:3300/0 conn(0x7f1e800d1b20 0x7f1e800ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e85800700 1 -- 172.21.15.99:0/3272220096 >> v1:172.21.15.133:6789/0 conn(0x7f1e80160cc0 legacy=0x7f1e800d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e85800700 1 -- 172.21.15.99:0/3272220096 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800ce660 con 0x7f1e801608e0 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e85800700 1 --2- 172.21.15.99:0/3272220096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800d0150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e740098f0 tx=0x7f1e74009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=494c7a6bd2c1b75b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:47.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e7fe00700 1 -- 172.21.15.99:0/3272220096 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74011830 con 0x7f1e801608e0 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e7fe00700 1 -- 172.21.15.99:0/3272220096 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e74011990 con 0x7f1e801608e0 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.623+0000 7f1e7fe00700 1 -- 172.21.15.99:0/3272220096 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e74011d20 con 0x7f1e801608e0 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 -- 172.21.15.99:0/3272220096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 msgr2=0x7f1e800d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 --2- 172.21.15.99:0/3272220096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800d0150 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1e740098f0 tx=0x7f1e74009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 -- 172.21.15.99:0/3272220096 shutdown_connections 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 --2- 172.21.15.99:0/3272220096 >> v2:172.21.15.133:3300/0 conn(0x7f1e800d1b20 0x7f1e800ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 --2- 172.21.15.99:0/3272220096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800d0150 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 -- 172.21.15.99:0/3272220096 >> 172.21.15.99:0/3272220096 conn(0x7f1e8009c490 msgr2=0x7f1e800d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 -- 172.21.15.99:0/3272220096 shutdown_connections 2024-03-29T09:10:47.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.624+0000 7f1e88200700 1 -- 172.21.15.99:0/3272220096 wait complete. 2024-03-29T09:10:47.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.626+0000 7f1e88200700 1 Processor -- start 2024-03-29T09:10:47.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.626+0000 7f1e88200700 1 -- start start 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e88200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e800d1b20 0x7f1e801f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e88200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e88200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e80160cc0 0x7f1e800ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e88200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e801592b0 con 0x7f1e80160cc0 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e88200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e801586a0 con 0x7f1e800d1b20 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e88200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e80158980 con 0x7f1e801608e0 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e80160cc0 0x7f1e800ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e80160cc0 0x7f1e800ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35862/0 (socket says 172.21.15.99:35862) 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 -- 172.21.15.99:0/757348869 learned_addr learned my addr 172.21.15.99:0/757348869 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e85800700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e800d1b20 0x7f1e801f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e84e00700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 -- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 msgr2=0x7f1e800ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 -- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e800d1b20 msgr2=0x7f1e801f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e800d1b20 0x7f1e801f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e86200700 1 -- 172.21.15.99:0/757348869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74009550 con 0x7f1e80160cc0 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e85800700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e800d1b20 0x7f1e801f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.627+0000 7f1e84e00700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.628+0000 7f1e86200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e80160cc0 0x7f1e800ca760 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800ef90 tx=0x7f1e7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.628+0000 7f1e7d600700 1 -- 172.21.15.99:0/757348869 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78009ca0 con 0x7f1e80160cc0 2024-03-29T09:10:47.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.628+0000 7f1e7d600700 1 -- 172.21.15.99:0/757348869 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7800f040 con 0x7f1e80160cc0 2024-03-29T09:10:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.628+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e80158c60 con 0x7f1e80160cc0 2024-03-29T09:10:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.628+0000 7f1e7d600700 1 -- 172.21.15.99:0/757348869 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78014480 con 0x7f1e80160cc0 2024-03-29T09:10:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.628+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e801f09f0 con 0x7f1e80160cc0 2024-03-29T09:10:47.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.629+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e800677a0 con 0x7f1e80160cc0 2024-03-29T09:10:47.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.630+0000 7f1e7d600700 1 -- 172.21.15.99:0/757348869 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1e780145e0 con 0x7f1e80160cc0 2024-03-29T09:10:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.631+0000 7f1e7d600700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e6006ac90 0x7f1e6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.632+0000 7f1e7d600700 1 -- 172.21.15.99:0/757348869 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1e7808a1c0 con 0x7f1e80160cc0 2024-03-29T09:10:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.632+0000 7f1e85800700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e6006ac90 0x7f1e6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:47.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.632+0000 7f1e85800700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e6006ac90 0x7f1e6006d150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009c40 tx=0x7f1e74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:47.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.636+0000 7f1e7d600700 1 -- 172.21.15.99:0/757348869 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1e780551c0 con 0x7f1e80160cc0 2024-03-29T09:10:47.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.839+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e800d0700 con 0x7f1e80160cc0 2024-03-29T09:10:47.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.840+0000 7f1e7d600700 1 -- 172.21.15.99:0/757348869 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16306 (secure 0 0 0) 0x7f1e780587e0 con 0x7f1e80160cc0 2024-03-29T09:10:47.840 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:47.840 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":39,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:44.926429+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.842+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e6006ac90 msgr2=0x7f1e6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.842+0000 7f1e88200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e6006ac90 0x7f1e6006d150 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009c40 tx=0x7f1e74008030 comp rx=0 tx=0).stop 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.842+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e80160cc0 msgr2=0x7f1e800ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.842+0000 7f1e88200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e80160cc0 0x7f1e800ca760 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800ef90 tx=0x7f1e7800c3a0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 shutdown_connections 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e6006ac90 0x7f1e6006d150 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e80160cc0 0x7f1e800ca760 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e800d1b20 0x7f1e801f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 --2- 172.21.15.99:0/757348869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e801608e0 0x7f1e800ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 >> 172.21.15.99:0/757348869 conn(0x7f1e8009c490 msgr2=0x7f1e800d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 shutdown_connections 2024-03-29T09:10:47.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:47.843+0000 7f1e88200700 1 -- 172.21.15.99:0/757348869 wait complete. 2024-03-29T09:10:47.861 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T09:10:47.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:10:48.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.051+0000 7f540c400700 1 Processor -- start 2024-03-29T09:10:48.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.051+0000 7f540c400700 1 -- start start 2024-03-29T09:10:48.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f540c400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f54040cfd70 0x7f54041615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f540c400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54040d0150 0x7f5404161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f540c400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54040d1be0 con 0x7f5404162020 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f540c400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54040b4980 con 0x7f54040d0150 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f540c400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54040b46a0 con 0x7f54040cfd70 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54040d0150 0x7f5404161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f54040d0150 0x7f5404161ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33188/0 (socket says 172.21.15.99:33188) 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409000700 1 -- 172.21.15.99:0/2088733143 learned_addr learned my addr 172.21.15.99:0/2088733143 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409a00700 1 --2- 172.21.15.99:0/2088733143 >> v2:172.21.15.133:3300/0 conn(0x7f54040cfd70 0x7f54041615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409000700 1 -- 172.21.15.99:0/2088733143 >> v2:172.21.15.133:3300/0 conn(0x7f54040cfd70 msgr2=0x7f54041615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409000700 1 --2- 172.21.15.99:0/2088733143 >> v2:172.21.15.133:3300/0 conn(0x7f54040cfd70 0x7f54041615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409000700 1 -- 172.21.15.99:0/2088733143 >> v1:172.21.15.133:6789/0 conn(0x7f5404162020 legacy=0x7f54040b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:48.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.053+0000 7f5409000700 1 -- 172.21.15.99:0/2088733143 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040d1590 con 0x7f54040d0150 2024-03-29T09:10:48.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f5409000700 1 --2- 172.21.15.99:0/2088733143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54040d0150 0x7f5404161ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f53f00098f0 tx=0x7f53f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4897239352e5d28c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:48.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f5403e00700 1 -- 172.21.15.99:0/2088733143 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0004230 con 0x7f54040d0150 2024-03-29T09:10:48.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f5403e00700 1 -- 172.21.15.99:0/2088733143 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f0004390 con 0x7f54040d0150 2024-03-29T09:10:48.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f5403e00700 1 -- 172.21.15.99:0/2088733143 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f00036a0 con 0x7f54040d0150 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f540c400700 1 -- 172.21.15.99:0/2088733143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54040d0150 msgr2=0x7f5404161ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f540c400700 1 --2- 172.21.15.99:0/2088733143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54040d0150 0x7f5404161ae0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f53f00098f0 tx=0x7f53f0009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f540c400700 1 -- 172.21.15.99:0/2088733143 shutdown_connections 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f540c400700 1 --2- 172.21.15.99:0/2088733143 >> v2:172.21.15.133:3300/0 conn(0x7f54040cfd70 0x7f54041615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f540c400700 1 --2- 172.21.15.99:0/2088733143 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54040d0150 0x7f5404161ae0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f540c400700 1 -- 172.21.15.99:0/2088733143 >> 172.21.15.99:0/2088733143 conn(0x7f540409c4a0 msgr2=0x7f54040d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.054+0000 7f540c400700 1 -- 172.21.15.99:0/2088733143 shutdown_connections 2024-03-29T09:10:48.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.055+0000 7f540c400700 1 -- 172.21.15.99:0/2088733143 wait complete. 2024-03-29T09:10:48.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.057+0000 7f540c400700 1 Processor -- start 2024-03-29T09:10:48.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.057+0000 7f540c400700 1 -- start start 2024-03-29T09:10:48.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f540c400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54040cfd70 0x7f54041f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f540c400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54040d0150 0x7f5404156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f540c400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5404162020 0x7f5404152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f540c400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041f6e70 con 0x7f5404162020 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f540c400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54040b5f70 con 0x7f54040d0150 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f540c400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54040cc7e0 con 0x7f54040cfd70 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f540a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5404162020 0x7f5404152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.058+0000 7f5409a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54040cfd70 0x7f54041f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5404162020 0x7f5404152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33202/0 (socket says 172.21.15.99:33202) 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 -- 172.21.15.99:0/818713385 learned_addr learned my addr 172.21.15.99:0/818713385 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f5409000700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54040d0150 0x7f5404156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 -- 172.21.15.99:0/818713385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54040cfd70 msgr2=0x7f54041f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54040cfd70 0x7f54041f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 -- 172.21.15.99:0/818713385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54040d0150 msgr2=0x7f5404156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54040d0150 0x7f5404156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 -- 172.21.15.99:0/818713385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0009550 con 0x7f5404162020 2024-03-29T09:10:48.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540a400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5404162020 0x7f5404152570 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00b970 tx=0x7f53fc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:48.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f5401600700 1 -- 172.21.15.99:0/818713385 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00d860 con 0x7f5404162020 2024-03-29T09:10:48.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f5401600700 1 -- 172.21.15.99:0/818713385 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc00d9c0 con 0x7f5404162020 2024-03-29T09:10:48.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f5401600700 1 -- 172.21.15.99:0/818713385 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00dd70 con 0x7f5404162020 2024-03-29T09:10:48.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.059+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54041f65d0 con 0x7f5404162020 2024-03-29T09:10:48.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.060+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54041f6b90 con 0x7f5404162020 2024-03-29T09:10:48.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.061+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54040b5d40 con 0x7f5404162020 2024-03-29T09:10:48.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.061+0000 7f5401600700 1 -- 172.21.15.99:0/818713385 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f53fc00e430 con 0x7f5404162020 2024-03-29T09:10:48.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.062+0000 7f5401600700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f53e406acc0 0x7f53e406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.062+0000 7f5401600700 1 -- 172.21.15.99:0/818713385 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f53fc089550 con 0x7f5404162020 2024-03-29T09:10:48.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.062+0000 7f5409a00700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f53e406acc0 0x7f53e406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.063+0000 7f5409a00700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f53e406acc0 0x7f53e406d180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006b20 tx=0x7f53f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:48.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.067+0000 7f5401600700 1 -- 172.21.15.99:0/818713385 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f54040b5d40 con 0x7f5404162020 2024-03-29T09:10:48.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.267+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54040d2070 con 0x7f53e406acc0 2024-03-29T09:10:48.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.276+0000 7f5401600700 1 -- 172.21.15.99:0/818713385 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274684 (secure 0 0 0) 0x7f53f8009e40 con 0x7f53e406acc0 2024-03-29T09:10:48.280 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:48.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f53e406acc0 msgr2=0x7f53e406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f53e406acc0 0x7f53e406d180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006b20 tx=0x7f53f8008040 comp rx=0 tx=0).stop 2024-03-29T09:10:48.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5404162020 msgr2=0x7f5404152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5404162020 0x7f5404152570 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00b970 tx=0x7f53fc00bc80 comp rx=0 tx=0).stop 2024-03-29T09:10:48.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 shutdown_connections 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f53e406acc0 0x7f53e406d180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5404162020 0x7f5404152570 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54040cfd70 0x7f54041f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 --2- 172.21.15.99:0/818713385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54040d0150 0x7f5404156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 >> 172.21.15.99:0/818713385 conn(0x7f540409c4a0 msgr2=0x7f54040c9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 shutdown_connections 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.280+0000 7f540c400700 1 -- 172.21.15.99:0/818713385 wait complete. 2024-03-29T09:10:48.284 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:10:48.299 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-03-29T09:10:46.473902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220500,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754220,"statfs":{"total":773094113280,"available":772868321280,"internally_reserved":0,"allocated":4399104,"data_stored":2516600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.062436"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:43.411059+0000","last_change":"2024-03-29T09:10:43.411059+0000","last_active":"2024-03-29T09:10:43.411059+0000","last_peered":"2024-03-29T09:10:43.411059+0000","last_clean":"2024-03-29T09:10:43.411059+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:43.411059+0000","last_undegraded":"2024-03-29T09:10:43.411059+0000","last_fullsized":"2024-03-29T09:10:43.411059+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:10:42.404239+0000","last_change":"2024-03-29T09:10:42.404239+0000","last_active":"2024-03-29T09:10:42.404239+0000","last_peered":"2024-03-29T09:10:42.404239+0000","last_clean":"2024-03-29T09:10:42.404239+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:10:42.404239+0000","last_undegraded":"2024-03-29T09:10:42.404239+0000","last_fullsized":"2024-03-29T09:10:42.404239+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-29T09:10:35.741195+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:10:35.741195+0000","last_peered":"2024-03-29T09:10:35.741195+0000","last_clean":"2024-03-29T09:10:35.741195+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:10:35.741195+0000","last_undegraded":"2024-03-29T09:10:35.741195+0000","last_fullsized":"2024-03-29T09:10:35.741195+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":166983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.495}]},{"osd":2,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.629}]},{"osd":3,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.751}]},{"osd":5,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.436}]},{"osd":6,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.64200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":166983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.74399999999999999}]},{"osd":1,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.2599999999999998,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.1070000000000002,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":995328,"data_stored":757351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.0119999999999996,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.9140000000000001,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.98599999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.91800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":166983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.442},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.747},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":166983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.433},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.47299999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.627},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.64200000000000002}]},{"osd":6,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":166983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.432},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.40500000000000003}]},{"osd":1,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.378},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.629}]},{"osd":5,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.60199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":401408,"data_stored":166983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.77700000000000002}]},{"osd":1,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.69999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":757351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.75},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.625}]},{"osd":4,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.85299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:10:48.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:10:48.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.529+0000 7f0b41800700 1 Processor -- start 2024-03-29T09:10:48.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.529+0000 7f0b41800700 1 -- start start 2024-03-29T09:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.530+0000 7f0b41800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0b3c0cecc0 0x7f0b3c168860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.530+0000 7f0b41800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0d1be0 con 0x7f0b3c0cfd70 2024-03-29T09:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.530+0000 7f0b41800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c0b4980 con 0x7f0b3c0d0150 2024-03-29T09:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.530+0000 7f0b41800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c1623c0 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.530+0000 7f0b3b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0b3c0cecc0 0x7f0b3c168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.530+0000 7f0b3aa00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0b3c0cfd70 0x7f0b3c1615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46524/0 (socket says 172.21.15.99:46524) 2024-03-29T09:10:48.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.530+0000 7f0b3aa00700 1 -- 172.21.15.99:0/3113333928 learned_addr learned my addr 172.21.15.99:0/3113333928 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:48.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b3b400700 1 -- 172.21.15.99:0/3113333928 >> v1:172.21.15.133:6789/0 conn(0x7f0b3c0d0150 legacy=0x7f0b3c161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:48.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b3b400700 1 -- 172.21.15.99:0/3113333928 >> v1:172.21.15.99:6789/0 conn(0x7f0b3c0cfd70 legacy=0x7f0b3c1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b3b400700 1 -- 172.21.15.99:0/3113333928 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0b46a0 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b39600700 1 -- 172.21.15.99:0/3113333928 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494934816 0 0) 0x7f0b3c0d1be0 con 0x7f0b3c0cfd70 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b3b400700 1 --2- 172.21.15.99:0/3113333928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c168860 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009dc0 tx=0x7f0b3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6987d2c68220355f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b39600700 1 -- 172.21.15.99:0/3113333928 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3400e070 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b39600700 1 -- 172.21.15.99:0/3113333928 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3400bb50 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.531+0000 7f0b39600700 1 -- 172.21.15.99:0/3113333928 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34003a20 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.532+0000 7f0b41800700 1 -- 172.21.15.99:0/3113333928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 msgr2=0x7f0b3c168860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.532+0000 7f0b41800700 1 --2- 172.21.15.99:0/3113333928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c168860 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009dc0 tx=0x7f0b3400b330 comp rx=0 tx=0).stop 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.532+0000 7f0b41800700 1 -- 172.21.15.99:0/3113333928 shutdown_connections 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.532+0000 7f0b41800700 1 --2- 172.21.15.99:0/3113333928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c168860 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.532+0000 7f0b41800700 1 -- 172.21.15.99:0/3113333928 >> 172.21.15.99:0/3113333928 conn(0x7f0b3c09c4a0 msgr2=0x7f0b3c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.532+0000 7f0b41800700 1 -- 172.21.15.99:0/3113333928 shutdown_connections 2024-03-29T09:10:48.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.532+0000 7f0b41800700 1 -- 172.21.15.99:0/3113333928 wait complete. 2024-03-29T09:10:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.534+0000 7f0b41800700 1 Processor -- start 2024-03-29T09:10:48.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.534+0000 7f0b41800700 1 -- start start 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b41800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b41800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b3c0cfd70 0x7f0b3c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b41800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0d0150 0x7f0b3c1e8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b41800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b5d40 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b41800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b6020 con 0x7f0b3c0d0150 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b41800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b6300 con 0x7f0b3c0cfd70 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33222/0 (socket says 172.21.15.99:33222) 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 -- 172.21.15.99:0/1454884160 learned_addr learned my addr 172.21.15.99:0/1454884160 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3b400700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0d0150 0x7f0b3c1e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3a000700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b3c0cfd70 0x7f0b3c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 -- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b3c0cfd70 msgr2=0x7f0b3c15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b3c0cfd70 0x7f0b3c15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 -- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0d0150 msgr2=0x7f0b3c1e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0d0150 0x7f0b3c1e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.535+0000 7f0b3aa00700 1 -- 172.21.15.99:0/1454884160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34009a70 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.536+0000 7f0b3aa00700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c15aff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0011e0 tx=0x7f0b2c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.536+0000 7f0b32a00700 1 -- 172.21.15.99:0/1454884160 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c004c80 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.536+0000 7f0b32a00700 1 -- 172.21.15.99:0/1454884160 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c004e00 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.536+0000 7f0b32a00700 1 -- 172.21.15.99:0/1454884160 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c007310 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.536+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c0b65e0 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.536+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c15b760 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.537+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c0677a0 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.538+0000 7f0b32a00700 1 -- 172.21.15.99:0/1454884160 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0b2c007470 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.539+0000 7f0b32a00700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0b1c06ac90 0x7f0b1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.539+0000 7f0b32a00700 1 -- 172.21.15.99:0/1454884160 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0b2c089200 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.539+0000 7f0b3a000700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0b1c06ac90 0x7f0b1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:48.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.540+0000 7f0b3a000700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0b1c06ac90 0x7f0b1c06d150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0cbe60 tx=0x7f0b28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:48.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.543+0000 7f0b32a00700 1 -- 172.21.15.99:0/1454884160 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0b2c054290 con 0x7f0b3c0cecc0 2024-03-29T09:10:48.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.744+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b3c0b2300 con 0x7f0b1c06ac90 2024-03-29T09:10:48.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.752+0000 7f0b32a00700 1 -- 172.21.15.99:0/1454884160 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289477 (secure 0 0 0) 0x7f0b28009510 con 0x7f0b1c06ac90 2024-03-29T09:10:48.756 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:48.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0b1c06ac90 msgr2=0x7f0b1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0b1c06ac90 0x7f0b1c06d150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0cbe60 tx=0x7f0b28008040 comp rx=0 tx=0).stop 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 msgr2=0x7f0b3c15aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c15aff0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0011e0 tx=0x7f0b2c00cae0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 shutdown_connections 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0b1c06ac90 0x7f0b1c06d150 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b3c0d0150 0x7f0b3c1e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0b3c0cecc0 0x7f0b3c15aff0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 --2- 172.21.15.99:0/1454884160 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b3c0cfd70 0x7f0b3c15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:48.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.756+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 >> 172.21.15.99:0/1454884160 conn(0x7f0b3c09c4a0 msgr2=0x7f0b3c161c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.757+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 shutdown_connections 2024-03-29T09:10:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:48.757+0000 7f0b41800700 1 -- 172.21.15.99:0/1454884160 wait complete. 2024-03-29T09:10:48.760 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:10:48.775 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2024-03-29T09:10:48.474199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220940,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753780,"statfs":{"total":773094113280,"available":772867870720,"internally_reserved":0,"allocated":4857856,"data_stored":2732900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001021"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483118+0000","last_change":"2024-03-29T09:10:43.954392+0000","last_active":"2024-03-29T09:10:45.483118+0000","last_peered":"2024-03-29T09:10:45.483118+0000","last_clean":"2024-03-29T09:10:45.483118+0000","last_became_active":"2024-03-29T09:10:43.954127+0000","last_became_peered":"2024-03-29T09:10:43.954127+0000","last_unstale":"2024-03-29T09:10:45.483118+0000","last_undegraded":"2024-03-29T09:10:45.483118+0000","last_fullsized":"2024-03-29T09:10:45.483118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:32.398873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.936237+0000","last_change":"2024-03-29T09:10:43.949301+0000","last_active":"2024-03-29T09:10:44.936237+0000","last_peered":"2024-03-29T09:10:44.936237+0000","last_clean":"2024-03-29T09:10:44.936237+0000","last_became_active":"2024-03-29T09:10:43.949063+0000","last_became_peered":"2024-03-29T09:10:43.949063+0000","last_unstale":"2024-03-29T09:10:44.936237+0000","last_undegraded":"2024-03-29T09:10:44.936237+0000","last_fullsized":"2024-03-29T09:10:44.936237+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:50:42.131772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483121+0000","last_change":"2024-03-29T09:10:43.962263+0000","last_active":"2024-03-29T09:10:45.483121+0000","last_peered":"2024-03-29T09:10:45.483121+0000","last_clean":"2024-03-29T09:10:45.483121+0000","last_became_active":"2024-03-29T09:10:43.960017+0000","last_became_peered":"2024-03-29T09:10:43.960017+0000","last_unstale":"2024-03-29T09:10:45.483121+0000","last_undegraded":"2024-03-29T09:10:45.483121+0000","last_fullsized":"2024-03-29T09:10:45.483121+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:23.632163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483222+0000","last_change":"2024-03-29T09:10:43.959166+0000","last_active":"2024-03-29T09:10:45.483222+0000","last_peered":"2024-03-29T09:10:45.483222+0000","last_clean":"2024-03-29T09:10:45.483222+0000","last_became_active":"2024-03-29T09:10:43.957508+0000","last_became_peered":"2024-03-29T09:10:43.957508+0000","last_unstale":"2024-03-29T09:10:45.483222+0000","last_undegraded":"2024-03-29T09:10:45.483222+0000","last_fullsized":"2024-03-29T09:10:45.483222+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:41:26.795268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085236+0000","last_change":"2024-03-29T09:10:43.958786+0000","last_active":"2024-03-29T09:10:45.085236+0000","last_peered":"2024-03-29T09:10:45.085236+0000","last_clean":"2024-03-29T09:10:45.085236+0000","last_became_active":"2024-03-29T09:10:43.958516+0000","last_became_peered":"2024-03-29T09:10:43.958516+0000","last_unstale":"2024-03-29T09:10:45.085236+0000","last_undegraded":"2024-03-29T09:10:45.085236+0000","last_fullsized":"2024-03-29T09:10:45.085236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:08.194029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935699+0000","last_change":"2024-03-29T09:10:43.945589+0000","last_active":"2024-03-29T09:10:44.935699+0000","last_peered":"2024-03-29T09:10:44.935699+0000","last_clean":"2024-03-29T09:10:44.935699+0000","last_became_active":"2024-03-29T09:10:43.945278+0000","last_became_peered":"2024-03-29T09:10:43.945278+0000","last_unstale":"2024-03-29T09:10:44.935699+0000","last_undegraded":"2024-03-29T09:10:44.935699+0000","last_fullsized":"2024-03-29T09:10:44.935699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:33.964184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482728+0000","last_change":"2024-03-29T09:10:43.953518+0000","last_active":"2024-03-29T09:10:45.482728+0000","last_peered":"2024-03-29T09:10:45.482728+0000","last_clean":"2024-03-29T09:10:45.482728+0000","last_became_active":"2024-03-29T09:10:43.947597+0000","last_became_peered":"2024-03-29T09:10:43.947597+0000","last_unstale":"2024-03-29T09:10:45.482728+0000","last_undegraded":"2024-03-29T09:10:45.482728+0000","last_fullsized":"2024-03-29T09:10:45.482728+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:44.328604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482665+0000","last_change":"2024-03-29T09:10:43.956133+0000","last_active":"2024-03-29T09:10:45.482665+0000","last_peered":"2024-03-29T09:10:45.482665+0000","last_clean":"2024-03-29T09:10:45.482665+0000","last_became_active":"2024-03-29T09:10:43.955881+0000","last_became_peered":"2024-03-29T09:10:43.955881+0000","last_unstale":"2024-03-29T09:10:45.482665+0000","last_undegraded":"2024-03-29T09:10:45.482665+0000","last_fullsized":"2024-03-29T09:10:45.482665+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:50:20.935003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085585+0000","last_change":"2024-03-29T09:10:43.949456+0000","last_active":"2024-03-29T09:10:45.085585+0000","last_peered":"2024-03-29T09:10:45.085585+0000","last_clean":"2024-03-29T09:10:45.085585+0000","last_became_active":"2024-03-29T09:10:43.949055+0000","last_became_peered":"2024-03-29T09:10:43.949055+0000","last_unstale":"2024-03-29T09:10:45.085585+0000","last_undegraded":"2024-03-29T09:10:45.085585+0000","last_fullsized":"2024-03-29T09:10:45.085585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:31:33.279465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483126+0000","last_change":"2024-03-29T09:10:43.957014+0000","last_active":"2024-03-29T09:10:45.483126+0000","last_peered":"2024-03-29T09:10:45.483126+0000","last_clean":"2024-03-29T09:10:45.483126+0000","last_became_active":"2024-03-29T09:10:43.956717+0000","last_became_peered":"2024-03-29T09:10:43.956717+0000","last_unstale":"2024-03-29T09:10:45.483126+0000","last_undegraded":"2024-03-29T09:10:45.483126+0000","last_fullsized":"2024-03-29T09:10:45.483126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:13:35.536641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482958+0000","last_change":"2024-03-29T09:10:43.950935+0000","last_active":"2024-03-29T09:10:45.482958+0000","last_peered":"2024-03-29T09:10:45.482958+0000","last_clean":"2024-03-29T09:10:45.482958+0000","last_became_active":"2024-03-29T09:10:43.950363+0000","last_became_peered":"2024-03-29T09:10:43.950363+0000","last_unstale":"2024-03-29T09:10:45.482958+0000","last_undegraded":"2024-03-29T09:10:45.482958+0000","last_fullsized":"2024-03-29T09:10:45.482958+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:41:20.769865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085402+0000","last_change":"2024-03-29T09:10:43.944446+0000","last_active":"2024-03-29T09:10:45.085402+0000","last_peered":"2024-03-29T09:10:45.085402+0000","last_clean":"2024-03-29T09:10:45.085402+0000","last_became_active":"2024-03-29T09:10:43.943183+0000","last_became_peered":"2024-03-29T09:10:43.943183+0000","last_unstale":"2024-03-29T09:10:45.085402+0000","last_undegraded":"2024-03-29T09:10:45.085402+0000","last_fullsized":"2024-03-29T09:10:45.085402+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:44.328604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482793+0000","last_change":"2024-03-29T09:10:43.957051+0000","last_active":"2024-03-29T09:10:45.482793+0000","last_peered":"2024-03-29T09:10:45.482793+0000","last_clean":"2024-03-29T09:10:45.482793+0000","last_became_active":"2024-03-29T09:10:43.955247+0000","last_became_peered":"2024-03-29T09:10:43.955247+0000","last_unstale":"2024-03-29T09:10:45.482793+0000","last_undegraded":"2024-03-29T09:10:45.482793+0000","last_fullsized":"2024-03-29T09:10:45.482793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:39:37.365296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482325+0000","last_change":"2024-03-29T09:10:43.950111+0000","last_active":"2024-03-29T09:10:45.482325+0000","last_peered":"2024-03-29T09:10:45.482325+0000","last_clean":"2024-03-29T09:10:45.482325+0000","last_became_active":"2024-03-29T09:10:43.949877+0000","last_became_peered":"2024-03-29T09:10:43.949877+0000","last_unstale":"2024-03-29T09:10:45.482325+0000","last_undegraded":"2024-03-29T09:10:45.482325+0000","last_fullsized":"2024-03-29T09:10:45.482325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:36:50.843200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934837+0000","last_change":"2024-03-29T09:10:43.961807+0000","last_active":"2024-03-29T09:10:44.934837+0000","last_peered":"2024-03-29T09:10:44.934837+0000","last_clean":"2024-03-29T09:10:44.934837+0000","last_became_active":"2024-03-29T09:10:43.961562+0000","last_became_peered":"2024-03-29T09:10:43.961562+0000","last_unstale":"2024-03-29T09:10:44.934837+0000","last_undegraded":"2024-03-29T09:10:44.934837+0000","last_fullsized":"2024-03-29T09:10:44.934837+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:25.599381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.481881+0000","last_change":"2024-03-29T09:10:43.948010+0000","last_active":"2024-03-29T09:10:45.481881+0000","last_peered":"2024-03-29T09:10:45.481881+0000","last_clean":"2024-03-29T09:10:45.481881+0000","last_became_active":"2024-03-29T09:10:43.947759+0000","last_became_peered":"2024-03-29T09:10:43.947759+0000","last_unstale":"2024-03-29T09:10:45.481881+0000","last_undegraded":"2024-03-29T09:10:45.481881+0000","last_fullsized":"2024-03-29T09:10:45.481881+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:26.500284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934957+0000","last_change":"2024-03-29T09:10:43.960144+0000","last_active":"2024-03-29T09:10:44.934957+0000","last_peered":"2024-03-29T09:10:44.934957+0000","last_clean":"2024-03-29T09:10:44.934957+0000","last_became_active":"2024-03-29T09:10:43.959908+0000","last_became_peered":"2024-03-29T09:10:43.959908+0000","last_unstale":"2024-03-29T09:10:44.934957+0000","last_undegraded":"2024-03-29T09:10:44.934957+0000","last_fullsized":"2024-03-29T09:10:44.934957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:36.611567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935105+0000","last_change":"2024-03-29T09:10:43.960781+0000","last_active":"2024-03-29T09:10:44.935105+0000","last_peered":"2024-03-29T09:10:44.935105+0000","last_clean":"2024-03-29T09:10:44.935105+0000","last_became_active":"2024-03-29T09:10:43.954830+0000","last_became_peered":"2024-03-29T09:10:43.954830+0000","last_unstale":"2024-03-29T09:10:44.935105+0000","last_undegraded":"2024-03-29T09:10:44.935105+0000","last_fullsized":"2024-03-29T09:10:44.935105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:22.570138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482858+0000","last_change":"2024-03-29T09:10:43.953331+0000","last_active":"2024-03-29T09:10:45.482858+0000","last_peered":"2024-03-29T09:10:45.482858+0000","last_clean":"2024-03-29T09:10:45.482858+0000","last_became_active":"2024-03-29T09:10:43.953059+0000","last_became_peered":"2024-03-29T09:10:43.953059+0000","last_unstale":"2024-03-29T09:10:45.482858+0000","last_undegraded":"2024-03-29T09:10:45.482858+0000","last_fullsized":"2024-03-29T09:10:45.482858+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:41:20.769865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482494+0000","last_change":"2024-03-29T09:10:43.955823+0000","last_active":"2024-03-29T09:10:45.482494+0000","last_peered":"2024-03-29T09:10:45.482494+0000","last_clean":"2024-03-29T09:10:45.482494+0000","last_became_active":"2024-03-29T09:10:43.955282+0000","last_became_peered":"2024-03-29T09:10:43.955282+0000","last_unstale":"2024-03-29T09:10:45.482494+0000","last_undegraded":"2024-03-29T09:10:45.482494+0000","last_fullsized":"2024-03-29T09:10:45.482494+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:46:00.163677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482288+0000","last_change":"2024-03-29T09:10:43.945508+0000","last_active":"2024-03-29T09:10:45.482288+0000","last_peered":"2024-03-29T09:10:45.482288+0000","last_clean":"2024-03-29T09:10:45.482288+0000","last_became_active":"2024-03-29T09:10:43.945250+0000","last_became_peered":"2024-03-29T09:10:43.945250+0000","last_unstale":"2024-03-29T09:10:45.482288+0000","last_undegraded":"2024-03-29T09:10:45.482288+0000","last_fullsized":"2024-03-29T09:10:45.482288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:37.550795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.481904+0000","last_change":"2024-03-29T09:10:43.952938+0000","last_active":"2024-03-29T09:10:45.481904+0000","last_peered":"2024-03-29T09:10:45.481904+0000","last_clean":"2024-03-29T09:10:45.481904+0000","last_became_active":"2024-03-29T09:10:43.952632+0000","last_became_peered":"2024-03-29T09:10:43.952632+0000","last_unstale":"2024-03-29T09:10:45.481904+0000","last_undegraded":"2024-03-29T09:10:45.481904+0000","last_fullsized":"2024-03-29T09:10:45.481904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:54.108526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934738+0000","last_change":"2024-03-29T09:10:43.953307+0000","last_active":"2024-03-29T09:10:44.934738+0000","last_peered":"2024-03-29T09:10:44.934738+0000","last_clean":"2024-03-29T09:10:44.934738+0000","last_became_active":"2024-03-29T09:10:43.953041+0000","last_became_peered":"2024-03-29T09:10:43.953041+0000","last_unstale":"2024-03-29T09:10:44.934738+0000","last_undegraded":"2024-03-29T09:10:44.934738+0000","last_fullsized":"2024-03-29T09:10:44.934738+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:55:33.731359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483503+0000","last_change":"2024-03-29T09:10:43.947404+0000","last_active":"2024-03-29T09:10:45.483503+0000","last_peered":"2024-03-29T09:10:45.483503+0000","last_clean":"2024-03-29T09:10:45.483503+0000","last_became_active":"2024-03-29T09:10:43.946665+0000","last_became_peered":"2024-03-29T09:10:43.946665+0000","last_unstale":"2024-03-29T09:10:45.483503+0000","last_undegraded":"2024-03-29T09:10:45.483503+0000","last_fullsized":"2024-03-29T09:10:45.483503+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:46.291248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482741+0000","last_change":"2024-03-29T09:10:43.941487+0000","last_active":"2024-03-29T09:10:45.482741+0000","last_peered":"2024-03-29T09:10:45.482741+0000","last_clean":"2024-03-29T09:10:45.482741+0000","last_became_active":"2024-03-29T09:10:43.941240+0000","last_became_peered":"2024-03-29T09:10:43.941240+0000","last_unstale":"2024-03-29T09:10:45.482741+0000","last_undegraded":"2024-03-29T09:10:45.482741+0000","last_fullsized":"2024-03-29T09:10:45.482741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:53:53.281012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483510+0000","last_change":"2024-03-29T09:10:43.959038+0000","last_active":"2024-03-29T09:10:45.483510+0000","last_peered":"2024-03-29T09:10:45.483510+0000","last_clean":"2024-03-29T09:10:45.483510+0000","last_became_active":"2024-03-29T09:10:43.950085+0000","last_became_peered":"2024-03-29T09:10:43.950085+0000","last_unstale":"2024-03-29T09:10:45.483510+0000","last_undegraded":"2024-03-29T09:10:45.483510+0000","last_fullsized":"2024-03-29T09:10:45.483510+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:05:44.797417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483052+0000","last_change":"2024-03-29T09:10:43.951125+0000","last_active":"2024-03-29T09:10:45.483052+0000","last_peered":"2024-03-29T09:10:45.483052+0000","last_clean":"2024-03-29T09:10:45.483052+0000","last_became_active":"2024-03-29T09:10:43.950049+0000","last_became_peered":"2024-03-29T09:10:43.950049+0000","last_unstale":"2024-03-29T09:10:45.483052+0000","last_undegraded":"2024-03-29T09:10:45.483052+0000","last_fullsized":"2024-03-29T09:10:45.483052+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:24:27.552028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482893+0000","last_change":"2024-03-29T09:10:43.952623+0000","last_active":"2024-03-29T09:10:45.482893+0000","last_peered":"2024-03-29T09:10:45.482893+0000","last_clean":"2024-03-29T09:10:45.482893+0000","last_became_active":"2024-03-29T09:10:43.952382+0000","last_became_peered":"2024-03-29T09:10:43.952382+0000","last_unstale":"2024-03-29T09:10:45.482893+0000","last_undegraded":"2024-03-29T09:10:45.482893+0000","last_fullsized":"2024-03-29T09:10:45.482893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:43:36.520305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934732+0000","last_change":"2024-03-29T09:10:43.944984+0000","last_active":"2024-03-29T09:10:44.934732+0000","last_peered":"2024-03-29T09:10:44.934732+0000","last_clean":"2024-03-29T09:10:44.934732+0000","last_became_active":"2024-03-29T09:10:43.944520+0000","last_became_peered":"2024-03-29T09:10:43.944520+0000","last_unstale":"2024-03-29T09:10:44.934732+0000","last_undegraded":"2024-03-29T09:10:44.934732+0000","last_fullsized":"2024-03-29T09:10:44.934732+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:12.175626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934679+0000","last_change":"2024-03-29T09:10:43.948829+0000","last_active":"2024-03-29T09:10:44.934679+0000","last_peered":"2024-03-29T09:10:44.934679+0000","last_clean":"2024-03-29T09:10:44.934679+0000","last_became_active":"2024-03-29T09:10:43.948586+0000","last_became_peered":"2024-03-29T09:10:43.948586+0000","last_unstale":"2024-03-29T09:10:44.934679+0000","last_undegraded":"2024-03-29T09:10:44.934679+0000","last_fullsized":"2024-03-29T09:10:44.934679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:18:03.591341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483231+0000","last_change":"2024-03-29T09:10:43.957607+0000","last_active":"2024-03-29T09:10:45.483231+0000","last_peered":"2024-03-29T09:10:45.483231+0000","last_clean":"2024-03-29T09:10:45.483231+0000","last_became_active":"2024-03-29T09:10:43.956699+0000","last_became_peered":"2024-03-29T09:10:43.956699+0000","last_unstale":"2024-03-29T09:10:45.483231+0000","last_undegraded":"2024-03-29T09:10:45.483231+0000","last_fullsized":"2024-03-29T09:10:45.483231+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:53:08.194029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483409+0000","last_change":"2024-03-29T09:10:43.949206+0000","last_active":"2024-03-29T09:10:45.483409+0000","last_peered":"2024-03-29T09:10:45.483409+0000","last_clean":"2024-03-29T09:10:45.483409+0000","last_became_active":"2024-03-29T09:10:43.948662+0000","last_became_peered":"2024-03-29T09:10:43.948662+0000","last_unstale":"2024-03-29T09:10:45.483409+0000","last_undegraded":"2024-03-29T09:10:45.483409+0000","last_fullsized":"2024-03-29T09:10:45.483409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:50.842837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934982+0000","last_change":"2024-03-29T09:10:43.951970+0000","last_active":"2024-03-29T09:10:44.934982+0000","last_peered":"2024-03-29T09:10:44.934982+0000","last_clean":"2024-03-29T09:10:44.934982+0000","last_became_active":"2024-03-29T09:10:43.950035+0000","last_became_peered":"2024-03-29T09:10:43.950035+0000","last_unstale":"2024-03-29T09:10:44.934982+0000","last_undegraded":"2024-03-29T09:10:44.934982+0000","last_fullsized":"2024-03-29T09:10:44.934982+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:53.721409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482736+0000","last_change":"2024-03-29T09:10:43.426761+0000","last_active":"2024-03-29T09:10:45.482736+0000","last_peered":"2024-03-29T09:10:45.482736+0000","last_clean":"2024-03-29T09:10:45.482736+0000","last_became_active":"2024-03-29T09:10:43.426120+0000","last_became_peered":"2024-03-29T09:10:43.426120+0000","last_unstale":"2024-03-29T09:10:45.482736+0000","last_undegraded":"2024-03-29T09:10:45.482736+0000","last_fullsized":"2024-03-29T09:10:45.482736+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:01:04.601553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.936188+0000","last_change":"2024-03-29T09:10:43.956958+0000","last_active":"2024-03-29T09:10:44.936188+0000","last_peered":"2024-03-29T09:10:44.936188+0000","last_clean":"2024-03-29T09:10:44.936188+0000","last_became_active":"2024-03-29T09:10:43.956669+0000","last_became_peered":"2024-03-29T09:10:43.956669+0000","last_unstale":"2024-03-29T09:10:44.936188+0000","last_undegraded":"2024-03-29T09:10:44.936188+0000","last_fullsized":"2024-03-29T09:10:44.936188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:45:51.168003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.936243+0000","last_change":"2024-03-29T09:10:43.445544+0000","last_active":"2024-03-29T09:10:44.936243+0000","last_peered":"2024-03-29T09:10:44.936243+0000","last_clean":"2024-03-29T09:10:44.936243+0000","last_became_active":"2024-03-29T09:10:43.445218+0000","last_became_peered":"2024-03-29T09:10:43.445218+0000","last_unstale":"2024-03-29T09:10:44.936243+0000","last_undegraded":"2024-03-29T09:10:44.936243+0000","last_fullsized":"2024-03-29T09:10:44.936243+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:17.693317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935174+0000","last_change":"2024-03-29T09:10:43.952538+0000","last_active":"2024-03-29T09:10:44.935174+0000","last_peered":"2024-03-29T09:10:44.935174+0000","last_clean":"2024-03-29T09:10:44.935174+0000","last_became_active":"2024-03-29T09:10:43.949263+0000","last_became_peered":"2024-03-29T09:10:43.949263+0000","last_unstale":"2024-03-29T09:10:44.935174+0000","last_undegraded":"2024-03-29T09:10:44.935174+0000","last_fullsized":"2024-03-29T09:10:44.935174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:40:31.461053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483149+0000","last_change":"2024-03-29T09:10:43.442861+0000","last_active":"2024-03-29T09:10:45.483149+0000","last_peered":"2024-03-29T09:10:45.483149+0000","last_clean":"2024-03-29T09:10:45.483149+0000","last_became_active":"2024-03-29T09:10:43.442411+0000","last_became_peered":"2024-03-29T09:10:43.442411+0000","last_unstale":"2024-03-29T09:10:45.483149+0000","last_undegraded":"2024-03-29T09:10:45.483149+0000","last_fullsized":"2024-03-29T09:10:45.483149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:41:19.763045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.936291+0000","last_change":"2024-03-29T09:10:43.957040+0000","last_active":"2024-03-29T09:10:44.936291+0000","last_peered":"2024-03-29T09:10:44.936291+0000","last_clean":"2024-03-29T09:10:44.936291+0000","last_became_active":"2024-03-29T09:10:43.956663+0000","last_became_peered":"2024-03-29T09:10:43.956663+0000","last_unstale":"2024-03-29T09:10:44.936291+0000","last_undegraded":"2024-03-29T09:10:44.936291+0000","last_fullsized":"2024-03-29T09:10:44.936291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:43:27.580498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482278+0000","last_change":"2024-03-29T09:10:43.438545+0000","last_active":"2024-03-29T09:10:45.482278+0000","last_peered":"2024-03-29T09:10:45.482278+0000","last_clean":"2024-03-29T09:10:45.482278+0000","last_became_active":"2024-03-29T09:10:43.438260+0000","last_became_peered":"2024-03-29T09:10:43.438260+0000","last_unstale":"2024-03-29T09:10:45.482278+0000","last_undegraded":"2024-03-29T09:10:45.482278+0000","last_fullsized":"2024-03-29T09:10:45.482278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:50:41.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483295+0000","last_change":"2024-03-29T09:10:43.947454+0000","last_active":"2024-03-29T09:10:45.483295+0000","last_peered":"2024-03-29T09:10:45.483295+0000","last_clean":"2024-03-29T09:10:45.483295+0000","last_became_active":"2024-03-29T09:10:43.946720+0000","last_became_peered":"2024-03-29T09:10:43.946720+0000","last_unstale":"2024-03-29T09:10:45.483295+0000","last_undegraded":"2024-03-29T09:10:45.483295+0000","last_fullsized":"2024-03-29T09:10:45.483295+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:22:48.343086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482261+0000","last_change":"2024-03-29T09:10:43.434487+0000","last_active":"2024-03-29T09:10:45.482261+0000","last_peered":"2024-03-29T09:10:45.482261+0000","last_clean":"2024-03-29T09:10:45.482261+0000","last_became_active":"2024-03-29T09:10:43.434187+0000","last_became_peered":"2024-03-29T09:10:43.434187+0000","last_unstale":"2024-03-29T09:10:45.482261+0000","last_undegraded":"2024-03-29T09:10:45.482261+0000","last_fullsized":"2024-03-29T09:10:45.482261+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:32.957364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482054+0000","last_change":"2024-03-29T09:10:43.948597+0000","last_active":"2024-03-29T09:10:45.482054+0000","last_peered":"2024-03-29T09:10:45.482054+0000","last_clean":"2024-03-29T09:10:45.482054+0000","last_became_active":"2024-03-29T09:10:43.948279+0000","last_became_peered":"2024-03-29T09:10:43.948279+0000","last_unstale":"2024-03-29T09:10:45.482054+0000","last_undegraded":"2024-03-29T09:10:45.482054+0000","last_fullsized":"2024-03-29T09:10:45.482054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:47:59.924286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482626+0000","last_change":"2024-03-29T09:10:43.438020+0000","last_active":"2024-03-29T09:10:45.482626+0000","last_peered":"2024-03-29T09:10:45.482626+0000","last_clean":"2024-03-29T09:10:45.482626+0000","last_became_active":"2024-03-29T09:10:43.437719+0000","last_became_peered":"2024-03-29T09:10:43.437719+0000","last_unstale":"2024-03-29T09:10:45.482626+0000","last_undegraded":"2024-03-29T09:10:45.482626+0000","last_fullsized":"2024-03-29T09:10:45.482626+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:40.437583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934798+0000","last_change":"2024-03-29T09:10:43.959694+0000","last_active":"2024-03-29T09:10:44.934798+0000","last_peered":"2024-03-29T09:10:44.934798+0000","last_clean":"2024-03-29T09:10:44.934798+0000","last_became_active":"2024-03-29T09:10:43.959409+0000","last_became_peered":"2024-03-29T09:10:43.959409+0000","last_unstale":"2024-03-29T09:10:44.934798+0000","last_undegraded":"2024-03-29T09:10:44.934798+0000","last_fullsized":"2024-03-29T09:10:44.934798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:38:25.745338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482811+0000","last_change":"2024-03-29T09:10:43.445165+0000","last_active":"2024-03-29T09:10:45.482811+0000","last_peered":"2024-03-29T09:10:45.482811+0000","last_clean":"2024-03-29T09:10:45.482811+0000","last_became_active":"2024-03-29T09:10:43.444846+0000","last_became_peered":"2024-03-29T09:10:43.444846+0000","last_unstale":"2024-03-29T09:10:45.482811+0000","last_undegraded":"2024-03-29T09:10:45.482811+0000","last_fullsized":"2024-03-29T09:10:45.482811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:47.213511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935843+0000","last_change":"2024-03-29T09:10:43.949247+0000","last_active":"2024-03-29T09:10:44.935843+0000","last_peered":"2024-03-29T09:10:44.935843+0000","last_clean":"2024-03-29T09:10:44.935843+0000","last_became_active":"2024-03-29T09:10:43.948962+0000","last_became_peered":"2024-03-29T09:10:43.948962+0000","last_unstale":"2024-03-29T09:10:44.935843+0000","last_undegraded":"2024-03-29T09:10:44.935843+0000","last_fullsized":"2024-03-29T09:10:44.935843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:01:05.608373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482175+0000","last_change":"2024-03-29T09:10:43.449717+0000","last_active":"2024-03-29T09:10:45.482175+0000","last_peered":"2024-03-29T09:10:45.482175+0000","last_clean":"2024-03-29T09:10:45.482175+0000","last_became_active":"2024-03-29T09:10:43.449356+0000","last_became_peered":"2024-03-29T09:10:43.449356+0000","last_unstale":"2024-03-29T09:10:45.482175+0000","last_undegraded":"2024-03-29T09:10:45.482175+0000","last_fullsized":"2024-03-29T09:10:45.482175+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:08:59.932513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483328+0000","last_change":"2024-03-29T09:10:43.960030+0000","last_active":"2024-03-29T09:10:45.483328+0000","last_peered":"2024-03-29T09:10:45.483328+0000","last_clean":"2024-03-29T09:10:45.483328+0000","last_became_active":"2024-03-29T09:10:43.957125+0000","last_became_peered":"2024-03-29T09:10:43.957125+0000","last_unstale":"2024-03-29T09:10:45.483328+0000","last_undegraded":"2024-03-29T09:10:45.483328+0000","last_fullsized":"2024-03-29T09:10:45.483328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:04:40.145370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483195+0000","last_change":"2024-03-29T09:10:43.435943+0000","last_active":"2024-03-29T09:10:45.483195+0000","last_peered":"2024-03-29T09:10:45.483195+0000","last_clean":"2024-03-29T09:10:45.483195+0000","last_became_active":"2024-03-29T09:10:43.432791+0000","last_became_peered":"2024-03-29T09:10:43.432791+0000","last_unstale":"2024-03-29T09:10:45.483195+0000","last_undegraded":"2024-03-29T09:10:45.483195+0000","last_fullsized":"2024-03-29T09:10:45.483195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:59.156857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482403+0000","last_change":"2024-03-29T09:10:43.955539+0000","last_active":"2024-03-29T09:10:45.482403+0000","last_peered":"2024-03-29T09:10:45.482403+0000","last_clean":"2024-03-29T09:10:45.482403+0000","last_became_active":"2024-03-29T09:10:43.955197+0000","last_became_peered":"2024-03-29T09:10:43.955197+0000","last_unstale":"2024-03-29T09:10:45.482403+0000","last_undegraded":"2024-03-29T09:10:45.482403+0000","last_fullsized":"2024-03-29T09:10:45.482403+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:28:24.125633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935054+0000","last_change":"2024-03-29T09:10:43.444657+0000","last_active":"2024-03-29T09:10:44.935054+0000","last_peered":"2024-03-29T09:10:44.935054+0000","last_clean":"2024-03-29T09:10:44.935054+0000","last_became_active":"2024-03-29T09:10:43.444360+0000","last_became_peered":"2024-03-29T09:10:43.444360+0000","last_unstale":"2024-03-29T09:10:44.935054+0000","last_undegraded":"2024-03-29T09:10:44.935054+0000","last_fullsized":"2024-03-29T09:10:44.935054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:41:25.788448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483301+0000","last_change":"2024-03-29T09:10:43.962702+0000","last_active":"2024-03-29T09:10:45.483301+0000","last_peered":"2024-03-29T09:10:45.483301+0000","last_clean":"2024-03-29T09:10:45.483301+0000","last_became_active":"2024-03-29T09:10:43.962455+0000","last_became_peered":"2024-03-29T09:10:43.962455+0000","last_unstale":"2024-03-29T09:10:45.483301+0000","last_undegraded":"2024-03-29T09:10:45.483301+0000","last_fullsized":"2024-03-29T09:10:45.483301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:53:29.225074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934770+0000","last_change":"2024-03-29T09:10:43.439277+0000","last_active":"2024-03-29T09:10:44.934770+0000","last_peered":"2024-03-29T09:10:44.934770+0000","last_clean":"2024-03-29T09:10:44.934770+0000","last_became_active":"2024-03-29T09:10:43.439020+0000","last_became_peered":"2024-03-29T09:10:43.439020+0000","last_unstale":"2024-03-29T09:10:44.934770+0000","last_undegraded":"2024-03-29T09:10:44.934770+0000","last_fullsized":"2024-03-29T09:10:44.934770+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:04:39.138550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483085+0000","last_change":"2024-03-29T09:10:43.950979+0000","last_active":"2024-03-29T09:10:45.483085+0000","last_peered":"2024-03-29T09:10:45.483085+0000","last_clean":"2024-03-29T09:10:45.483085+0000","last_became_active":"2024-03-29T09:10:43.948037+0000","last_became_peered":"2024-03-29T09:10:43.948037+0000","last_unstale":"2024-03-29T09:10:45.483085+0000","last_undegraded":"2024-03-29T09:10:45.483085+0000","last_fullsized":"2024-03-29T09:10:45.483085+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:00.614898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483489+0000","last_change":"2024-03-29T09:10:43.436780+0000","last_active":"2024-03-29T09:10:45.483489+0000","last_peered":"2024-03-29T09:10:45.483489+0000","last_clean":"2024-03-29T09:10:45.483489+0000","last_became_active":"2024-03-29T09:10:43.433402+0000","last_became_peered":"2024-03-29T09:10:43.433402+0000","last_unstale":"2024-03-29T09:10:45.483489+0000","last_undegraded":"2024-03-29T09:10:45.483489+0000","last_fullsized":"2024-03-29T09:10:45.483489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:38:55.089471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482816+0000","last_change":"2024-03-29T09:10:43.949603+0000","last_active":"2024-03-29T09:10:45.482816+0000","last_peered":"2024-03-29T09:10:45.482816+0000","last_clean":"2024-03-29T09:10:45.482816+0000","last_became_active":"2024-03-29T09:10:43.946991+0000","last_became_peered":"2024-03-29T09:10:43.946991+0000","last_unstale":"2024-03-29T09:10:45.482816+0000","last_undegraded":"2024-03-29T09:10:45.482816+0000","last_fullsized":"2024-03-29T09:10:45.482816+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:15:58.350659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483052+0000","last_change":"2024-03-29T09:10:43.437463+0000","last_active":"2024-03-29T09:10:45.483052+0000","last_peered":"2024-03-29T09:10:45.483052+0000","last_clean":"2024-03-29T09:10:45.483052+0000","last_became_active":"2024-03-29T09:10:43.436908+0000","last_became_peered":"2024-03-29T09:10:43.436908+0000","last_unstale":"2024-03-29T09:10:45.483052+0000","last_undegraded":"2024-03-29T09:10:45.483052+0000","last_fullsized":"2024-03-29T09:10:45.483052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:38.731385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935697+0000","last_change":"2024-03-29T09:10:43.958436+0000","last_active":"2024-03-29T09:10:44.935697+0000","last_peered":"2024-03-29T09:10:44.935697+0000","last_clean":"2024-03-29T09:10:44.935697+0000","last_became_active":"2024-03-29T09:10:43.958142+0000","last_became_peered":"2024-03-29T09:10:43.958142+0000","last_unstale":"2024-03-29T09:10:44.935697+0000","last_undegraded":"2024-03-29T09:10:44.935697+0000","last_fullsized":"2024-03-29T09:10:44.935697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:07:34.257247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935897+0000","last_change":"2024-03-29T09:10:43.446061+0000","last_active":"2024-03-29T09:10:44.935897+0000","last_peered":"2024-03-29T09:10:44.935897+0000","last_clean":"2024-03-29T09:10:44.935897+0000","last_became_active":"2024-03-29T09:10:43.445026+0000","last_became_peered":"2024-03-29T09:10:43.445026+0000","last_unstale":"2024-03-29T09:10:44.935897+0000","last_undegraded":"2024-03-29T09:10:44.935897+0000","last_fullsized":"2024-03-29T09:10:44.935897+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:47:58.917466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482024+0000","last_change":"2024-03-29T09:10:43.943469+0000","last_active":"2024-03-29T09:10:45.482024+0000","last_peered":"2024-03-29T09:10:45.482024+0000","last_clean":"2024-03-29T09:10:45.482024+0000","last_became_active":"2024-03-29T09:10:43.943095+0000","last_became_peered":"2024-03-29T09:10:43.943095+0000","last_unstale":"2024-03-29T09:10:45.482024+0000","last_undegraded":"2024-03-29T09:10:45.482024+0000","last_fullsized":"2024-03-29T09:10:45.482024+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:36.397490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934932+0000","last_change":"2024-03-29T09:10:43.431068+0000","last_active":"2024-03-29T09:10:44.934932+0000","last_peered":"2024-03-29T09:10:44.934932+0000","last_clean":"2024-03-29T09:10:44.934932+0000","last_became_active":"2024-03-29T09:10:43.430298+0000","last_became_peered":"2024-03-29T09:10:43.430298+0000","last_unstale":"2024-03-29T09:10:44.934932+0000","last_undegraded":"2024-03-29T09:10:44.934932+0000","last_fullsized":"2024-03-29T09:10:44.934932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:22.625343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085174+0000","last_change":"2024-03-29T09:10:43.948473+0000","last_active":"2024-03-29T09:10:45.085174+0000","last_peered":"2024-03-29T09:10:45.085174+0000","last_clean":"2024-03-29T09:10:45.085174+0000","last_became_active":"2024-03-29T09:10:43.947817+0000","last_became_peered":"2024-03-29T09:10:43.947817+0000","last_unstale":"2024-03-29T09:10:45.085174+0000","last_undegraded":"2024-03-29T09:10:45.085174+0000","last_fullsized":"2024-03-29T09:10:45.085174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:13:35.536641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":19,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482011+0000","last_change":"2024-03-29T09:10:43.443511+0000","last_active":"2024-03-29T09:10:45.482011+0000","last_peered":"2024-03-29T09:10:45.482011+0000","last_clean":"2024-03-29T09:10:45.482011+0000","last_became_active":"2024-03-29T09:10:43.443245+0000","last_became_peered":"2024-03-29T09:10:43.443245+0000","last_unstale":"2024-03-29T09:10:45.482011+0000","last_undegraded":"2024-03-29T09:10:45.482011+0000","last_fullsized":"2024-03-29T09:10:45.482011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:43:26.573678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.936120+0000","last_change":"2024-03-29T09:10:43.957691+0000","last_active":"2024-03-29T09:10:44.936120+0000","last_peered":"2024-03-29T09:10:44.936120+0000","last_clean":"2024-03-29T09:10:44.936120+0000","last_became_active":"2024-03-29T09:10:43.957319+0000","last_became_peered":"2024-03-29T09:10:43.957319+0000","last_unstale":"2024-03-29T09:10:44.936120+0000","last_undegraded":"2024-03-29T09:10:44.936120+0000","last_fullsized":"2024-03-29T09:10:44.936120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:28.498042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483180+0000","last_change":"2024-03-29T09:10:43.447699+0000","last_active":"2024-03-29T09:10:45.483180+0000","last_peered":"2024-03-29T09:10:45.483180+0000","last_clean":"2024-03-29T09:10:45.483180+0000","last_became_active":"2024-03-29T09:10:43.447348+0000","last_became_peered":"2024-03-29T09:10:43.447348+0000","last_unstale":"2024-03-29T09:10:45.483180+0000","last_undegraded":"2024-03-29T09:10:45.483180+0000","last_fullsized":"2024-03-29T09:10:45.483180+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:10.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483054+0000","last_change":"2024-03-29T09:10:43.956864+0000","last_active":"2024-03-29T09:10:45.483054+0000","last_peered":"2024-03-29T09:10:45.483054+0000","last_clean":"2024-03-29T09:10:45.483054+0000","last_became_active":"2024-03-29T09:10:43.956592+0000","last_became_peered":"2024-03-29T09:10:43.956592+0000","last_unstale":"2024-03-29T09:10:45.483054+0000","last_undegraded":"2024-03-29T09:10:45.483054+0000","last_fullsized":"2024-03-29T09:10:45.483054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:03:28.859805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934994+0000","last_change":"2024-03-29T09:10:43.440428+0000","last_active":"2024-03-29T09:10:44.934994+0000","last_peered":"2024-03-29T09:10:44.934994+0000","last_clean":"2024-03-29T09:10:44.934994+0000","last_became_active":"2024-03-29T09:10:43.440176+0000","last_became_peered":"2024-03-29T09:10:43.440176+0000","last_unstale":"2024-03-29T09:10:44.934994+0000","last_undegraded":"2024-03-29T09:10:44.934994+0000","last_fullsized":"2024-03-29T09:10:44.934994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:53:28.218254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483174+0000","last_change":"2024-03-29T09:10:43.945026+0000","last_active":"2024-03-29T09:10:45.483174+0000","last_peered":"2024-03-29T09:10:45.483174+0000","last_clean":"2024-03-29T09:10:45.483174+0000","last_became_active":"2024-03-29T09:10:43.944760+0000","last_became_peered":"2024-03-29T09:10:43.944760+0000","last_unstale":"2024-03-29T09:10:45.483174+0000","last_undegraded":"2024-03-29T09:10:45.483174+0000","last_fullsized":"2024-03-29T09:10:45.483174+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:31:33.279465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085320+0000","last_change":"2024-03-29T09:10:43.447433+0000","last_active":"2024-03-29T09:10:45.085320+0000","last_peered":"2024-03-29T09:10:45.085320+0000","last_clean":"2024-03-29T09:10:45.085320+0000","last_became_active":"2024-03-29T09:10:43.447148+0000","last_became_peered":"2024-03-29T09:10:43.447148+0000","last_unstale":"2024-03-29T09:10:45.085320+0000","last_undegraded":"2024-03-29T09:10:45.085320+0000","last_fullsized":"2024-03-29T09:10:45.085320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:09.065578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483435+0000","last_change":"2024-03-29T09:10:43.955619+0000","last_active":"2024-03-29T09:10:45.483435+0000","last_peered":"2024-03-29T09:10:45.483435+0000","last_clean":"2024-03-29T09:10:45.483435+0000","last_became_active":"2024-03-29T09:10:43.952841+0000","last_became_peered":"2024-03-29T09:10:43.952841+0000","last_unstale":"2024-03-29T09:10:45.483435+0000","last_undegraded":"2024-03-29T09:10:45.483435+0000","last_fullsized":"2024-03-29T09:10:45.483435+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:39.738205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482714+0000","last_change":"2024-03-29T09:10:43.440484+0000","last_active":"2024-03-29T09:10:45.482714+0000","last_peered":"2024-03-29T09:10:45.482714+0000","last_clean":"2024-03-29T09:10:45.482714+0000","last_became_active":"2024-03-29T09:10:43.439983+0000","last_became_peered":"2024-03-29T09:10:43.439983+0000","last_unstale":"2024-03-29T09:10:45.482714+0000","last_undegraded":"2024-03-29T09:10:45.482714+0000","last_fullsized":"2024-03-29T09:10:45.482714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:24:26.545208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934637+0000","last_change":"2024-03-29T09:10:43.961322+0000","last_active":"2024-03-29T09:10:44.934637+0000","last_peered":"2024-03-29T09:10:44.934637+0000","last_clean":"2024-03-29T09:10:44.934637+0000","last_became_active":"2024-03-29T09:10:43.960606+0000","last_became_peered":"2024-03-29T09:10:43.960606+0000","last_unstale":"2024-03-29T09:10:44.934637+0000","last_undegraded":"2024-03-29T09:10:44.934637+0000","last_fullsized":"2024-03-29T09:10:44.934637+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:53:07.630538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482928+0000","last_change":"2024-03-29T09:10:43.436322+0000","last_active":"2024-03-29T09:10:45.482928+0000","last_peered":"2024-03-29T09:10:45.482928+0000","last_clean":"2024-03-29T09:10:45.482928+0000","last_became_active":"2024-03-29T09:10:43.435240+0000","last_became_peered":"2024-03-29T09:10:43.435240+0000","last_unstale":"2024-03-29T09:10:45.482928+0000","last_undegraded":"2024-03-29T09:10:45.482928+0000","last_fullsized":"2024-03-29T09:10:45.482928+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:17.862417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482814+0000","last_change":"2024-03-29T09:10:43.959943+0000","last_active":"2024-03-29T09:10:45.482814+0000","last_peered":"2024-03-29T09:10:45.482814+0000","last_clean":"2024-03-29T09:10:45.482814+0000","last_became_active":"2024-03-29T09:10:43.959596+0000","last_became_peered":"2024-03-29T09:10:43.959596+0000","last_unstale":"2024-03-29T09:10:45.482814+0000","last_undegraded":"2024-03-29T09:10:45.482814+0000","last_fullsized":"2024-03-29T09:10:45.482814+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:16.539862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.936024+0000","last_change":"2024-03-29T09:10:43.442957+0000","last_active":"2024-03-29T09:10:44.936024+0000","last_peered":"2024-03-29T09:10:44.936024+0000","last_clean":"2024-03-29T09:10:44.936024+0000","last_became_active":"2024-03-29T09:10:43.442135+0000","last_became_peered":"2024-03-29T09:10:43.442135+0000","last_unstale":"2024-03-29T09:10:44.936024+0000","last_undegraded":"2024-03-29T09:10:44.936024+0000","last_fullsized":"2024-03-29T09:10:44.936024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:24:26.545208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482197+0000","last_change":"2024-03-29T09:10:43.948397+0000","last_active":"2024-03-29T09:10:45.482197+0000","last_peered":"2024-03-29T09:10:45.482197+0000","last_clean":"2024-03-29T09:10:45.482197+0000","last_became_active":"2024-03-29T09:10:43.948133+0000","last_became_peered":"2024-03-29T09:10:43.948133+0000","last_unstale":"2024-03-29T09:10:45.482197+0000","last_undegraded":"2024-03-29T09:10:45.482197+0000","last_fullsized":"2024-03-29T09:10:45.482197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:48.220331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482888+0000","last_change":"2024-03-29T09:10:43.447717+0000","last_active":"2024-03-29T09:10:45.482888+0000","last_peered":"2024-03-29T09:10:45.482888+0000","last_clean":"2024-03-29T09:10:45.482888+0000","last_became_active":"2024-03-29T09:10:43.447407+0000","last_became_peered":"2024-03-29T09:10:43.447407+0000","last_unstale":"2024-03-29T09:10:45.482888+0000","last_undegraded":"2024-03-29T09:10:45.482888+0000","last_fullsized":"2024-03-29T09:10:45.482888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:23.628369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483573+0000","last_change":"2024-03-29T09:10:43.959637+0000","last_active":"2024-03-29T09:10:45.483573+0000","last_peered":"2024-03-29T09:10:45.483573+0000","last_clean":"2024-03-29T09:10:45.483573+0000","last_became_active":"2024-03-29T09:10:43.959127+0000","last_became_peered":"2024-03-29T09:10:43.959127+0000","last_unstale":"2024-03-29T09:10:45.483573+0000","last_undegraded":"2024-03-29T09:10:45.483573+0000","last_fullsized":"2024-03-29T09:10:45.483573+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:59:33.808837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935596+0000","last_change":"2024-03-29T09:10:43.439933+0000","last_active":"2024-03-29T09:10:44.935596+0000","last_peered":"2024-03-29T09:10:44.935596+0000","last_clean":"2024-03-29T09:10:44.935596+0000","last_became_active":"2024-03-29T09:10:43.439587+0000","last_became_peered":"2024-03-29T09:10:43.439587+0000","last_unstale":"2024-03-29T09:10:44.935596+0000","last_undegraded":"2024-03-29T09:10:44.935596+0000","last_fullsized":"2024-03-29T09:10:44.935596+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:17.862417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085256+0000","last_change":"2024-03-29T09:10:43.944305+0000","last_active":"2024-03-29T09:10:45.085256+0000","last_peered":"2024-03-29T09:10:45.085256+0000","last_clean":"2024-03-29T09:10:45.085256+0000","last_became_active":"2024-03-29T09:10:43.943523+0000","last_became_peered":"2024-03-29T09:10:43.943523+0000","last_unstale":"2024-03-29T09:10:45.085256+0000","last_undegraded":"2024-03-29T09:10:45.085256+0000","last_fullsized":"2024-03-29T09:10:45.085256+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:39:37.365296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935117+0000","last_change":"2024-03-29T09:10:43.436112+0000","last_active":"2024-03-29T09:10:44.935117+0000","last_peered":"2024-03-29T09:10:44.935117+0000","last_clean":"2024-03-29T09:10:44.935117+0000","last_became_active":"2024-03-29T09:10:43.435866+0000","last_became_peered":"2024-03-29T09:10:43.435866+0000","last_unstale":"2024-03-29T09:10:44.935117+0000","last_undegraded":"2024-03-29T09:10:44.935117+0000","last_fullsized":"2024-03-29T09:10:44.935117+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:15.533042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483155+0000","last_change":"2024-03-29T09:10:43.950602+0000","last_active":"2024-03-29T09:10:45.483155+0000","last_peered":"2024-03-29T09:10:45.483155+0000","last_clean":"2024-03-29T09:10:45.483155+0000","last_became_active":"2024-03-29T09:10:43.949767+0000","last_became_peered":"2024-03-29T09:10:43.949767+0000","last_unstale":"2024-03-29T09:10:45.483155+0000","last_undegraded":"2024-03-29T09:10:45.483155+0000","last_fullsized":"2024-03-29T09:10:45.483155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:39.738205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.624478+0000","last_change":"2024-03-29T09:10:43.449529+0000","last_active":"2024-03-29T09:10:45.624478+0000","last_peered":"2024-03-29T09:10:45.624478+0000","last_clean":"2024-03-29T09:10:45.624478+0000","last_became_active":"2024-03-29T09:10:43.449326+0000","last_became_peered":"2024-03-29T09:10:43.449326+0000","last_unstale":"2024-03-29T09:10:45.624478+0000","last_undegraded":"2024-03-29T09:10:45.624478+0000","last_fullsized":"2024-03-29T09:10:45.624478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:53.073776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483046+0000","last_change":"2024-03-29T09:10:43.952895+0000","last_active":"2024-03-29T09:10:45.483046+0000","last_peered":"2024-03-29T09:10:45.483046+0000","last_clean":"2024-03-29T09:10:45.483046+0000","last_became_active":"2024-03-29T09:10:43.952592+0000","last_became_peered":"2024-03-29T09:10:43.952592+0000","last_unstale":"2024-03-29T09:10:45.483046+0000","last_undegraded":"2024-03-29T09:10:45.483046+0000","last_fullsized":"2024-03-29T09:10:45.483046+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:37.550795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085322+0000","last_change":"2024-03-29T09:10:43.442236+0000","last_active":"2024-03-29T09:10:45.085322+0000","last_peered":"2024-03-29T09:10:45.085322+0000","last_clean":"2024-03-29T09:10:45.085322+0000","last_became_active":"2024-03-29T09:10:43.441945+0000","last_became_peered":"2024-03-29T09:10:43.441945+0000","last_unstale":"2024-03-29T09:10:45.085322+0000","last_undegraded":"2024-03-29T09:10:45.085322+0000","last_fullsized":"2024-03-29T09:10:45.085322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:43:26.046153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.085126+0000","last_change":"2024-03-29T09:10:43.948081+0000","last_active":"2024-03-29T09:10:45.085126+0000","last_peered":"2024-03-29T09:10:45.085126+0000","last_clean":"2024-03-29T09:10:45.085126+0000","last_became_active":"2024-03-29T09:10:43.947814+0000","last_became_peered":"2024-03-29T09:10:43.947814+0000","last_unstale":"2024-03-29T09:10:45.085126+0000","last_undegraded":"2024-03-29T09:10:45.085126+0000","last_fullsized":"2024-03-29T09:10:45.085126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:26.500284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.483179+0000","last_change":"2024-03-29T09:10:43.437647+0000","last_active":"2024-03-29T09:10:45.483179+0000","last_peered":"2024-03-29T09:10:45.483179+0000","last_clean":"2024-03-29T09:10:45.483179+0000","last_became_active":"2024-03-29T09:10:43.437293+0000","last_became_peered":"2024-03-29T09:10:43.437293+0000","last_unstale":"2024-03-29T09:10:45.483179+0000","last_undegraded":"2024-03-29T09:10:45.483179+0000","last_fullsized":"2024-03-29T09:10:45.483179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:19:36.543975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935804+0000","last_change":"2024-03-29T09:10:43.955415+0000","last_active":"2024-03-29T09:10:44.935804+0000","last_peered":"2024-03-29T09:10:44.935804+0000","last_clean":"2024-03-29T09:10:44.935804+0000","last_became_active":"2024-03-29T09:10:43.955107+0000","last_became_peered":"2024-03-29T09:10:43.955107+0000","last_unstale":"2024-03-29T09:10:44.935804+0000","last_undegraded":"2024-03-29T09:10:44.935804+0000","last_fullsized":"2024-03-29T09:10:44.935804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:58.717598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482352+0000","last_change":"2024-03-29T09:10:43.442152+0000","last_active":"2024-03-29T09:10:45.482352+0000","last_peered":"2024-03-29T09:10:45.482352+0000","last_clean":"2024-03-29T09:10:45.482352+0000","last_became_active":"2024-03-29T09:10:43.441862+0000","last_became_peered":"2024-03-29T09:10:43.441862+0000","last_unstale":"2024-03-29T09:10:45.482352+0000","last_undegraded":"2024-03-29T09:10:45.482352+0000","last_fullsized":"2024-03-29T09:10:45.482352+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:32:25.493464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.934910+0000","last_change":"2024-03-29T09:10:43.945931+0000","last_active":"2024-03-29T09:10:44.934910+0000","last_peered":"2024-03-29T09:10:44.934910+0000","last_clean":"2024-03-29T09:10:44.934910+0000","last_became_active":"2024-03-29T09:10:43.945673+0000","last_became_peered":"2024-03-29T09:10:43.945673+0000","last_unstale":"2024-03-29T09:10:44.934910+0000","last_undegraded":"2024-03-29T09:10:44.934910+0000","last_fullsized":"2024-03-29T09:10:44.934910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:26:20.813311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935017+0000","last_change":"2024-03-29T09:10:43.439915+0000","last_active":"2024-03-29T09:10:44.935017+0000","last_peered":"2024-03-29T09:10:44.935017+0000","last_clean":"2024-03-29T09:10:44.935017+0000","last_became_active":"2024-03-29T09:10:43.439669+0000","last_became_peered":"2024-03-29T09:10:43.439669+0000","last_unstale":"2024-03-29T09:10:44.935017+0000","last_undegraded":"2024-03-29T09:10:44.935017+0000","last_fullsized":"2024-03-29T09:10:44.935017+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:59:32.802017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482501+0000","last_change":"2024-03-29T09:10:43.954985+0000","last_active":"2024-03-29T09:10:45.482501+0000","last_peered":"2024-03-29T09:10:45.482501+0000","last_clean":"2024-03-29T09:10:45.482501+0000","last_became_active":"2024-03-29T09:10:43.954481+0000","last_became_peered":"2024-03-29T09:10:43.954481+0000","last_unstale":"2024-03-29T09:10:45.482501+0000","last_undegraded":"2024-03-29T09:10:45.482501+0000","last_fullsized":"2024-03-29T09:10:45.482501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:38:56.096291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482662+0000","last_change":"2024-03-29T09:10:43.440605+0000","last_active":"2024-03-29T09:10:45.482662+0000","last_peered":"2024-03-29T09:10:45.482662+0000","last_clean":"2024-03-29T09:10:45.482662+0000","last_became_active":"2024-03-29T09:10:43.440305+0000","last_became_peered":"2024-03-29T09:10:43.440305+0000","last_unstale":"2024-03-29T09:10:45.482662+0000","last_undegraded":"2024-03-29T09:10:45.482662+0000","last_fullsized":"2024-03-29T09:10:45.482662+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:53.101706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":10,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:44.935055+0000","last_change":"2024-03-29T09:10:43.946940+0000","last_active":"2024-03-29T09:10:44.935055+0000","last_peered":"2024-03-29T09:10:44.935055+0000","last_clean":"2024-03-29T09:10:44.935055+0000","last_became_active":"2024-03-29T09:10:43.946634+0000","last_became_peered":"2024-03-29T09:10:43.946634+0000","last_unstale":"2024-03-29T09:10:44.935055+0000","last_undegraded":"2024-03-29T09:10:44.935055+0000","last_fullsized":"2024-03-29T09:10:44.935055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:43.411059+0000","last_clean_scrub_stamp":"2024-03-29T09:10:43.411059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:32.345021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-29T09:10:45.482496+0000","last_change":"2024-03-29T09:10:43.440385+0000","last_active":"2024-03-29T09:10:45.482496+0000","last_peered":"2024-03-29T09:10:45.482496+0000","last_clean":"2024-03-29T09:10:45.482496+0000","last_became_active":"2024-03-29T09:10:43.440059+0000","last_became_peered":"2024-03-29T09:10:43.440059+0000","last_unstale":"2024-03-29T09:10:45.482496+0000","last_undegraded":"2024-03-29T09:10:45.482496+0000","last_fullsized":"2024-03-29T09:10:45.482496+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:10:42.404239+0000","last_clean_scrub_stamp":"2024-03-29T09:10:42.404239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:43.321784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":180,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-03-29T09:10:43.934802+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:10:43.934802+0000","last_peered":"2024-03-29T09:10:43.934802+0000","last_clean":"2024-03-29T09:10:43.934802+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:10:43.934802+0000","last_undegraded":"2024-03-29T09:10:43.934802+0000","last_fullsized":"2024-03-29T09:10:43.934802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":193500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.8380000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.8740000000000001,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.75}]},{"osd":5,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.83699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":193464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.621},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.58899999999999997}]},{"osd":2,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.2599999999999998,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":3.1070000000000002,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:09:51 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.51400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":783820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.308},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.0119999999999996,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":4.2050000000000001,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.9809999999999999,"5min":3.9809999999999999,"15min":3.9809999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.82599999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.9529999999999998,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.81499999999999995}]},{"osd":6,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.9140000000000001,"5min":3.9140000000000001,"15min":3.9140000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.745},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.78400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":193522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.41299999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.308},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48599999999999999}]},{"osd":4,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.57899999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.52900000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":195576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.443}]},{"osd":3,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.415,"5min":2.415,"15min":2.415},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.624},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.996,"5min":2.996,"15min":2.996},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:09:50 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.64700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.84699999999999998}]},{"osd":5,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.88400000000000001}]},{"osd":6,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.751}]},{"osd":7,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.90500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":471040,"data_stored":195118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.5790000000000002,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.81299999999999994}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":784380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Mar 29 09:09:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.754},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.70299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:10:48.780 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T09:10:48.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T09:10:49.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.012+0000 7fb8c0400700 1 Processor -- start 2024-03-29T09:10:49.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.013+0000 7fb8c0400700 1 -- start start 2024-03-29T09:10:49.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.014+0000 7fb8c0400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb8b815d2f0 0x7fb8b80d3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.014+0000 7fb8c0400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80cf5d0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.014+0000 7fb8c0400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80ce6f0 con 0x7fb8b80d3550 2024-03-29T09:10:49.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.014+0000 7fb8c0400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b80ce9c0 con 0x7fb8b815d2f0 2024-03-29T09:10:49.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.014+0000 7fb8be400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb8b80ce2e0 0x7fb8b80b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46530/0 (socket says 172.21.15.99:46530) 2024-03-29T09:10:49.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.014+0000 7fb8be400700 1 -- 172.21.15.99:0/3015297826 learned_addr learned my addr 172.21.15.99:0/3015297826 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:49.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.014+0000 7fb8bda00700 1 --2- 172.21.15.99:0/3015297826 >> v2:172.21.15.133:3300/0 conn(0x7fb8b815d2f0 0x7fb8b80d3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8b7e00700 1 -- 172.21.15.99:0/3015297826 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427068992 0 0) 0x7fb8b80cf5d0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8b7e00700 1 -- 172.21.15.99:0/3015297826 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8a00035b0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8bda00700 1 -- 172.21.15.99:0/3015297826 >> v1:172.21.15.133:6789/0 conn(0x7fb8b80d3550 legacy=0x7fb8b80cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8bda00700 1 -- 172.21.15.99:0/3015297826 >> v1:172.21.15.99:6789/0 conn(0x7fb8b80ce2e0 legacy=0x7fb8b80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8bda00700 1 -- 172.21.15.99:0/3015297826 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80ceca0 con 0x7fb8b815d2f0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8bda00700 1 --2- 172.21.15.99:0/3015297826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b815d2f0 0x7fb8b80d3010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009840 tx=0x7fb8ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff0708b8d1830835 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8b7e00700 1 -- 172.21.15.99:0/3015297826 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac011830 con 0x7fb8b815d2f0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8b7e00700 1 -- 172.21.15.99:0/3015297826 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac011990 con 0x7fb8b815d2f0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.015+0000 7fb8b7e00700 1 -- 172.21.15.99:0/3015297826 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac011d20 con 0x7fb8b815d2f0 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.016+0000 7fb8c0400700 1 -- 172.21.15.99:0/3015297826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b815d2f0 msgr2=0x7fb8b80d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.016+0000 7fb8c0400700 1 --2- 172.21.15.99:0/3015297826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b815d2f0 0x7fb8b80d3010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009840 tx=0x7fb8ac009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:49.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.016+0000 7fb8c0400700 1 -- 172.21.15.99:0/3015297826 shutdown_connections 2024-03-29T09:10:49.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.016+0000 7fb8c0400700 1 --2- 172.21.15.99:0/3015297826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b815d2f0 0x7fb8b80d3010 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.016+0000 7fb8c0400700 1 -- 172.21.15.99:0/3015297826 >> 172.21.15.99:0/3015297826 conn(0x7fb8b808afd0 msgr2=0x7fb8b809c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:49.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.016+0000 7fb8c0400700 1 -- 172.21.15.99:0/3015297826 shutdown_connections 2024-03-29T09:10:49.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.016+0000 7fb8c0400700 1 -- 172.21.15.99:0/3015297826 wait complete. 2024-03-29T09:10:49.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.018+0000 7fb8c0400700 1 Processor -- start 2024-03-29T09:10:49.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.018+0000 7fb8c0400700 1 -- start start 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8c0400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8b80ce2e0 0x7fb8b80c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8c0400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b80d3550 0x7fb8b80c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8c0400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8b815d2f0 0x7fb8b80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8c0400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b815d970 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8c0400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80c8010 con 0x7fb8b80d3550 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8c0400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8160720 con 0x7fb8b815d2f0 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8b80ce2e0 0x7fb8b80c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8b80ce2e0 0x7fb8b80c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33234/0 (socket says 172.21.15.99:33234) 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 -- 172.21.15.99:0/1611125409 learned_addr learned my addr 172.21.15.99:0/1611125409 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bd000700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b80d3550 0x7fb8b80c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 -- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8b815d2f0 msgr2=0x7fb8b80c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8b815d2f0 0x7fb8b80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 -- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b80d3550 msgr2=0x7fb8b80c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b80d3550 0x7fb8b80c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bda00700 1 -- 172.21.15.99:0/1611125409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0094a0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.019+0000 7fb8bd000700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b80d3550 0x7fb8b80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.020+0000 7fb8bda00700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8b80ce2e0 0x7fb8b80c8950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0090b0 tx=0x7fb8ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.020+0000 7fb8b5600700 1 -- 172.21.15.99:0/1611125409 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0036a0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.020+0000 7fb8b5600700 1 -- 172.21.15.99:0/1611125409 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac01e070 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.020+0000 7fb8b5600700 1 -- 172.21.15.99:0/1611125409 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac003a60 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.020+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b80c76e0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.020+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b80c7ca0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.021+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b80678b0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.022+0000 7fb8b5600700 1 -- 172.21.15.99:0/1611125409 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb8ac003bc0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.023+0000 7fb8b5600700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb89806ac90 0x7fb89806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.023+0000 7fb8b5600700 1 -- 172.21.15.99:0/1611125409 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb8ac08a4e0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.023+0000 7fb8bd000700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb89806ac90 0x7fb89806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.023+0000 7fb8bd000700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb89806ac90 0x7fb89806d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4001710 tx=0x7fb8a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:49.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.027+0000 7fb8b5600700 1 -- 172.21.15.99:0/1611125409 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb8ac055570 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.312+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb8b80cb2b0 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.316+0000 7fb8b5600700 1 -- 172.21.15.99:0/1611125409 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb8ac058b90 con 0x7fb8b80ce2e0 2024-03-29T09:10:49.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb89806ac90 msgr2=0x7fb89806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb89806ac90 0x7fb89806d150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4001710 tx=0x7fb8a4009040 comp rx=0 tx=0).stop 2024-03-29T09:10:49.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8b80ce2e0 msgr2=0x7fb8b80c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8b80ce2e0 0x7fb8b80c8950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0090b0 tx=0x7fb8ac004750 comp rx=0 tx=0).stop 2024-03-29T09:10:49.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 shutdown_connections 2024-03-29T09:10:49.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb89806ac90 0x7fb89806d150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb8b815d2f0 0x7fb8b80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8b80ce2e0 0x7fb8b80c8950 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 --2- 172.21.15.99:0/1611125409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8b80d3550 0x7fb8b80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 >> 172.21.15.99:0/1611125409 conn(0x7fb8b808afd0 msgr2=0x7fb8b80d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 shutdown_connections 2024-03-29T09:10:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.319+0000 7fb8c0400700 1 -- 172.21.15.99:0/1611125409 wait complete. 2024-03-29T09:10:49.320 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:10:49.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:10:49.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.526+0000 7ffa85c00700 1 Processor -- start 2024-03-29T09:10:49.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.526+0000 7ffa85c00700 1 -- start start 2024-03-29T09:10:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa85c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa8015db00 0x7ffa800ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa85c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015df90 0x7ffa800ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa85c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa800cd820 con 0x7ffa80143940 2024-03-29T09:10:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa85c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa80143d20 con 0x7ffa8015db00 2024-03-29T09:10:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa85c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8015eaf0 con 0x7ffa8015df90 2024-03-29T09:10:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa77e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa8015db00 0x7ffa800ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa7f400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015df90 0x7ffa800ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa77e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa8015db00 0x7ffa800ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:38130/0 (socket says 172.21.15.99:38130) 2024-03-29T09:10:49.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.528+0000 7ffa77e00700 1 -- 172.21.15.99:0/718388861 learned_addr learned my addr 172.21.15.99:0/718388861 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:49.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7f400700 1 -- 172.21.15.99:0/718388861 >> v2:172.21.15.133:3300/0 conn(0x7ffa8015db00 msgr2=0x7ffa800ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:49.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7f400700 1 --2- 172.21.15.99:0/718388861 >> v2:172.21.15.133:3300/0 conn(0x7ffa8015db00 0x7ffa800ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7f400700 1 -- 172.21.15.99:0/718388861 >> v1:172.21.15.133:6789/0 conn(0x7ffa80143940 legacy=0x7ffa80070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7f400700 1 -- 172.21.15.99:0/718388861 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8015e530 con 0x7ffa8015df90 2024-03-29T09:10:49.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa77e00700 1 --2- 172.21.15.99:0/718388861 >> v2:172.21.15.133:3300/0 conn(0x7ffa8015db00 0x7ffa800ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:49.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7f400700 1 --2- 172.21.15.99:0/718388861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015df90 0x7ffa800ce800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009d90 tx=0x7ffa7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73abd5475dd0fc10 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:49.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7e000700 1 -- 172.21.15.99:0/718388861 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7800bd20 con 0x7ffa8015df90 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7e000700 1 -- 172.21.15.99:0/718388861 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7800be80 con 0x7ffa8015df90 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.529+0000 7ffa7e000700 1 -- 172.21.15.99:0/718388861 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa780116a0 con 0x7ffa8015df90 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 -- 172.21.15.99:0/718388861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015df90 msgr2=0x7ffa800ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 --2- 172.21.15.99:0/718388861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015df90 0x7ffa800ce800 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009d90 tx=0x7ffa7800b330 comp rx=0 tx=0).stop 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 -- 172.21.15.99:0/718388861 shutdown_connections 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 --2- 172.21.15.99:0/718388861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015df90 0x7ffa800ce800 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 --2- 172.21.15.99:0/718388861 >> v2:172.21.15.133:3300/0 conn(0x7ffa8015db00 0x7ffa800ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 -- 172.21.15.99:0/718388861 >> 172.21.15.99:0/718388861 conn(0x7ffa80066540 msgr2=0x7ffa80066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 -- 172.21.15.99:0/718388861 shutdown_connections 2024-03-29T09:10:49.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.530+0000 7ffa85c00700 1 -- 172.21.15.99:0/718388861 wait complete. 2024-03-29T09:10:49.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.532+0000 7ffa85c00700 1 Processor -- start 2024-03-29T09:10:49.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.532+0000 7ffa85c00700 1 -- start start 2024-03-29T09:10:49.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa85c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80143940 0x7ffa80136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa85c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015db00 0x7ffa8015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa85c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa8015df90 0x7ffa80138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa85c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80137770 con 0x7ffa80143940 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa85c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80137a50 con 0x7ffa8015df90 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa85c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80137d30 con 0x7ffa8015db00 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa7ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80143940 0x7ffa80136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa7ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80143940 0x7ffa80136eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33240/0 (socket says 172.21.15.99:33240) 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa7ea00700 1 -- 172.21.15.99:0/2097619174 learned_addr learned my addr 172.21.15.99:0/2097619174 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa77e00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015db00 0x7ffa8015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.533+0000 7ffa7f400700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa8015df90 0x7ffa80138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa7ea00700 1 -- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015db00 msgr2=0x7ffa8015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa7ea00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015db00 0x7ffa8015cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa7ea00700 1 -- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa8015df90 msgr2=0x7ffa80138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa7ea00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa8015df90 0x7ffa80138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa7ea00700 1 -- 172.21.15.99:0/2097619174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa68009550 con 0x7ffa80143940 2024-03-29T09:10:49.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa7ea00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80143940 0x7ffa80136eb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000c4e0 tx=0x7ffa7000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:49.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa76a00700 1 -- 172.21.15.99:0/2097619174 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa700052d0 con 0x7ffa80143940 2024-03-29T09:10:49.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa76a00700 1 -- 172.21.15.99:0/2097619174 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa70005430 con 0x7ffa80143940 2024-03-29T09:10:49.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa76a00700 1 -- 172.21.15.99:0/2097619174 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa700038c0 con 0x7ffa80143940 2024-03-29T09:10:49.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa78009a70 con 0x7ffa80143940 2024-03-29T09:10:49.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.534+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa801382f0 con 0x7ffa80143940 2024-03-29T09:10:49.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.535+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa800cd1d0 con 0x7ffa80143940 2024-03-29T09:10:49.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.536+0000 7ffa76a00700 1 -- 172.21.15.99:0/2097619174 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ffa70003a20 con 0x7ffa80143940 2024-03-29T09:10:49.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.538+0000 7ffa76a00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffa5c06acc0 0x7ffa5c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:49.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.538+0000 7ffa76a00700 1 -- 172.21.15.99:0/2097619174 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffa70088810 con 0x7ffa80143940 2024-03-29T09:10:49.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.538+0000 7ffa77e00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffa5c06acc0 0x7ffa5c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:49.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.538+0000 7ffa77e00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffa5c06acc0 0x7ffa5c06d180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa800b5390 tx=0x7ffa680092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:49.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.542+0000 7ffa76a00700 1 -- 172.21.15.99:0/2097619174 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ffa700538a0 con 0x7ffa80143940 2024-03-29T09:10:49.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.789+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa800677a0 con 0x7ffa80143940 2024-03-29T09:10:49.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.789+0000 7ffa76a00700 1 -- 172.21.15.99:0/2097619174 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4740 (secure 0 0 0) 0x7ffa70056ec0 con 0x7ffa80143940 2024-03-29T09:10:49.789 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:49.790 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:10:43.923687+0000","modified":"2024-03-29T09:10:46.934239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/1841077010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1841077010},{"type":"v1","addr":"172.21.15.99:6835","nonce":1841077010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T09:10:49.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffa5c06acc0 msgr2=0x7ffa5c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffa5c06acc0 0x7ffa5c06d180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa800b5390 tx=0x7ffa680092c0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80143940 msgr2=0x7ffa80136eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:49.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80143940 0x7ffa80136eb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffa7000c4e0 tx=0x7ffa7000c7f0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 shutdown_connections 2024-03-29T09:10:49.792 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ffa5c06acc0 0x7ffa5c06d180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa8015df90 0x7ffa80138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa80143940 0x7ffa80136eb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 --2- 172.21.15.99:0/2097619174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa8015db00 0x7ffa8015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:49.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 >> 172.21.15.99:0/2097619174 conn(0x7ffa80066540 msgr2=0x7ffa8013efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:49.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 shutdown_connections 2024-03-29T09:10:49.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:49.792+0000 7ffa85c00700 1 -- 172.21.15.99:0/2097619174 wait complete. 2024-03-29T09:10:49.793 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 16 2024-03-29T09:10:50.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T09:10:43.923687+0000', 'modified': '2024-03-29T09:10:46.934239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5433}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5433': {'gid': 5433, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.99:6835/1841077010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 1841077010}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 1841077010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T09:10:50.213 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T09:10:50.213 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T09:10:50.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.404+0000 7fd85ba00700 1 Processor -- start 2024-03-29T09:10:50.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.405+0000 7fd85ba00700 1 -- start start 2024-03-29T09:10:50.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd85ba00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd8541608e0 0x7fd8540d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd85ba00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540cddc0 con 0x7fd854160cc0 2024-03-29T09:10:50.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd85ba00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540ce0a0 con 0x7fd8540d1b20 2024-03-29T09:10:50.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd85ba00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8540ce380 con 0x7fd8541608e0 2024-03-29T09:10:50.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd859000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd8541608e0 0x7fd8540d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd853e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd854160cc0 0x7fd8540d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46534/0 (socket says 172.21.15.99:46534) 2024-03-29T09:10:50.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd853e00700 1 -- 172.21.15.99:0/3380354226 learned_addr learned my addr 172.21.15.99:0/3380354226 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd859000700 1 -- 172.21.15.99:0/3380354226 >> v1:172.21.15.133:6789/0 conn(0x7fd8540d1b20 legacy=0x7fd8540afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd859000700 1 -- 172.21.15.99:0/3380354226 >> v1:172.21.15.99:6789/0 conn(0x7fd854160cc0 legacy=0x7fd8540d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd859000700 1 -- 172.21.15.99:0/3380354226 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540ce660 con 0x7fd8541608e0 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.406+0000 7fd853400700 1 -- 172.21.15.99:0/3380354226 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231679150 0 0) 0x7fd8540cddc0 con 0x7fd854160cc0 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd859000700 1 --2- 172.21.15.99:0/3380354226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8541608e0 0x7fd8540d0150 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd848009840 tx=0x7fd848009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec4d6199e769953 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd853400700 1 -- 172.21.15.99:0/3380354226 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848004230 con 0x7fd8541608e0 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd853400700 1 -- 172.21.15.99:0/3380354226 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848004390 con 0x7fd8541608e0 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd853400700 1 -- 172.21.15.99:0/3380354226 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480036a0 con 0x7fd8541608e0 2024-03-29T09:10:50.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd85ba00700 1 -- 172.21.15.99:0/3380354226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8541608e0 msgr2=0x7fd8540d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd85ba00700 1 --2- 172.21.15.99:0/3380354226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8541608e0 0x7fd8540d0150 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd848009840 tx=0x7fd848009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd85ba00700 1 -- 172.21.15.99:0/3380354226 shutdown_connections 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd85ba00700 1 --2- 172.21.15.99:0/3380354226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8541608e0 0x7fd8540d0150 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.407+0000 7fd85ba00700 1 -- 172.21.15.99:0/3380354226 >> 172.21.15.99:0/3380354226 conn(0x7fd85409c490 msgr2=0x7fd8540d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.408+0000 7fd85ba00700 1 -- 172.21.15.99:0/3380354226 shutdown_connections 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.408+0000 7fd85ba00700 1 -- 172.21.15.99:0/3380354226 wait complete. 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.409+0000 7fd85ba00700 1 Processor -- start 2024-03-29T09:10:50.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.409+0000 7fd85ba00700 1 -- start start 2024-03-29T09:10:50.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.410+0000 7fd85ba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8540d1b20 0x7fd854157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.410+0000 7fd85ba00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8541608e0 0x7fd854157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.410+0000 7fd85ba00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd854160cc0 0x7fd8540cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.410+0000 7fd85ba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540c9660 con 0x7fd8540d1b20 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.410+0000 7fd85ba00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd854158390 con 0x7fd854160cc0 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.410+0000 7fd85ba00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd854158670 con 0x7fd8541608e0 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.410+0000 7fd859000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8540d1b20 0x7fd854157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8540d1b20 0x7fd854157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33262/0 (socket says 172.21.15.99:33262) 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 -- 172.21.15.99:0/1580531510 learned_addr learned my addr 172.21.15.99:0/1580531510 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859a00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd854160cc0 0x7fd8540cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd853e00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8541608e0 0x7fd854157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 -- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8541608e0 msgr2=0x7fd854157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8541608e0 0x7fd854157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 -- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd854160cc0 msgr2=0x7fd8540cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd854160cc0 0x7fd8540cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 -- 172.21.15.99:0/1580531510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480094a0 con 0x7fd8540d1b20 2024-03-29T09:10:50.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.411+0000 7fd859000700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8540d1b20 0x7fd854157620 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd848009b90 tx=0x7fd848004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.412+0000 7fd850c00700 1 -- 172.21.15.99:0/1580531510 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848011630 con 0x7fd8540d1b20 2024-03-29T09:10:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.412+0000 7fd850c00700 1 -- 172.21.15.99:0/1580531510 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84801e070 con 0x7fd8540d1b20 2024-03-29T09:10:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.412+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd854158950 con 0x7fd8540d1b20 2024-03-29T09:10:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.412+0000 7fd850c00700 1 -- 172.21.15.99:0/1580531510 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480119b0 con 0x7fd8540d1b20 2024-03-29T09:10:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.412+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8541580a0 con 0x7fd8540d1b20 2024-03-29T09:10:50.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.414+0000 7fd850c00700 1 -- 172.21.15.99:0/1580531510 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd848011b10 con 0x7fd8540d1b20 2024-03-29T09:10:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.414+0000 7fd850c00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd83406ac90 0x7fd83406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.414+0000 7fd850c00700 1 -- 172.21.15.99:0/1580531510 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd84808a950 con 0x7fd8540d1b20 2024-03-29T09:10:50.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.415+0000 7fd853e00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd83406ac90 0x7fd83406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.415+0000 7fd853e00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd83406ac90 0x7fd83406d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd85414c060 tx=0x7fd840007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.416+0000 7fd86f4b2b80 1 Processor -- start 2024-03-29T09:10:50.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.417+0000 7fd86f4b2b80 1 -- start start 2024-03-29T09:10:50.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.417+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f675a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.417+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f64d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.417+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f64db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.417+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f5a6940 con 0x555c9f676fc0 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.417+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f5a4300 con 0x555c9f64c980 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.417+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c9f59fbf0 con 0x555c9f671c20 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f64d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f64d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33272/0 (socket says 172.21.15.99:33272) 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 -- 172.21.15.99:0/1435468265 learned_addr learned my addr 172.21.15.99:0/1435468265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd859000700 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f675a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd859a00700 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f64db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 -- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 msgr2=0x555c9f675a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f675a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 -- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 msgr2=0x555c9f64db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f64db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 -- 172.21.15.99:0/1435468265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c9f3cdb10 con 0x555c9f676fc0 2024-03-29T09:10:50.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.418+0000 7fd853e00700 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f64d5f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd840015680 tx=0x7fd840012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e1c39b6e5c7e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd845600700 1 -- 172.21.15.99:0/1435468265 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840015320 con 0x555c9f676fc0 2024-03-29T09:10:50.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd845600700 1 -- 172.21.15.99:0/1435468265 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840021450 con 0x555c9f676fc0 2024-03-29T09:10:50.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd845600700 1 -- 172.21.15.99:0/1435468265 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400217c0 con 0x555c9f676fc0 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/1435468265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 msgr2=0x555c9f64d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f64d5f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd840015680 tx=0x7fd840012000 comp rx=0 tx=0).stop 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/1435468265 shutdown_connections 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f64db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f675a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/1435468265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f64d5f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/1435468265 >> 172.21.15.99:0/1435468265 conn(0x555c9f5ad650 msgr2=0x555c9f6943e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.419+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/1435468265 shutdown_connections 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.420+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/1435468265 wait complete. 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.420+0000 7fd86f4b2b80 1 -- start start 2024-03-29T09:10:50.424 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.420+0000 7fd86f4b2b80 1 Processor -- start 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.421+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f670fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.421+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.421+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.421+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f53bd30 con 0x555c9f676fc0 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f53b9d0 con 0x555c9f64c980 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c9f423650 con 0x555c9f671c20 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd86f4b2b80 10 client.0 _ll_register_callbacks cb 0x555c9f545420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33288/0 (socket says 172.21.15.99:33288) 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 -- 172.21.15.99:0/2053798915 learned_addr learned my addr 172.21.15.99:0/2053798915 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd853e00700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 -- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 msgr2=0x555c9f67c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.425 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 -- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 msgr2=0x555c9f670fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859000700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f670fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f670fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 -- 172.21.15.99:0/2053798915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84000f040 con 0x555c9f676fc0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859000700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f670fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd859a00700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0115c0 tx=0x7fd84c011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344bb604dc07e563 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.422+0000 7fd83a000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd83a000700 1 -- 172.21.15.99:0/2053798915 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c003c10 con 0x555c9f676fc0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd83a000700 1 -- 172.21.15.99:0/2053798915 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd84c003d70 con 0x555c9f676fc0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c9f560330 con 0x555c9f676fc0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd83a000700 1 -- 172.21.15.99:0/2053798915 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd84c007e80 con 0x555c9f676fc0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 10 client.5601 fetch_fsmap learned FSMap version 16 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x555c9f565e50 con 0x555c9f676fc0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd83a000700 1 -- 172.21.15.99:0/2053798915 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c9f560330 con 0x555c9f676fc0 2024-03-29T09:10:50.426 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd83a000700 1 -- 172.21.15.99:0/2053798915 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd84c0050d0 con 0x555c9f676fc0 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 10 client.5601 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 10 client.5601 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 20 client.5601 populate_metadata read hostname 'smithi099' 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f6789e0 0x555c9f678dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 4 client.5601 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.423+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 --> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c9f5986b0 con 0x555c9f6789e0 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.424+0000 7fd859000700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f6789e0 0x555c9f678dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.424+0000 7fd859000700 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f6789e0 0x555c9f678dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c779d6df04a254 server_cookie=b4b5aef5b50f3baa in_seq=0 out_seq=0 2024-03-29T09:10:50.427 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.424+0000 7fd83a000700 10 client.5601 ms_handle_connect on v2:172.21.15.99:6834/1841077010 2024-03-29T09:10:50.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd83a000700 1 -- 172.21.15.99:0/2053798915 <== mds.0 v2:172.21.15.99:6834/1841077010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555c9f5986b0 con 0x555c9f6789e0 2024-03-29T09:10:50.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd83a000700 10 client.5601 handle_command_reply: tid=0 2024-03-29T09:10:50.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 client.5601 shutdown 2024-03-29T09:10:50.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 msgr2=0x555c9f67cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.428 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0115c0 tx=0x7fd84c011980 comp rx=0 tx=0).stop 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 shutdown_connections 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f6789e0 0x555c9f678dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f670fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd83a000700 0 client.5601 ms_handle_reset on v2:172.21.15.99:6834/1841077010 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2053798915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 >> 172.21.15.99:0/2053798915 conn(0x555c9f5ad650 msgr2=0x555c9f694370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 shutdown_connections 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2053798915 wait complete. 2024-03-29T09:10:50.429 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.425+0000 7fd86f4b2b80 20 client.5601 trim_cache size 0 max 16384 2024-03-29T09:10:50.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.443+0000 7fd86f4b2b80 1 Processor -- start 2024-03-29T09:10:50.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.443+0000 7fd86f4b2b80 1 -- start start 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f64c980 0x555c9f694790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f6461e0 con 0x555c9f64c980 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f64a380 con 0x555c9f676fc0 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c9f650210 con 0x555c9f671c20 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd859000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f64c980 0x555c9f694790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd859000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f64c980 0x555c9f694790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33304/0 (socket says 172.21.15.99:33304) 2024-03-29T09:10:50.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.444+0000 7fd859000700 1 -- 172.21.15.99:0/3346434145 learned_addr learned my addr 172.21.15.99:0/3346434145 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd853e00700 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd859a00700 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd859000700 1 -- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 msgr2=0x555c9f67c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd859000700 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd859000700 1 -- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f676fc0 msgr2=0x555c9f67cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd859000700 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd859000700 1 -- 172.21.15.99:0/3346434145 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c9f425c80 con 0x555c9f64c980 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd859000700 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f64c980 0x555c9f694790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd848059310 tx=0x7fd848088b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4385ebb50c550d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd846000700 1 -- 172.21.15.99:0/3346434145 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848011dc0 con 0x555c9f64c980 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd846000700 1 -- 172.21.15.99:0/3346434145 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848088db0 con 0x555c9f64c980 2024-03-29T09:10:50.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.445+0000 7fd846000700 1 -- 172.21.15.99:0/3346434145 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84805d160 con 0x555c9f64c980 2024-03-29T09:10:50.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/3346434145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f64c980 msgr2=0x555c9f694790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f64c980 0x555c9f694790 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd848059310 tx=0x7fd848088b70 comp rx=0 tx=0).stop 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/3346434145 shutdown_connections 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f676fc0 0x555c9f67cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f64c980 0x555c9f694790 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/3346434145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f67c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/3346434145 >> 172.21.15.99:0/3346434145 conn(0x555c9f5ad650 msgr2=0x555c9f6b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/3346434145 shutdown_connections 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/3346434145 wait complete. 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.446+0000 7fd86f4b2b80 1 -- start start 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.447+0000 7fd86f4b2b80 1 Processor -- start 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.447+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f66ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.447+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f670500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd86f4b2b80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f6773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f64d120 con 0x555c9f676fc0 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c9f696d30 con 0x555c9f64c980 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd86f4b2b80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x555c9f6457f0 con 0x555c9f671c20 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd86f4b2b80 10 client.0 _ll_register_callbacks cb 0x555c9f545420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f6773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f6773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33314/0 (socket says 172.21.15.99:33314) 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 -- 172.21.15.99:0/2202302470 learned_addr learned my addr 172.21.15.99:0/2202302470 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd853e00700 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f670500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859000700 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f66ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 -- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 msgr2=0x555c9f670500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f670500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 -- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 msgr2=0x555c9f66ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f66ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 -- 172.21.15.99:0/2202302470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8541580a0 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd859a00700 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f6773a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0115c0 tx=0x7fd84c0013a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476341c66d5ae6b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.448+0000 7fd83a000700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd83a000700 1 -- 172.21.15.99:0/2202302470 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c005870 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd83a000700 1 -- 172.21.15.99:0/2202302470 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd84c005630 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd840021450 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd83a000700 1 -- 172.21.15.99:0/2202302470 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd84c011d80 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 10 client.5607 fetch_fsmap learned FSMap version 16 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x555c9f3cbfe0 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd83a000700 1 -- 172.21.15.99:0/2202302470 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840021450 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd83a000700 1 -- 172.21.15.99:0/2202302470 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd84c005070 con 0x555c9f676fc0 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 10 client.5607 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 10 client.5607 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 20 client.5607 populate_metadata read hostname 'smithi099' 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f677d60 0x555c9f6fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 4 client.5607 mds_command: new command op to 5433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T09:10:50.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.449+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 --> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555c9f3eeca0 con 0x555c9f677d60 2024-03-29T09:10:50.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd859000700 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f677d60 0x555c9f6fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd83a000700 10 client.5607 ms_handle_connect on v2:172.21.15.99:6834/1841077010 2024-03-29T09:10:50.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd859000700 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f677d60 0x555c9f6fab30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46ce633c1499b3c server_cookie=1cd2dcec3c014115 in_seq=0 out_seq=0 2024-03-29T09:10:50.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd83a000700 1 -- 172.21.15.99:0/2202302470 <== mds.0 v2:172.21.15.99:6834/1841077010 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555c9f3eeca0 con 0x555c9f677d60 2024-03-29T09:10:50.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd83a000700 10 client.5607 handle_command_reply: tid=0 2024-03-29T09:10:50.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 client.5607 shutdown 2024-03-29T09:10:50.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 msgr2=0x555c9f6773a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f6773a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0115c0 tx=0x7fd84c0013a0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 shutdown_connections 2024-03-29T09:10:50.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:6834/1841077010,v1:172.21.15.99:6835/1841077010] conn(0x555c9f677d60 0x555c9f6fab30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c9f676fc0 0x555c9f6773a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555c9f64c980 0x555c9f66ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd83a000700 0 client.5607 ms_handle_reset on v2:172.21.15.99:6834/1841077010 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 --2- 172.21.15.99:0/2202302470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x555c9f671c20 0x555c9f670500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.450+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 >> 172.21.15.99:0/2202302470 conn(0x555c9f5ad650 msgr2=0x555c9f67c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 shutdown_connections 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd86f4b2b80 1 -- 172.21.15.99:0/2202302470 wait complete. 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd86f4b2b80 20 client.5607 trim_cache size 0 max 16384 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "83e3f490-14b9-4a33-8e2d-bbf792b299cc", 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-29T09:10:50.455 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5433, 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 6.5253137109999999, 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 16, 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 39, 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 39, 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 14.833948235999999 2024-03-29T09:10:50.456 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd83406ac90 msgr2=0x7fd83406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd83406ac90 0x7fd83406d150 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd85414c060 tx=0x7fd840007c90 comp rx=0 tx=0).stop 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8540d1b20 msgr2=0x7fd854157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8540d1b20 0x7fd854157620 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd848009b90 tx=0x7fd848004750 comp rx=0 tx=0).stop 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 shutdown_connections 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd83406ac90 0x7fd83406d150 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd854160cc0 0x7fd8540cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8540d1b20 0x7fd854157620 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 --2- 172.21.15.99:0/1580531510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8541608e0 0x7fd854157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.451+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 >> 172.21.15.99:0/1580531510 conn(0x7fd85409c490 msgr2=0x7fd8540b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.452+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 shutdown_connections 2024-03-29T09:10:50.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.452+0000 7fd85ba00700 1 -- 172.21.15.99:0/1580531510 wait complete. 2024-03-29T09:10:50.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:10:50.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.659+0000 7f158d600700 1 Processor -- start 2024-03-29T09:10:50.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.659+0000 7f158d600700 1 -- start start 2024-03-29T09:10:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.660+0000 7f158d600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f15880d1b20 0x7f15880d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.660+0000 7f158d600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880cddc0 con 0x7f15881608e0 2024-03-29T09:10:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.660+0000 7f158d600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880ce0a0 con 0x7f1588160cc0 2024-03-29T09:10:50.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.660+0000 7f158d600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880ce380 con 0x7f15880d1b20 2024-03-29T09:10:50.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.660+0000 7f1586000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f15880d1b20 0x7f15880d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.660+0000 7f1586a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f15881608e0 0x7f15880d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46544/0 (socket says 172.21.15.99:46544) 2024-03-29T09:10:50.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.660+0000 7f1586a00700 1 -- 172.21.15.99:0/3841511098 learned_addr learned my addr 172.21.15.99:0/3841511098 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1586000700 1 -- 172.21.15.99:0/3841511098 >> v1:172.21.15.133:6789/0 conn(0x7f1588160cc0 legacy=0x7f15880afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1586000700 1 -- 172.21.15.99:0/3841511098 >> v1:172.21.15.99:6789/0 conn(0x7f15881608e0 legacy=0x7f15880d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1586000700 1 -- 172.21.15.99:0/3841511098 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880ce660 con 0x7f15880d1b20 2024-03-29T09:10:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1585600700 1 -- 172.21.15.99:0/3841511098 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830443722 0 0) 0x7f15880cddc0 con 0x7f15881608e0 2024-03-29T09:10:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1586000700 1 --2- 172.21.15.99:0/3841511098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f15880d0150 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1574009840 tx=0x7f1574009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820ccb91a3e09d80 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1585600700 1 -- 172.21.15.99:0/3841511098 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574011830 con 0x7f15880d1b20 2024-03-29T09:10:50.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1585600700 1 -- 172.21.15.99:0/3841511098 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1574011990 con 0x7f15880d1b20 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.661+0000 7f1585600700 1 -- 172.21.15.99:0/3841511098 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574011d20 con 0x7f15880d1b20 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.662+0000 7f158d600700 1 -- 172.21.15.99:0/3841511098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 msgr2=0x7f15880d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.662+0000 7f158d600700 1 --2- 172.21.15.99:0/3841511098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f15880d0150 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1574009840 tx=0x7f1574009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.662+0000 7f158d600700 1 -- 172.21.15.99:0/3841511098 shutdown_connections 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.662+0000 7f158d600700 1 --2- 172.21.15.99:0/3841511098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f15880d0150 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.662+0000 7f158d600700 1 -- 172.21.15.99:0/3841511098 >> 172.21.15.99:0/3841511098 conn(0x7f158809c490 msgr2=0x7f15880d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.662+0000 7f158d600700 1 -- 172.21.15.99:0/3841511098 shutdown_connections 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.662+0000 7f158d600700 1 -- 172.21.15.99:0/3841511098 wait complete. 2024-03-29T09:10:50.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.664+0000 7f158d600700 1 Processor -- start 2024-03-29T09:10:50.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.664+0000 7f158d600700 1 -- start start 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f158d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f1588159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f158d600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15881608e0 0x7f158806bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f158d600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1588160cc0 0x7f158815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f158d600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880b3d80 con 0x7f15880d1b20 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f158d600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158815bc30 con 0x7f1588160cc0 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f158d600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158815a390 con 0x7f15881608e0 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f1588159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f1588159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33332/0 (socket says 172.21.15.99:33332) 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 -- 172.21.15.99:0/1226958674 learned_addr learned my addr 172.21.15.99:0/1226958674 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586000700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15881608e0 0x7f158806bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1587400700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1588160cc0 0x7f158815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 -- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15881608e0 msgr2=0x7f158806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15881608e0 0x7f158806bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 -- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1588160cc0 msgr2=0x7f158815c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1588160cc0 0x7f158815c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.665+0000 7f1586a00700 1 -- 172.21.15.99:0/1226958674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740094a0 con 0x7f15880d1b20 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.666+0000 7f1586a00700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f1588159e50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f157800c520 tx=0x7f157800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.666+0000 7f157ea00700 1 -- 172.21.15.99:0/1226958674 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578004c80 con 0x7f15880d1b20 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.666+0000 7f157ea00700 1 -- 172.21.15.99:0/1226958674 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1578004e00 con 0x7f15880d1b20 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.666+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158806c150 con 0x7f15880d1b20 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.666+0000 7f157ea00700 1 -- 172.21.15.99:0/1226958674 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578007310 con 0x7f15880d1b20 2024-03-29T09:10:50.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.666+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158815b300 con 0x7f15880d1b20 2024-03-29T09:10:50.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.667+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158815bf10 con 0x7f15880d1b20 2024-03-29T09:10:50.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.668+0000 7f157ea00700 1 -- 172.21.15.99:0/1226958674 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1578007470 con 0x7f15880d1b20 2024-03-29T09:10:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.669+0000 7f157ea00700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f156806ac90 0x7f156806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.669+0000 7f157ea00700 1 -- 172.21.15.99:0/1226958674 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15780890f0 con 0x7f15880d1b20 2024-03-29T09:10:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.669+0000 7f1586000700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f156806ac90 0x7f156806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:50.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.670+0000 7f1586000700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f156806ac90 0x7f156806d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15740090b0 tx=0x7f157401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:50.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.673+0000 7f157ea00700 1 -- 172.21.15.99:0/1226958674 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f15780540f0 con 0x7f15880d1b20 2024-03-29T09:10:50.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.923+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15880677a0 con 0x7f15880d1b20 2024-03-29T09:10:50.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.924+0000 7f157ea00700 1 -- 172.21.15.99:0/1226958674 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1578057710 con 0x7f15880d1b20 2024-03-29T09:10:50.924 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T09:10:50.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f156806ac90 msgr2=0x7f156806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f156806ac90 0x7f156806d150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f15740090b0 tx=0x7f157401c040 comp rx=0 tx=0).stop 2024-03-29T09:10:50.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 msgr2=0x7f1588159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:50.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f1588159e50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f157800c520 tx=0x7f157800c830 comp rx=0 tx=0).stop 2024-03-29T09:10:50.926 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 shutdown_connections 2024-03-29T09:10:50.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f156806ac90 0x7f156806d150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1588160cc0 0x7f158815c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15880d1b20 0x7f1588159e50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 --2- 172.21.15.99:0/1226958674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15881608e0 0x7f158806bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:50.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 >> 172.21.15.99:0/1226958674 conn(0x7f158809c490 msgr2=0x7f15880cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:50.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 shutdown_connections 2024-03-29T09:10:50.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:50.926+0000 7f158d600700 1 -- 172.21.15.99:0/1226958674 wait complete. 2024-03-29T09:10:50.945 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:10:50.945 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:10:50.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:10:50.945 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T09:10:50.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:10:50.946 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:10:50.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:10:50.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:10:50.946 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:10:50.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:50.946 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:10:50.989 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:10:50.989 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:10:50.989 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:10:50.989 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:10:50.990 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42137sec preferred_lft 42137sec 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::3caa:acff:feac:fccd/64 scope link 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:10:50.991 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5832:eeff:fe0b:ee92/64 scope link 2024-03-29T09:10:50.992 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:10:50.992 INFO:teuthology.orchestra.run.smithi099.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:10:50.992 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether fe:d0:b6:f6:cb:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:10:50.992 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::fcd0:b6ff:fef6:cba2/64 scope link 2024-03-29T09:10:50.992 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:10:50.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:50.992 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:10:51.092 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:10:51.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:51.095 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:10:51.139 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:10:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:10:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:10:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:10:51.140 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:10:51.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:51.141 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:10:51.141 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:10:51.141 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:10:51.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:10:51.142 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:10:51.142 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:10:51.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:51.342 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:10:51.383 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:10:51.384 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T09:10:51.385 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:10:51.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:51.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:10:51.490 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T09:10:51.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:10:51.491 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T09:10:51.532 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:10:51.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:10:51.532 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:10:51.614 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:10:51.721 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:10:51.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:10:51.784 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:10:51.786 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:10:51.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:51.787 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:10:51.828 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:10:51.828 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T09:10:51.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:10:51.937 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:10:51.937 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:10:52.004 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:10:52.007 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:10:52.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:52.007 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:10:52.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-29T09:10:52.042+0000 7f9f8f8565c0 -1 init, newargv = 0x556aa04e0090 newargc=15 2024-03-29T09:10:52.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[60135]: starting ceph client 2024-03-29T09:10:52.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[60135]: starting fuse 2024-03-29T09:10:52.054 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:10:52.055 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T09:10:52.055 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:10:52.056 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:10:52.057 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-29T09:10:52.057 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:10:52.232 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.60135.asok 2024-03-29T09:10:52.411 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60135.asok 2024-03-29T09:10:52.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:52.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60135.asok status 2024-03-29T09:10:52.629 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:10:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:10:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:10:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:10:52.629 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "60135", 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5625, 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:10:52.630 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5625 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1915681316 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:10:52.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1915681316 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5625 192.168.0.1:0/1915681316", 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/1915681316", 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 16, 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 39, 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:10:52.632 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:10:52.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:52.645 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:10:52.645 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T09:10:52.688 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:10:52.688 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T09:10:52.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:52.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:10:52.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-03-29T09:10:52.903 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:10:52.908 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:10:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.993+0000 7f1525000700 1 Processor -- start 2024-03-29T09:10:52.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.994+0000 7f1525000700 1 -- start start 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f1525000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1520160b30 0x7f15200cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f1525000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f15200b1f50 0x7f15200d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f1525000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15200b2b90 0x7f15200b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f1525000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200cf440 con 0x7f1520160b30 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f1525000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200d1fa0 con 0x7f15200b1f50 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f1525000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1520160f10 con 0x7f15200b2b90 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f151de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1520160b30 0x7f15200cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f151de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1520160b30 0x7f15200cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33348/0 (socket says 172.21.15.99:33348) 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f151de00700 1 -- 172.21.15.99:0/4005334799 learned_addr learned my addr 172.21.15.99:0/4005334799 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:52.996 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f151e800700 1 --2- 172.21.15.99:0/4005334799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15200b2b90 0x7f15200b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f151d400700 1 --2- 172.21.15.99:0/4005334799 >> v2:172.21.15.133:3300/0 conn(0x7f15200b1f50 0x7f15200d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.995+0000 7f151de00700 1 -- 172.21.15.99:0/4005334799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15200b2b90 msgr2=0x7f15200b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151de00700 1 --2- 172.21.15.99:0/4005334799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15200b2b90 0x7f15200b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151de00700 1 -- 172.21.15.99:0/4005334799 >> v2:172.21.15.133:3300/0 conn(0x7f15200b1f50 msgr2=0x7f15200d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151de00700 1 --2- 172.21.15.99:0/4005334799 >> v2:172.21.15.133:3300/0 conn(0x7f15200b1f50 0x7f15200d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151de00700 1 -- 172.21.15.99:0/4005334799 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200d1670 con 0x7f1520160b30 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151d400700 1 --2- 172.21.15.99:0/4005334799 >> v2:172.21.15.133:3300/0 conn(0x7f15200b1f50 0x7f15200d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151de00700 1 --2- 172.21.15.99:0/4005334799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1520160b30 0x7f15200cfb50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f15200d42a0 tx=0x7f1510009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ddac61754a64e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151ca00700 1 -- 172.21.15.99:0/4005334799 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151000b870 con 0x7f1520160b30 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151ca00700 1 -- 172.21.15.99:0/4005334799 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151000b9d0 con 0x7f1520160b30 2024-03-29T09:10:52.997 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.996+0000 7f151ca00700 1 -- 172.21.15.99:0/4005334799 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151000bd60 con 0x7f1520160b30 2024-03-29T09:10:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 -- 172.21.15.99:0/4005334799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1520160b30 msgr2=0x7f15200cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 --2- 172.21.15.99:0/4005334799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1520160b30 0x7f15200cfb50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f15200d42a0 tx=0x7f1510009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:52.998 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 -- 172.21.15.99:0/4005334799 shutdown_connections 2024-03-29T09:10:52.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 --2- 172.21.15.99:0/4005334799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f15200b2b90 0x7f15200b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:52.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 --2- 172.21.15.99:0/4005334799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1520160b30 0x7f15200cfb50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:52.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 --2- 172.21.15.99:0/4005334799 >> v2:172.21.15.133:3300/0 conn(0x7f15200b1f50 0x7f15200d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:52.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 -- 172.21.15.99:0/4005334799 >> 172.21.15.99:0/4005334799 conn(0x7f152009c4d0 msgr2=0x7f15200d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:52.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 -- 172.21.15.99:0/4005334799 shutdown_connections 2024-03-29T09:10:52.999 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.997+0000 7f1525000700 1 -- 172.21.15.99:0/4005334799 wait complete. 2024-03-29T09:10:53.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.999+0000 7f1525000700 1 Processor -- start 2024-03-29T09:10:53.001 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:52.999+0000 7f1525000700 1 -- start start 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f1525000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15200b1f50 0x7f15200ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f1525000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15200b2b90 0x7f152015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f1525000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1520160b30 0x7f15200cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f1525000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200cb1d0 con 0x7f15200b1f50 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f1525000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200ca270 con 0x7f15200b2b90 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f1525000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15200ca550 con 0x7f1520160b30 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f151de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15200b1f50 0x7f15200ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f151de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15200b1f50 0x7f15200ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33364/0 (socket says 172.21.15.99:33364) 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f151de00700 1 -- 172.21.15.99:0/95961221 learned_addr learned my addr 172.21.15.99:0/95961221 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f151e800700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1520160b30 0x7f15200cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.000+0000 7f151d400700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15200b2b90 0x7f152015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:53.002 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f151de00700 1 -- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1520160b30 msgr2=0x7f15200cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f151de00700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1520160b30 0x7f15200cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f151de00700 1 -- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15200b2b90 msgr2=0x7f152015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f151de00700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15200b2b90 0x7f152015bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f151de00700 1 -- 172.21.15.99:0/95961221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510009550 con 0x7f15200b1f50 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f151de00700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15200b1f50 0x7f15200ccf60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1510000c00 tx=0x7f1510004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f1516000700 1 -- 172.21.15.99:0/95961221 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510004310 con 0x7f15200b1f50 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f1516000700 1 -- 172.21.15.99:0/95961221 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151001e070 con 0x7f15200b1f50 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200ca830 con 0x7f15200b1f50 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f1516000700 1 -- 172.21.15.99:0/95961221 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15100036a0 con 0x7f15200b1f50 2024-03-29T09:10:53.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.001+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15200cadf0 con 0x7f15200b1f50 2024-03-29T09:10:53.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.002+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15200cb310 con 0x7f15200b1f50 2024-03-29T09:10:53.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.003+0000 7f1516000700 1 -- 172.21.15.99:0/95961221 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1510020030 con 0x7f15200b1f50 2024-03-29T09:10:53.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.004+0000 7f1516000700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f150006ac90 0x7f150006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:53.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.005+0000 7f1516000700 1 -- 172.21.15.99:0/95961221 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f151008a360 con 0x7f15200b1f50 2024-03-29T09:10:53.004 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.005+0000 7f151d400700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f150006ac90 0x7f150006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:53.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.005+0000 7f151d400700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f150006ac90 0x7f150006d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f150c006ba0 tx=0x7f150c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:53.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.009+0000 7f1516000700 1 -- 172.21.15.99:0/95961221 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f15200cb310 con 0x7f15200b1f50 2024-03-29T09:10:53.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.216+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f15200678b0 con 0x7f15200b1f50 2024-03-29T09:10:53.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.956+0000 7f1516000700 1 -- 172.21.15.99:0/95961221 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v40) v1 ==== 124+0+0 (secure 0 0 0) 0x7f15100553f0 con 0x7f15200b1f50 2024-03-29T09:10:53.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.962+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f150006ac90 msgr2=0x7f150006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:53.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.962+0000 7f1525000700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f150006ac90 0x7f150006d150 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f150c006ba0 tx=0x7f150c008040 comp rx=0 tx=0).stop 2024-03-29T09:10:53.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.962+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15200b1f50 msgr2=0x7f15200ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:53.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.962+0000 7f1525000700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15200b1f50 0x7f15200ccf60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1510000c00 tx=0x7f1510004750 comp rx=0 tx=0).stop 2024-03-29T09:10:53.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.963+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 shutdown_connections 2024-03-29T09:10:53.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.963+0000 7f1525000700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f150006ac90 0x7f150006d150 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f150c006ba0 tx=0x7f150c008040 comp rx=0 tx=0).stop 2024-03-29T09:10:53.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.963+0000 7f1525000700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1520160b30 0x7f15200cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:53.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.963+0000 7f1525000700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f15200b1f50 0x7f15200ccf60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:53.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.963+0000 7f1525000700 1 --2- 172.21.15.99:0/95961221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f15200b2b90 0x7f152015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:53.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.963+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 >> 172.21.15.99:0/95961221 conn(0x7f152009c4d0 msgr2=0x7f152006cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:53.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.964+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 shutdown_connections 2024-03-29T09:10:53.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:53.964+0000 7f1525000700 1 -- 172.21.15.99:0/95961221 wait complete. 2024-03-29T09:10:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'data_new' created 2024-03-29T09:10:53.989 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-03-29T09:10:54.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.226+0000 7f5f5c600700 1 Processor -- start 2024-03-29T09:10:54.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.226+0000 7f5f5c600700 1 -- start start 2024-03-29T09:10:54.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.227+0000 7f5f5c600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 0x7f5f540d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.227+0000 7f5f5c600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5f540d3380 0x7f5f540ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.227 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.227+0000 7f5f5c600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f540d3810 con 0x7f5f54160770 2024-03-29T09:10:54.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.227+0000 7f5f5c600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f540b4140 con 0x7f5f540d3380 2024-03-29T09:10:54.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.227+0000 7f5f5c600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540d1520 con 0x7f5f540b3cc0 2024-03-29T09:10:54.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.227+0000 7f5f59c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5f54160770 0x7f5f540d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46548/0 (socket says 172.21.15.99:46548) 2024-03-29T09:10:54.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.227+0000 7f5f59c00700 1 -- 172.21.15.99:0/3600156665 learned_addr learned my addr 172.21.15.99:0/3600156665 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:54.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f5a600700 1 --2- 172.21.15.99:0/3600156665 >> v2:172.21.15.133:3300/0 conn(0x7f5f540d3380 0x7f5f540ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f59200700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 0x7f5f540d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207221948 0 0) 0x7f5f540d3810 con 0x7f5f54160770 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f3c0035b0 con 0x7f5f54160770 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089109602 0 0) 0x7f5f3c0035b0 con 0x7f5f54160770 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 msgr2=0x7f5f540d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 0x7f5f540d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 >> v2:172.21.15.133:3300/0 conn(0x7f5f540d3380 msgr2=0x7f5f540ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 --2- 172.21.15.99:0/3600156665 >> v2:172.21.15.133:3300/0 conn(0x7f5f540d3380 0x7f5f540ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f540d1800 con 0x7f5f54160770 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f5f48001670 con 0x7f5f54160770 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f59200700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 0x7f5f540d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:54.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f5a600700 1 --2- 172.21.15.99:0/3600156665 >> v2:172.21.15.133:3300/0 conn(0x7f5f540d3380 0x7f5f540ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 >> v1:172.21.15.99:6789/0 conn(0x7f5f54160770 legacy=0x7f5f540d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c003a00 0x7f5f3c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 0x7f5f3c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540d1520 con 0x7f5f3c004d90 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c00ba10 con 0x7f5f3c003a00 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.228+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c00bb50 con 0x7f5f540b3cc0 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f5a600700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f59200700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 0x7f5f3c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f59c00700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c003a00 0x7f5f3c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f5a600700 1 -- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 msgr2=0x7f5f3c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f5a600700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f540b3cc0 0x7f5f3c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f5a600700 1 -- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c003a00 msgr2=0x7f5f3c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:54.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f5a600700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f3c003a00 0x7f5f3c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f5a600700 1 -- 172.21.15.99:0/3600156665 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f400094a0 con 0x7f5f3c004d90 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.229+0000 7f5f5a600700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00e350 tx=0x7f5f3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff22058628c6e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00edf0 con 0x7f5f3c004d90 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00f040 con 0x7f5f3c004d90 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f53e00700 1 -- 172.21.15.99:0/3600156665 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c0146a0 con 0x7f5f3c004d90 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f5c600700 1 -- 172.21.15.99:0/3600156665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f3c004d90 msgr2=0x7f5f3c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f5c600700 1 --2- 172.21.15.99:0/3600156665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f3c004d90 0x7f5f3c005210 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00e350 tx=0x7f5f3c005880 comp rx=0 tx=0).stop 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f5a600700 1 -- 172.21.15.99:0/3600156665 reap_dead start 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f5c600700 1 -- 172.21.15.99:0/3600156665 shutdown_connections 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f5c600700 1 -- 172.21.15.99:0/3600156665 >> 172.21.15.99:0/3600156665 conn(0x7f5f5408afd0 msgr2=0x7f5f5409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f5c600700 1 -- 172.21.15.99:0/3600156665 shutdown_connections 2024-03-29T09:10:54.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.230+0000 7f5f5c600700 1 -- 172.21.15.99:0/3600156665 wait complete. 2024-03-29T09:10:54.235 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.232+0000 7f5f5c600700 1 Processor -- start 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.232+0000 7f5f5c600700 1 -- start start 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f5c600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f3c004d90 0x7f5f5414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f5c600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f540ce910 0x7f5f5409fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f5c600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f540ced90 0x7f5f5409ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f5c600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5414be60 con 0x7f5f540ce910 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f5c600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f541f71e0 con 0x7f5f540ced90 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f5c600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f541f6f00 con 0x7f5f3c004d90 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f59200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f540ce910 0x7f5f5409fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f59200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f540ce910 0x7f5f5409fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33386/0 (socket says 172.21.15.99:33386) 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f59200700 1 -- 172.21.15.99:0/641476264 learned_addr learned my addr 172.21.15.99:0/641476264 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f59c00700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f3c004d90 0x7f5f5414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.236 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.233+0000 7f5f5a600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f540ced90 0x7f5f5409ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f59200700 1 -- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f3c004d90 msgr2=0x7f5f5414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f59200700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f3c004d90 0x7f5f5414eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f59200700 1 -- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f540ced90 msgr2=0x7f5f5409ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f59200700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f540ced90 0x7f5f5409ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f59200700 1 -- 172.21.15.99:0/641476264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5414e2c0 con 0x7f5f540ce910 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f59200700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f540ce910 0x7f5f5409fa30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f40011900 tx=0x7f5f40011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f51600700 1 -- 172.21.15.99:0/641476264 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000fae0 con 0x7f5f540ce910 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f51600700 1 -- 172.21.15.99:0/641476264 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4000fc40 con 0x7f5f540ce910 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f51600700 1 -- 172.21.15.99:0/641476264 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40011ea0 con 0x7f5f540ce910 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5414e5a0 con 0x7f5f540ce910 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.234+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f541f6280 con 0x7f5f540ce910 2024-03-29T09:10:54.237 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.235+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5414ebf0 con 0x7f5f540ce910 2024-03-29T09:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.236+0000 7f5f51600700 1 -- 172.21.15.99:0/641476264 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5f40015460 con 0x7f5f540ce910 2024-03-29T09:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.237+0000 7f5f51600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5f3406ac90 0x7f5f3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.237+0000 7f5f51600700 1 -- 172.21.15.99:0/641476264 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6392+0+0 (secure 0 0 0) 0x7f5f40089b10 con 0x7f5f540ce910 2024-03-29T09:10:54.238 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.238+0000 7f5f59c00700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5f3406ac90 0x7f5f3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.239 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.238+0000 7f5f59c00700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5f3406ac90 0x7f5f3406d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5f48001050 tx=0x7f5f48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:54.242 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.243+0000 7f5f51600700 1 -- 172.21.15.99:0/641476264 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5f40054af0 con 0x7f5f540ce910 2024-03-29T09:10:54.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.491+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7f5f540d0ab0 con 0x7f5f540ce910 2024-03-29T09:10:54.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.498+0000 7f5f51600700 1 -- 172.21.15.99:0/641476264 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 7 to fsmap v17) v1 ==== 131+0+0 (secure 0 0 0) 0x7f5f400220b0 con 0x7f5f540ce910 2024-03-29T09:10:54.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5f3406ac90 msgr2=0x7f5f3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5f3406ac90 0x7f5f3406d150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5f48001050 tx=0x7f5f48009040 comp rx=0 tx=0).stop 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f540ce910 msgr2=0x7f5f5409fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f540ce910 0x7f5f5409fa30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f40011900 tx=0x7f5f40011cc0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 shutdown_connections 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5f3406ac90 0x7f5f3406d150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f540ced90 0x7f5f5409ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f3c004d90 0x7f5f5414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 --2- 172.21.15.99:0/641476264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5f540ce910 0x7f5f5409fa30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.502+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 >> 172.21.15.99:0/641476264 conn(0x7f5f5408afd0 msgr2=0x7f5f5415f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:54.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.503+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 shutdown_connections 2024-03-29T09:10:54.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.504+0000 7f5f5c600700 1 -- 172.21.15.99:0/641476264 wait complete. 2024-03-29T09:10:54.506 INFO:teuthology.orchestra.run.smithi099.stderr:added data pool 7 to fsmap 2024-03-29T09:10:54.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:10:54.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.716+0000 7f09e8a00700 1 Processor -- start 2024-03-29T09:10:54.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.716+0000 7f09e8a00700 1 -- start start 2024-03-29T09:10:54.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.717+0000 7f09e8a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e4070540 0x7f09e415d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.717+0000 7f09e8a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e40ccf60 con 0x7f09e40709a0 2024-03-29T09:10:54.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.717+0000 7f09e8a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e4071750 con 0x7f09e4153d20 2024-03-29T09:10:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.717+0000 7f09e8a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e40b12d0 con 0x7f09e4070540 2024-03-29T09:10:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.717+0000 7f09e2800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f09e40709a0 0x7f09e41646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46558/0 (socket says 172.21.15.99:46558) 2024-03-29T09:10:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e2800700 1 -- 172.21.15.99:0/601120990 learned_addr learned my addr 172.21.15.99:0/601120990 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:54.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e1400700 1 --2- 172.21.15.99:0/601120990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e4070540 0x7f09e415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e0a00700 1 -- 172.21.15.99:0/601120990 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357256565 0 0) 0x7f09e40ccf60 con 0x7f09e40709a0 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e0a00700 1 -- 172.21.15.99:0/601120990 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c80035b0 con 0x7f09e40709a0 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e0a00700 1 -- 172.21.15.99:0/601120990 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769135988 0 0) 0x7f09e4071750 con 0x7f09e4153d20 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e0a00700 1 -- 172.21.15.99:0/601120990 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e40ccf60 con 0x7f09e4153d20 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e1400700 1 -- 172.21.15.99:0/601120990 >> v1:172.21.15.133:6789/0 conn(0x7f09e4153d20 legacy=0x7f09e40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e1400700 1 -- 172.21.15.99:0/601120990 >> v1:172.21.15.99:6789/0 conn(0x7f09e40709a0 legacy=0x7f09e41646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e1400700 1 -- 172.21.15.99:0/601120990 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4070eb0 con 0x7f09e4070540 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.718+0000 7f09e1400700 1 --2- 172.21.15.99:0/601120990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e4070540 0x7f09e415d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009870 tx=0x7f09d0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1e5207aebddbc58 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e0a00700 1 -- 172.21.15.99:0/601120990 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0011830 con 0x7f09e4070540 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e0a00700 1 -- 172.21.15.99:0/601120990 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0011990 con 0x7f09e4070540 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e0a00700 1 -- 172.21.15.99:0/601120990 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0011d20 con 0x7f09e4070540 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e8a00700 1 -- 172.21.15.99:0/601120990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e4070540 msgr2=0x7f09e415d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e8a00700 1 --2- 172.21.15.99:0/601120990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e4070540 0x7f09e415d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009870 tx=0x7f09d0009b80 comp rx=0 tx=0).stop 2024-03-29T09:10:54.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e8a00700 1 -- 172.21.15.99:0/601120990 shutdown_connections 2024-03-29T09:10:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e8a00700 1 --2- 172.21.15.99:0/601120990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e4070540 0x7f09e415d330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e8a00700 1 -- 172.21.15.99:0/601120990 >> 172.21.15.99:0/601120990 conn(0x7f09e4066540 msgr2=0x7f09e4066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e8a00700 1 -- 172.21.15.99:0/601120990 shutdown_connections 2024-03-29T09:10:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.719+0000 7f09e8a00700 1 -- 172.21.15.99:0/601120990 wait complete. 2024-03-29T09:10:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.720+0000 7f09e8a00700 1 Processor -- start 2024-03-29T09:10:54.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.720+0000 7f09e8a00700 1 -- start start 2024-03-29T09:10:54.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.722+0000 7f09e8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09e4070540 0x7f09e414e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.722+0000 7f09e8a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e40709a0 0x7f09e414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.722+0000 7f09e8a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e4153d20 0x7f09e4150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.722+0000 7f09e8a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e4155ee0 con 0x7f09e4070540 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.722+0000 7f09e8a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e40b2e50 con 0x7f09e4153d20 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.722+0000 7f09e8a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e40b3130 con 0x7f09e40709a0 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.722+0000 7f09e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09e4070540 0x7f09e414e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09e4070540 0x7f09e414e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33390/0 (socket says 172.21.15.99:33390) 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 -- 172.21.15.99:0/1082929103 learned_addr learned my addr 172.21.15.99:0/1082929103 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1400700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e40709a0 0x7f09e414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e2800700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e4153d20 0x7f09e4150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 -- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e40709a0 msgr2=0x7f09e414fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e40709a0 0x7f09e414fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 -- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e4153d20 msgr2=0x7f09e4150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e4153d20 0x7f09e4150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 -- 172.21.15.99:0/1082929103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00094a0 con 0x7f09e4070540 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09e1e00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09e4070540 0x7f09e414e760 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f09d400c500 tx=0x7f09d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09da000700 1 -- 172.21.15.99:0/1082929103 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d40052d0 con 0x7f09e4070540 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09da000700 1 -- 172.21.15.99:0/1082929103 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d4005450 con 0x7f09e4070540 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.723+0000 7f09da000700 1 -- 172.21.15.99:0/1082929103 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d40038c0 con 0x7f09e4070540 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.724+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e4155b70 con 0x7f09e4070540 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.724+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e41557b0 con 0x7f09e4070540 2024-03-29T09:10:54.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.725+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e41559a0 con 0x7f09e4070540 2024-03-29T09:10:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.725+0000 7f09da000700 1 -- 172.21.15.99:0/1082929103 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f09d4003a20 con 0x7f09e4070540 2024-03-29T09:10:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.726+0000 7f09da000700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09c406acc0 0x7f09c406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.726+0000 7f09da000700 1 -- 172.21.15.99:0/1082929103 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f09d4088a80 con 0x7f09e4070540 2024-03-29T09:10:54.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.727+0000 7f09e1400700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09c406acc0 0x7f09c406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:54.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.727+0000 7f09e1400700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09c406acc0 0x7f09c406d180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f09d00097c0 tx=0x7f09d001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:54.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.730+0000 7f09da000700 1 -- 172.21.15.99:0/1082929103 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f09d4053990 con 0x7f09e4070540 2024-03-29T09:10:54.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.978+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09e40677a0 con 0x7f09e4070540 2024-03-29T09:10:54.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.979+0000 7f09da000700 1 -- 172.21.15.99:0/1082929103 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f09d4017090 con 0x7f09e4070540 2024-03-29T09:10:54.978 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:54.978 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:10:43.923687+0000","modified":"2024-03-29T09:10:54.493701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/1841077010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1841077010},{"type":"v1","addr":"172.21.15.99:6835","nonce":1841077010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09c406acc0 msgr2=0x7f09c406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09c406acc0 0x7f09c406d180 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f09d00097c0 tx=0x7f09d001c040 comp rx=0 tx=0).stop 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09e4070540 msgr2=0x7f09e414e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09e4070540 0x7f09e414e760 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f09d400c500 tx=0x7f09d400c8c0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 shutdown_connections 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09c406acc0 0x7f09c406d180 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09e4153d20 0x7f09e4150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09e4070540 0x7f09e414e760 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.981+0000 7f09e8a00700 1 --2- 172.21.15.99:0/1082929103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09e40709a0 0x7f09e414fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.982+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 >> 172.21.15.99:0/1082929103 conn(0x7f09e4066540 msgr2=0x7f09e4152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:54.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.982+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 shutdown_connections 2024-03-29T09:10:54.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:54.982+0000 7f09e8a00700 1 -- 172.21.15.99:0/1082929103 wait complete. 2024-03-29T09:10:54.982 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2024-03-29T09:10:54.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:55.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.190+0000 7ff777e00700 1 Processor -- start 2024-03-29T09:10:55.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.190+0000 7ff777e00700 1 -- start start 2024-03-29T09:10:55.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.191+0000 7ff777e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff770143940 0x7ff770070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.191+0000 7ff777e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff770070ca0 0x7ff77015db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.191+0000 7ff777e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff77015e040 0x7ff7700ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.191+0000 7ff777e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff770143d20 con 0x7ff77015e040 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.191+0000 7ff777e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77015eaf0 con 0x7ff770143940 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.191+0000 7ff777e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700cd820 con 0x7ff770070ca0 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff77015e040 0x7ff7700ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff77015e040 0x7ff7700ce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33402/0 (socket says 172.21.15.99:33402) 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 -- 172.21.15.99:0/2576463508 learned_addr learned my addr 172.21.15.99:0/2576463508 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775400700 1 --2- 172.21.15.99:0/2576463508 >> v2:172.21.15.133:3300/0 conn(0x7ff770143940 0x7ff770070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 -- 172.21.15.99:0/2576463508 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff770070ca0 msgr2=0x7ff77015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 --2- 172.21.15.99:0/2576463508 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff770070ca0 0x7ff77015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 -- 172.21.15.99:0/2576463508 >> v2:172.21.15.133:3300/0 conn(0x7ff770143940 msgr2=0x7ff770070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 --2- 172.21.15.99:0/2576463508 >> v2:172.21.15.133:3300/0 conn(0x7ff770143940 0x7ff770070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 -- 172.21.15.99:0/2576463508 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77015e530 con 0x7ff77015e040 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.192+0000 7ff775e00700 1 --2- 172.21.15.99:0/2576463508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77015e040 0x7ff7700ce2c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff77006c160 tx=0x7ff76800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c142eebf1d8816f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff76fe00700 1 -- 172.21.15.99:0/2576463508 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800e070 con 0x7ff77015e040 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff76fe00700 1 -- 172.21.15.99:0/2576463508 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76800b9c0 con 0x7ff77015e040 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff76fe00700 1 -- 172.21.15.99:0/2576463508 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800bd70 con 0x7ff77015e040 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 -- 172.21.15.99:0/2576463508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77015e040 msgr2=0x7ff7700ce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 --2- 172.21.15.99:0/2576463508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77015e040 0x7ff7700ce2c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff77006c160 tx=0x7ff76800b330 comp rx=0 tx=0).stop 2024-03-29T09:10:55.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 -- 172.21.15.99:0/2576463508 shutdown_connections 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 --2- 172.21.15.99:0/2576463508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff77015e040 0x7ff7700ce2c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 --2- 172.21.15.99:0/2576463508 >> v2:172.21.15.133:3300/0 conn(0x7ff770143940 0x7ff770070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 --2- 172.21.15.99:0/2576463508 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff770070ca0 0x7ff77015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 -- 172.21.15.99:0/2576463508 >> 172.21.15.99:0/2576463508 conn(0x7ff770066540 msgr2=0x7ff770066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 -- 172.21.15.99:0/2576463508 shutdown_connections 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.193+0000 7ff777e00700 1 -- 172.21.15.99:0/2576463508 wait complete. 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.195+0000 7ff777e00700 1 Processor -- start 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.195+0000 7ff777e00700 1 -- start start 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff777e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff770070ca0 0x7ff77013e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff777e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff770143940 0x7ff7701f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff777e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff77015e040 0x7ff7701f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff777e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7701f6e10 con 0x7ff770143940 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff777e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7701f3830 con 0x7ff770070ca0 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff777e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7701f2f90 con 0x7ff77015e040 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff774a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff770143940 0x7ff7701f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.196+0000 7ff774a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff770143940 0x7ff7701f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33416/0 (socket says 172.21.15.99:33416) 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff774a00700 1 -- 172.21.15.99:0/1614920022 learned_addr learned my addr 172.21.15.99:0/1614920022 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff775400700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff770070ca0 0x7ff77013e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff775e00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff77015e040 0x7ff7701f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff774a00700 1 -- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff77015e040 msgr2=0x7ff7701f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff774a00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff77015e040 0x7ff7701f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff774a00700 1 -- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff770070ca0 msgr2=0x7ff77013e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff774a00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff770070ca0 0x7ff77013e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff774a00700 1 -- 172.21.15.99:0/1614920022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768009b30 con 0x7ff770143940 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff774a00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff770143940 0x7ff7701f2880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00b500 tx=0x7ff75c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.197+0000 7ff76d600700 1 -- 172.21.15.99:0/1614920022 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c011830 con 0x7ff770143940 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.198+0000 7ff76d600700 1 -- 172.21.15.99:0/1614920022 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c011990 con 0x7ff770143940 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.198+0000 7ff76d600700 1 -- 172.21.15.99:0/1614920022 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c011d40 con 0x7ff770143940 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.198+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7701f3270 con 0x7ff770143940 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.198+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77013f5d0 con 0x7ff770143940 2024-03-29T09:10:55.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.199+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7700cd1d0 con 0x7ff770143940 2024-03-29T09:10:55.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.200+0000 7ff76d600700 1 -- 172.21.15.99:0/1614920022 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff75c01a400 con 0x7ff770143940 2024-03-29T09:10:55.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.201+0000 7ff76d600700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff75006ac90 0x7ff75006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.201+0000 7ff76d600700 1 -- 172.21.15.99:0/1614920022 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6424+0+0 (secure 0 0 0) 0x7ff75c089470 con 0x7ff770143940 2024-03-29T09:10:55.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.201+0000 7ff775400700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff75006ac90 0x7ff75006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.201+0000 7ff775400700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff75006ac90 0x7ff75006d150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff764006b20 tx=0x7ff764008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:55.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.205+0000 7ff76d600700 1 -- 172.21.15.99:0/1614920022 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff75c054410 con 0x7ff770143940 2024-03-29T09:10:55.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.408+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7700677a0 con 0x7ff770143940 2024-03-29T09:10:55.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.409+0000 7ff76d600700 1 -- 172.21.15.99:0/1614920022 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18094 (secure 0 0 0) 0x7ff75c057a30 con 0x7ff770143940 2024-03-29T09:10:55.409 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:55.409 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":41,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:54.493431+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff75006ac90 msgr2=0x7ff75006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff75006ac90 0x7ff75006d150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff764006b20 tx=0x7ff764008040 comp rx=0 tx=0).stop 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff770143940 msgr2=0x7ff7701f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff770143940 0x7ff7701f2880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00b500 tx=0x7ff75c00b810 comp rx=0 tx=0).stop 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 shutdown_connections 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff75006ac90 0x7ff75006d150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff77015e040 0x7ff7701f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff770070ca0 0x7ff77013e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 --2- 172.21.15.99:0/1614920022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff770143940 0x7ff7701f2880 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 >> 172.21.15.99:0/1614920022 conn(0x7ff770066540 msgr2=0x7ff7700b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:55.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 shutdown_connections 2024-03-29T09:10:55.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.412+0000 7ff777e00700 1 -- 172.21.15.99:0/1614920022 wait complete. 2024-03-29T09:10:55.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:55.429 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2024-03-29T09:10:55.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:10:55.479 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2024-03-29T09:10:55.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-03-29T09:10:55.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.726+0000 7f841bc00700 1 Processor -- start 2024-03-29T09:10:55.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.726+0000 7f841bc00700 1 -- start start 2024-03-29T09:10:55.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.727+0000 7f841bc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8414160770 0x7f84140b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.727+0000 7f841bc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84140d0d60 0x7f84140d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.727+0000 7f841bc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f84140d3380 0x7f84140cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.727+0000 7f841bc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84140d3810 con 0x7f8414160770 2024-03-29T09:10:55.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.727+0000 7f841bc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84140d1520 con 0x7f84140d3380 2024-03-29T09:10:55.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.727+0000 7f841bc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84140d1800 con 0x7f84140d0d60 2024-03-29T09:10:55.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8414160770 0x7f84140b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8414160770 0x7f84140b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33418/0 (socket says 172.21.15.99:33418) 2024-03-29T09:10:55.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 -- 172.21.15.99:0/2245476420 learned_addr learned my addr 172.21.15.99:0/2245476420 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:55.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8413e00700 1 --2- 172.21.15.99:0/2245476420 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84140d0d60 0x7f84140d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 -- 172.21.15.99:0/2245476420 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84140d0d60 msgr2=0x7f84140d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:55.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 --2- 172.21.15.99:0/2245476420 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84140d0d60 0x7f84140d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 -- 172.21.15.99:0/2245476420 >> v2:172.21.15.133:3300/0 conn(0x7f84140d3380 msgr2=0x7f84140cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:55.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 --2- 172.21.15.99:0/2245476420 >> v2:172.21.15.133:3300/0 conn(0x7f84140d3380 0x7f84140cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 -- 172.21.15.99:0/2245476420 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140d1ae0 con 0x7f8414160770 2024-03-29T09:10:55.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8413e00700 1 --2- 172.21.15.99:0/2245476420 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84140d0d60 0x7f84140d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:55.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.728+0000 7f8419200700 1 --2- 172.21.15.99:0/2245476420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8414160770 0x7f84140b3cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84140cd7e0 tx=0x7f8408009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614778ae316b9853 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:55.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f8413400700 1 -- 172.21.15.99:0/2245476420 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800b870 con 0x7f8414160770 2024-03-29T09:10:55.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f8413400700 1 -- 172.21.15.99:0/2245476420 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800b9d0 con 0x7f8414160770 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f8413400700 1 -- 172.21.15.99:0/2245476420 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800bd60 con 0x7f8414160770 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 -- 172.21.15.99:0/2245476420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8414160770 msgr2=0x7f84140b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 --2- 172.21.15.99:0/2245476420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8414160770 0x7f84140b3cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f84140cd7e0 tx=0x7f8408009bd0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 -- 172.21.15.99:0/2245476420 shutdown_connections 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 --2- 172.21.15.99:0/2245476420 >> v2:172.21.15.133:3300/0 conn(0x7f84140d3380 0x7f84140cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 --2- 172.21.15.99:0/2245476420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8414160770 0x7f84140b3cc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 --2- 172.21.15.99:0/2245476420 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84140d0d60 0x7f84140d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 -- 172.21.15.99:0/2245476420 >> 172.21.15.99:0/2245476420 conn(0x7f841408afd0 msgr2=0x7f841409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 -- 172.21.15.99:0/2245476420 shutdown_connections 2024-03-29T09:10:55.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.729+0000 7f841bc00700 1 -- 172.21.15.99:0/2245476420 wait complete. 2024-03-29T09:10:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.731+0000 7f841bc00700 1 Processor -- start 2024-03-29T09:10:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.731+0000 7f841bc00700 1 -- start start 2024-03-29T09:10:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f841bc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140d0d60 0x7f84140b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f841bc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84140d3380 0x7f84140ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f841bc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8414160770 0x7f8414146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f841bc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414147360 con 0x7f84140d3380 2024-03-29T09:10:55.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f841bc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84141f24f0 con 0x7f84140d0d60 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f841bc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84140b82c0 con 0x7f8414160770 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f8413e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84140d3380 0x7f84140ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f8413e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84140d3380 0x7f84140ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33434/0 (socket says 172.21.15.99:33434) 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f8413e00700 1 -- 172.21.15.99:0/2704533470 learned_addr learned my addr 172.21.15.99:0/2704533470 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8419200700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140d0d60 0x7f84140b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.732+0000 7f8419c00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8414160770 0x7f8414146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8413e00700 1 -- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8414160770 msgr2=0x7f8414146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8413e00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8414160770 0x7f8414146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8413e00700 1 -- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140d0d60 msgr2=0x7f84140b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8413e00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140d0d60 0x7f84140b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8413e00700 1 -- 172.21.15.99:0/2704533470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408009550 con 0x7f84140d3380 2024-03-29T09:10:55.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8413e00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84140d3380 0x7f84140ccc80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f840000c750 tx=0x7f840000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:55.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8410c00700 1 -- 172.21.15.99:0/2704533470 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400011800 con 0x7f84140d3380 2024-03-29T09:10:55.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8410c00700 1 -- 172.21.15.99:0/2704533470 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8400011960 con 0x7f84140d3380 2024-03-29T09:10:55.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f8410c00700 1 -- 172.21.15.99:0/2704533470 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400011cd0 con 0x7f84140d3380 2024-03-29T09:10:55.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.733+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84140b85a0 con 0x7f84140d3380 2024-03-29T09:10:55.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.734+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84140b8b60 con 0x7f84140d3380 2024-03-29T09:10:55.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.734+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84140678b0 con 0x7f84140d3380 2024-03-29T09:10:55.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.735+0000 7f8410c00700 1 -- 172.21.15.99:0/2704533470 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8400021350 con 0x7f84140d3380 2024-03-29T09:10:55.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.736+0000 7f8410c00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f83f406ac90 0x7f83f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:55.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.736+0000 7f8410c00700 1 -- 172.21.15.99:0/2704533470 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f84000895f0 con 0x7f84140d3380 2024-03-29T09:10:55.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.736+0000 7f8419200700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f83f406ac90 0x7f83f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:55.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.737+0000 7f8419200700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f83f406ac90 0x7f83f406d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8408014040 tx=0x7f8408008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:55.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:55.740+0000 7f8410c00700 1 -- 172.21.15.99:0/2704533470 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f84000543d0 con 0x7f84140d3380 2024-03-29T09:10:56.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.025+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f84140c96e0 con 0x7f84140d3380 2024-03-29T09:10:56.029 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.030+0000 7f8410c00700 1 -- 172.21.15.99:0/2704533470 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v16) v1 ==== 210+0+0 (secure 0 0 0) 0x7f84000579f0 con 0x7f84140d3380 2024-03-29T09:10:56.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f83f406ac90 msgr2=0x7f83f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f83f406ac90 0x7f83f406d150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8408014040 tx=0x7f8408008030 comp rx=0 tx=0).stop 2024-03-29T09:10:56.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84140d3380 msgr2=0x7f84140ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84140d3380 0x7f84140ccc80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f840000c750 tx=0x7f840000ca60 comp rx=0 tx=0).stop 2024-03-29T09:10:56.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 shutdown_connections 2024-03-29T09:10:56.032 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f83f406ac90 0x7f83f406d150 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8414160770 0x7f8414146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84140d0d60 0x7f84140b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 --2- 172.21.15.99:0/2704533470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f84140d3380 0x7f84140ccc80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 >> 172.21.15.99:0/2704533470 conn(0x7f841408afd0 msgr2=0x7f84140cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:56.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 shutdown_connections 2024-03-29T09:10:56.033 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.033+0000 7f841bc00700 1 -- 172.21.15.99:0/2704533470 wait complete. 2024-03-29T09:10:56.033 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:10:56.050 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:56.050 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:56.092 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:56.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.322+0000 7f88a3600700 1 Processor -- start 2024-03-29T09:10:56.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.322+0000 7f88a3600700 1 -- start start 2024-03-29T09:10:56.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.323+0000 7f88a3600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c15ae70 0x7f889c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.323+0000 7f88a3600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f889c0b3e00 0x7f889c0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.323+0000 7f88a3600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c0b14f0 con 0x7f889c0b0b50 2024-03-29T09:10:56.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.323+0000 7f88a3600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c15c220 con 0x7f889c0b3e00 2024-03-29T09:10:56.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.323+0000 7f88a3600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c15b8f0 con 0x7f889c15ae70 2024-03-29T09:10:56.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a1600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f889c0b3e00 0x7f889c0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a0c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c15ae70 0x7f889c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f889be00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f889c0b0b50 0x7f889c0b36f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:33568/0 (socket says 172.21.15.99:33568) 2024-03-29T09:10:56.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a1600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f889c0b3e00 0x7f889c0b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:38292/0 (socket says 172.21.15.99:38292) 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f889be00700 1 -- 172.21.15.99:0/1044136967 learned_addr learned my addr 172.21.15.99:0/1044136967 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a0c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c15ae70 0x7f889c15b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:44824/0 (socket says 172.21.15.99:44824) 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a0c00700 1 -- 172.21.15.99:0/1044136967 >> v2:172.21.15.133:3300/0 conn(0x7f889c0b3e00 msgr2=0x7f889c0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a0c00700 1 --2- 172.21.15.99:0/1044136967 >> v2:172.21.15.133:3300/0 conn(0x7f889c0b3e00 0x7f889c0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a0c00700 1 -- 172.21.15.99:0/1044136967 >> v1:172.21.15.133:6789/0 conn(0x7f889c0b0b50 legacy=0x7f889c0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a0c00700 1 -- 172.21.15.99:0/1044136967 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c15bbd0 con 0x7f889c15ae70 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f88a1600700 1 --2- 172.21.15.99:0/1044136967 >> v2:172.21.15.133:3300/0 conn(0x7f889c0b3e00 0x7f889c0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.324+0000 7f889b400700 1 -- 172.21.15.99:0/1044136967 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934957651 0 0) 0x7f889c0b14f0 con 0x7f889c0b0b50 2024-03-29T09:10:56.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.325+0000 7f88a0c00700 1 --2- 172.21.15.99:0/1044136967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c15ae70 0x7f889c15b250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8890009840 tx=0x7f8890009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc855467d8bf0057 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:56.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.325+0000 7f889b400700 1 -- 172.21.15.99:0/1044136967 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890011830 con 0x7f889c15ae70 2024-03-29T09:10:56.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.325+0000 7f889b400700 1 -- 172.21.15.99:0/1044136967 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8890011990 con 0x7f889c15ae70 2024-03-29T09:10:56.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.325+0000 7f889b400700 1 -- 172.21.15.99:0/1044136967 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890011d20 con 0x7f889c15ae70 2024-03-29T09:10:56.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 -- 172.21.15.99:0/1044136967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c15ae70 msgr2=0x7f889c15b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 --2- 172.21.15.99:0/1044136967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c15ae70 0x7f889c15b250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8890009840 tx=0x7f8890009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:56.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 -- 172.21.15.99:0/1044136967 shutdown_connections 2024-03-29T09:10:56.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 --2- 172.21.15.99:0/1044136967 >> v2:172.21.15.133:3300/0 conn(0x7f889c0b3e00 0x7f889c0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 --2- 172.21.15.99:0/1044136967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c15ae70 0x7f889c15b250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 -- 172.21.15.99:0/1044136967 >> 172.21.15.99:0/1044136967 conn(0x7f889c066540 msgr2=0x7f889c066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:56.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 -- 172.21.15.99:0/1044136967 shutdown_connections 2024-03-29T09:10:56.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.326+0000 7f88a3600700 1 -- 172.21.15.99:0/1044136967 wait complete. 2024-03-29T09:10:56.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.328+0000 7f88a3600700 1 Processor -- start 2024-03-29T09:10:56.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.328+0000 7f88a3600700 1 -- start start 2024-03-29T09:10:56.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a3600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f889c0b0b50 0x7f889c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a3600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c0b3e00 0x7f889c152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a3600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f889c15ae70 0x7f889c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a3600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c153220 con 0x7f889c0b0b50 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a3600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c1f6710 con 0x7f889c15ae70 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a3600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c1f6960 con 0x7f889c0b3e00 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f889c0b0b50 0x7f889c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f889c0b0b50 0x7f889c1517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33448/0 (socket says 172.21.15.99:33448) 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 -- 172.21.15.99:0/3886358191 learned_addr learned my addr 172.21.15.99:0/3886358191 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f889be00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c0b3e00 0x7f889c152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a1600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f889c15ae70 0x7f889c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 -- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c0b3e00 msgr2=0x7f889c152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c0b3e00 0x7f889c152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 -- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f889c15ae70 msgr2=0x7f889c070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f889c15ae70 0x7f889c070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a0c00700 1 -- 172.21.15.99:0/3886358191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894009a70 con 0x7f889c0b0b50 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f889be00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c0b3e00 0x7f889c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:56.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.329+0000 7f88a1600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f889c15ae70 0x7f889c070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.330+0000 7f88a0c00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f889c0b0b50 0x7f889c1517f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8890009fd0 tx=0x7f8890004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.330+0000 7f8898c00700 1 -- 172.21.15.99:0/3886358191 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88900038a0 con 0x7f889c0b0b50 2024-03-29T09:10:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.330+0000 7f8898c00700 1 -- 172.21.15.99:0/3886358191 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889001f070 con 0x7f889c0b0b50 2024-03-29T09:10:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.330+0000 7f8898c00700 1 -- 172.21.15.99:0/3886358191 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890003c40 con 0x7f889c0b0b50 2024-03-29T09:10:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.330+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88900094a0 con 0x7f889c0b0b50 2024-03-29T09:10:56.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.330+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c1f6f20 con 0x7f889c0b0b50 2024-03-29T09:10:56.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.331+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c0677a0 con 0x7f889c0b0b50 2024-03-29T09:10:56.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.332+0000 7f8898c00700 1 -- 172.21.15.99:0/3886358191 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8890026480 con 0x7f889c0b0b50 2024-03-29T09:10:56.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.333+0000 7f8898c00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f887c06ac90 0x7f887c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.333+0000 7f8898c00700 1 -- 172.21.15.99:0/3886358191 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f8890021070 con 0x7f889c0b0b50 2024-03-29T09:10:56.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.333+0000 7f889be00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f887c06ac90 0x7f887c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.334+0000 7f889be00700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f887c06ac90 0x7f887c06d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f889c154350 tx=0x7f8888007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:56.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.337+0000 7f8898c00700 1 -- 172.21.15.99:0/3886358191 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8890055e10 con 0x7f889c0b0b50 2024-03-29T09:10:56.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.546+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f889c0b1a90 con 0x7f889c0b0b50 2024-03-29T09:10:56.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.547+0000 7f8898c00700 1 -- 172.21.15.99:0/3886358191 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7f889002b090 con 0x7f889c0b0b50 2024-03-29T09:10:56.547 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:56.547 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f887c06ac90 msgr2=0x7f887c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f887c06ac90 0x7f887c06d150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f889c154350 tx=0x7f8888007ca0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f889c0b0b50 msgr2=0x7f889c1517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f889c0b0b50 0x7f889c1517f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8890009fd0 tx=0x7f8890004750 comp rx=0 tx=0).stop 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 shutdown_connections 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f887c06ac90 0x7f887c06d150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f889c15ae70 0x7f889c070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f889c0b0b50 0x7f889c1517f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 --2- 172.21.15.99:0/3886358191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f889c0b3e00 0x7f889c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 >> 172.21.15.99:0/3886358191 conn(0x7f889c066540 msgr2=0x7f889c0add20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:56.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 shutdown_connections 2024-03-29T09:10:56.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.550+0000 7f88a3600700 1 -- 172.21.15.99:0/3886358191 wait complete. 2024-03-29T09:10:56.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:10:42.037884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:10:42.900862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-03-29T09:10:53.257132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:10:56.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:56.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.761+0000 7f21cc000700 1 Processor -- start 2024-03-29T09:10:56.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.761+0000 7f21cc000700 1 -- start start 2024-03-29T09:10:56.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.762+0000 7f21cc000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 0x7f21c40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.762+0000 7f21cc000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f21c40d0d60 0x7f21c40d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.762+0000 7f21cc000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40d3810 con 0x7f21c40d3380 2024-03-29T09:10:56.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21cc000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40d1520 con 0x7f21c40d0d60 2024-03-29T09:10:56.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21cc000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40d1800 con 0x7f21c4160770 2024-03-29T09:10:56.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21ca000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f21c40d3380 0x7f21c40cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46568/0 (socket says 172.21.15.99:46568) 2024-03-29T09:10:56.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21ca000700 1 -- 172.21.15.99:0/781749601 learned_addr learned my addr 172.21.15.99:0/781749601 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:56.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21c8c00700 1 --2- 172.21.15.99:0/781749601 >> v2:172.21.15.133:3300/0 conn(0x7f21c40d0d60 0x7f21c40d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21c9600700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 0x7f21c40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840355667 0 0) 0x7f21c40d3810 con 0x7f21c40d3380 2024-03-29T09:10:56.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21ac0035b0 con 0x7f21c40d3380 2024-03-29T09:10:56.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3901155396 0 0) 0x7f21ac0035b0 con 0x7f21c40d3380 2024-03-29T09:10:56.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.763+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 msgr2=0x7f21c40b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 0x7f21c40b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 >> v2:172.21.15.133:3300/0 conn(0x7f21c40d0d60 msgr2=0x7f21c40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 --2- 172.21.15.99:0/781749601 >> v2:172.21.15.133:3300/0 conn(0x7f21c40d0d60 0x7f21c40d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40d1ae0 con 0x7f21c40d3380 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f21bc0019f0 con 0x7f21c40d3380 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 >> v1:172.21.15.99:6789/0 conn(0x7f21c40d3380 legacy=0x7f21c40cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21ac003a00 0x7f21ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21ac004850 0x7f21ac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 0x7f21ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40d1520 con 0x7f21ac004850 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40d1800 con 0x7f21ac003a00 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac00ba60 con 0x7f21c4160770 2024-03-29T09:10:56.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c8c00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21ac004850 0x7f21ac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c9600700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21ac003a00 0x7f21ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21ca000700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 0x7f21ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c8c00700 1 -- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 msgr2=0x7f21ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c8c00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21c4160770 0x7f21ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c8c00700 1 -- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21ac003a00 msgr2=0x7f21ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c8c00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21ac003a00 0x7f21ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c8c00700 1 -- 172.21.15.99:0/781749601 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0035b0 con 0x7f21ac004850 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.764+0000 7f21c9600700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21ac003a00 0x7f21ac003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21c8c00700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21ac004850 0x7f21ac004cd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009b90 tx=0x7f21ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258b0aed80c95af6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b000e6c0 con 0x7f21ac004850 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b000e820 con 0x7f21ac004850 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21c3e00700 1 -- 172.21.15.99:0/781749601 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b000eb90 con 0x7f21ac004850 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21cc000700 1 -- 172.21.15.99:0/781749601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21ac004850 msgr2=0x7f21ac004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21cc000700 1 --2- 172.21.15.99:0/781749601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21ac004850 0x7f21ac004cd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009b90 tx=0x7f21ac005880 comp rx=0 tx=0).stop 2024-03-29T09:10:56.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21ca000700 1 -- 172.21.15.99:0/781749601 reap_dead start 2024-03-29T09:10:56.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21cc000700 1 -- 172.21.15.99:0/781749601 shutdown_connections 2024-03-29T09:10:56.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.765+0000 7f21cc000700 1 -- 172.21.15.99:0/781749601 >> 172.21.15.99:0/781749601 conn(0x7f21c408afd0 msgr2=0x7f21c409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:56.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.766+0000 7f21cc000700 1 -- 172.21.15.99:0/781749601 shutdown_connections 2024-03-29T09:10:56.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.766+0000 7f21cc000700 1 -- 172.21.15.99:0/781749601 wait complete. 2024-03-29T09:10:56.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.767+0000 7f21cc000700 1 Processor -- start 2024-03-29T09:10:56.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.767+0000 7f21cc000700 1 -- start start 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.768+0000 7f21cc000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21ac004850 0x7f21c40afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.768+0000 7f21cc000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c41500b0 0x7f21c414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.768+0000 7f21cc000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21c406b540 0x7f21c406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.768+0000 7f21cc000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c414d610 con 0x7f21c406b540 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.768+0000 7f21cc000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c406bf00 con 0x7f21c41500b0 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.768+0000 7f21cc000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40b04a0 con 0x7f21ac004850 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21c406b540 0x7f21c406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21c406b540 0x7f21c406b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33466/0 (socket says 172.21.15.99:33466) 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 -- 172.21.15.99:0/3400305103 learned_addr learned my addr 172.21.15.99:0/3400305103 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21c9600700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21ac004850 0x7f21c40afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21c8c00700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c41500b0 0x7f21c414cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 -- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21ac004850 msgr2=0x7f21c40afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21ac004850 0x7f21c40afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 -- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c41500b0 msgr2=0x7f21c414cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c41500b0 0x7f21c414cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 -- 172.21.15.99:0/3400305103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40b0780 con 0x7f21c406b540 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21ca000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21c406b540 0x7f21c406b9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f21c40b5d20 tx=0x7f21bc00a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21c1600700 1 -- 172.21.15.99:0/3400305103 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc00acc0 con 0x7f21c406b540 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21c1600700 1 -- 172.21.15.99:0/3400305103 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc00ae40 con 0x7f21c406b540 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.769+0000 7f21c1600700 1 -- 172.21.15.99:0/3400305103 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc013630 con 0x7f21c406b540 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.770+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c40b0a60 con 0x7f21c406b540 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.770+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c4150530 con 0x7f21c406b540 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.771+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c40d11c0 con 0x7f21c406b540 2024-03-29T09:10:56.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.772+0000 7f21c1600700 1 -- 172.21.15.99:0/3400305103 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f21bc013790 con 0x7f21c406b540 2024-03-29T09:10:56.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.773+0000 7f21c1600700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f21a406ac90 0x7f21a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:56.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.773+0000 7f21c1600700 1 -- 172.21.15.99:0/3400305103 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f21bc089e80 con 0x7f21c406b540 2024-03-29T09:10:56.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.773+0000 7f21c9600700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f21a406ac90 0x7f21a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:56.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.774+0000 7f21c9600700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f21a406ac90 0x7f21a406d150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009b80 tx=0x7f21b8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:56.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.777+0000 7f21c1600700 1 -- 172.21.15.99:0/3400305103 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f21c40d11c0 con 0x7f21c406b540 2024-03-29T09:10:56.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.979+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21c40d11c0 con 0x7f21c406b540 2024-03-29T09:10:56.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.980+0000 7f21c1600700 1 -- 172.21.15.99:0/3400305103 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21c40d11c0 con 0x7f21c406b540 2024-03-29T09:10:56.979 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f21a406ac90 msgr2=0x7f21a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f21a406ac90 0x7f21a406d150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009b80 tx=0x7f21b8009210 comp rx=0 tx=0).stop 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21c406b540 msgr2=0x7f21c406b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21c406b540 0x7f21c406b9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f21c40b5d20 tx=0x7f21bc00a550 comp rx=0 tx=0).stop 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 shutdown_connections 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f21a406ac90 0x7f21a406d150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f21c406b540 0x7f21c406b9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21ac004850 0x7f21c40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 --2- 172.21.15.99:0/3400305103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21c41500b0 0x7f21c414cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:56.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 >> 172.21.15.99:0/3400305103 conn(0x7f21c408afd0 msgr2=0x7f21c40b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:56.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 shutdown_connections 2024-03-29T09:10:56.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:56.982+0000 7f21cc000700 1 -- 172.21.15.99:0/3400305103 wait complete. 2024-03-29T09:10:56.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:57.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.192+0000 7f90aee00700 1 Processor -- start 2024-03-29T09:10:57.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.192+0000 7f90aee00700 1 -- start start 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f90aee00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f90a80b3cc0 0x7f90a80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f90aee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f90aee00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a80d3810 con 0x7f90a8160770 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f90aee00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a80b4140 con 0x7f90a80b3cc0 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f90aee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80d1520 con 0x7f90a80d3380 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f909fe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f90a80b3cc0 0x7f90a80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f90a7e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f90a8160770 0x7f90a80d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46584/0 (socket says 172.21.15.99:46584) 2024-03-29T09:10:57.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.194+0000 7f90a7e00700 1 -- 172.21.15.99:0/1570627077 learned_addr learned my addr 172.21.15.99:0/1570627077 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f90ace00700 1 --2- 172.21.15.99:0/1570627077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f909fe00700 1 -- 172.21.15.99:0/1570627077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 msgr2=0x7f90a80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f909fe00700 1 --2- 172.21.15.99:0/1570627077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f909fe00700 1 -- 172.21.15.99:0/1570627077 >> v1:172.21.15.99:6789/0 conn(0x7f90a8160770 legacy=0x7f90a80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f909fe00700 1 -- 172.21.15.99:0/1570627077 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a80d1800 con 0x7f90a80b3cc0 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f90ace00700 1 --2- 172.21.15.99:0/1570627077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f909fe00700 1 --2- 172.21.15.99:0/1570627077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80b3cc0 0x7f90a80d2e40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9090009840 tx=0x7f9090009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc63000d1da8c16d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f90a7400700 1 -- 172.21.15.99:0/1570627077 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090011830 con 0x7f90a80b3cc0 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f90a7400700 1 -- 172.21.15.99:0/1570627077 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090011990 con 0x7f90a80b3cc0 2024-03-29T09:10:57.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.195+0000 7f90a7400700 1 -- 172.21.15.99:0/1570627077 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090011d20 con 0x7f90a80b3cc0 2024-03-29T09:10:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 -- 172.21.15.99:0/1570627077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80b3cc0 msgr2=0x7f90a80d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 --2- 172.21.15.99:0/1570627077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80b3cc0 0x7f90a80d2e40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9090009840 tx=0x7f9090009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 -- 172.21.15.99:0/1570627077 shutdown_connections 2024-03-29T09:10:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 --2- 172.21.15.99:0/1570627077 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 --2- 172.21.15.99:0/1570627077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a80b3cc0 0x7f90a80d2e40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 -- 172.21.15.99:0/1570627077 >> 172.21.15.99:0/1570627077 conn(0x7f90a808afd0 msgr2=0x7f90a809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 -- 172.21.15.99:0/1570627077 shutdown_connections 2024-03-29T09:10:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.196+0000 7f90aee00700 1 -- 172.21.15.99:0/1570627077 wait complete. 2024-03-29T09:10:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.198+0000 7f90aee00700 1 Processor -- start 2024-03-29T09:10:57.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.198+0000 7f90aee00700 1 -- start start 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.199+0000 7f90aee00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90a80b3cc0 0x7f90a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90aee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90aee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a8160770 0x7f90a80b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90aee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8160060 con 0x7f90a8160770 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90aee00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a815b110 con 0x7f90a80b3cc0 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90aee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80cd050 con 0x7f90a80d3380 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a8160770 0x7f90a80b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a8160770 0x7f90a80b66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33476/0 (socket says 172.21.15.99:33476) 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 -- 172.21.15.99:0/2166327907 learned_addr learned my addr 172.21.15.99:0/2166327907 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f909fe00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 -- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 msgr2=0x7f90a80cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90a7e00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90a80b3cc0 0x7f90a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 -- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90a80b3cc0 msgr2=0x7f90a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90a80b3cc0 0x7f90a815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90ace00700 1 -- 172.21.15.99:0/2166327907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900094a0 con 0x7f90a8160770 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f909fe00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.200+0000 7f90a7e00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90a80b3cc0 0x7f90a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.201+0000 7f90ace00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a8160770 0x7f90a80b66d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f90a000cef0 tx=0x7f90a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:57.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.201+0000 7f90a4c00700 1 -- 172.21.15.99:0/2166327907 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000ae50 con 0x7f90a8160770 2024-03-29T09:10:57.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.201+0000 7f90a4c00700 1 -- 172.21.15.99:0/2166327907 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0015440 con 0x7f90a8160770 2024-03-29T09:10:57.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.201+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a80cbfb0 con 0x7f90a8160770 2024-03-29T09:10:57.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.201+0000 7f90a4c00700 1 -- 172.21.15.99:0/2166327907 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00157f0 con 0x7f90a8160770 2024-03-29T09:10:57.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.201+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a80cc570 con 0x7f90a8160770 2024-03-29T09:10:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.202+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a80678b0 con 0x7f90a8160770 2024-03-29T09:10:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.203+0000 7f90a4c00700 1 -- 172.21.15.99:0/2166327907 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f90a000d430 con 0x7f90a8160770 2024-03-29T09:10:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.204+0000 7f90a4c00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f908406ac90 0x7f908406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.204+0000 7f90a4c00700 1 -- 172.21.15.99:0/2166327907 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f90a0089c50 con 0x7f90a8160770 2024-03-29T09:10:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.204+0000 7f90a7e00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f908406ac90 0x7f908406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.204+0000 7f90a7e00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f908406ac90 0x7f908406d150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9098005910 tx=0x7f9098009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:57.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.208+0000 7f90a4c00700 1 -- 172.21.15.99:0/2166327907 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f90a0054b50 con 0x7f90a8160770 2024-03-29T09:10:57.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.410+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90a80b3410 con 0x7f90a8160770 2024-03-29T09:10:57.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.411+0000 7f90a4c00700 1 -- 172.21.15.99:0/2166327907 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90a001a0a0 con 0x7f90a8160770 2024-03-29T09:10:57.410 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:57.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f908406ac90 msgr2=0x7f908406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f908406ac90 0x7f908406d150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9098005910 tx=0x7f9098009040 comp rx=0 tx=0).stop 2024-03-29T09:10:57.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a8160770 msgr2=0x7f90a80b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a8160770 0x7f90a80b66d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f90a000cef0 tx=0x7f90a000a340 comp rx=0 tx=0).stop 2024-03-29T09:10:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 shutdown_connections 2024-03-29T09:10:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f908406ac90 0x7f908406d150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90a8160770 0x7f90a80b66d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90a80b3cc0 0x7f90a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 --2- 172.21.15.99:0/2166327907 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90a80d3380 0x7f90a80cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 >> 172.21.15.99:0/2166327907 conn(0x7f90a808afd0 msgr2=0x7f90a806ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:57.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 shutdown_connections 2024-03-29T09:10:57.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.413+0000 7f90aee00700 1 -- 172.21.15.99:0/2166327907 wait complete. 2024-03-29T09:10:57.431 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:10:57.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.624+0000 7f1d9f400700 1 Processor -- start 2024-03-29T09:10:57.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.624+0000 7f1d9f400700 1 -- start start 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.625+0000 7f1d9f400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1d98160770 0x7f1d980b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.625+0000 7f1d9f400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1d980d3550 0x7f1d980ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.625+0000 7f1d9f400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d980d1520 con 0x7f1d980d0d60 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.625+0000 7f1d9f400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d980d1800 con 0x7f1d980d3550 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.625+0000 7f1d9f400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d980d1ae0 con 0x7f1d98160770 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.625+0000 7f1d97e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1d980d0d60 0x7f1d980d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:46592/0 (socket says 172.21.15.99:46592) 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.625+0000 7f1d97e00700 1 -- 172.21.15.99:0/2958753341 learned_addr learned my addr 172.21.15.99:0/2958753341 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d9d400700 1 --2- 172.21.15.99:0/2958753341 >> v2:172.21.15.99:3300/0 conn(0x7f1d980d3550 0x7f1d980ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d9ca00700 1 --2- 172.21.15.99:0/2958753341 >> v2:172.21.15.133:3300/0 conn(0x7f1d98160770 0x7f1d980b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d97400700 1 -- 172.21.15.99:0/2958753341 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113511930 0 0) 0x7f1d980d1520 con 0x7f1d980d0d60 2024-03-29T09:10:57.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d97400700 1 -- 172.21.15.99:0/2958753341 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d800035b0 con 0x7f1d980d0d60 2024-03-29T09:10:57.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d9d400700 1 -- 172.21.15.99:0/2958753341 >> v2:172.21.15.133:3300/0 conn(0x7f1d98160770 msgr2=0x7f1d980b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:57.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d9d400700 1 --2- 172.21.15.99:0/2958753341 >> v2:172.21.15.133:3300/0 conn(0x7f1d98160770 0x7f1d980b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d9d400700 1 -- 172.21.15.99:0/2958753341 >> v1:172.21.15.99:6789/0 conn(0x7f1d980d0d60 legacy=0x7f1d980d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d9d400700 1 -- 172.21.15.99:0/2958753341 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980d1dc0 con 0x7f1d980d3550 2024-03-29T09:10:57.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.626+0000 7f1d9d400700 1 --2- 172.21.15.99:0/2958753341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d980ce080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1d980b5d20 tx=0x7f1d9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f181268de61398e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:57.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d97400700 1 -- 172.21.15.99:0/2958753341 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9000e070 con 0x7f1d980d3550 2024-03-29T09:10:57.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d97400700 1 -- 172.21.15.99:0/2958753341 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9000bb20 con 0x7f1d980d3550 2024-03-29T09:10:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d97400700 1 -- 172.21.15.99:0/2958753341 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90012430 con 0x7f1d980d3550 2024-03-29T09:10:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 -- 172.21.15.99:0/2958753341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 msgr2=0x7f1d980ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 --2- 172.21.15.99:0/2958753341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d980ce080 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1d980b5d20 tx=0x7f1d9000b330 comp rx=0 tx=0).stop 2024-03-29T09:10:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 -- 172.21.15.99:0/2958753341 shutdown_connections 2024-03-29T09:10:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 --2- 172.21.15.99:0/2958753341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d980ce080 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 --2- 172.21.15.99:0/2958753341 >> v2:172.21.15.133:3300/0 conn(0x7f1d98160770 0x7f1d980b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 -- 172.21.15.99:0/2958753341 >> 172.21.15.99:0/2958753341 conn(0x7f1d9808afd0 msgr2=0x7f1d9809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 -- 172.21.15.99:0/2958753341 shutdown_connections 2024-03-29T09:10:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.627+0000 7f1d9f400700 1 -- 172.21.15.99:0/2958753341 wait complete. 2024-03-29T09:10:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.629+0000 7f1d9f400700 1 Processor -- start 2024-03-29T09:10:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.629+0000 7f1d9f400700 1 -- start start 2024-03-29T09:10:57.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9f400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d980d0d60 0x7f1d980ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d981f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9f400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d98160770 0x7f1d981f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9f400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981f2b90 con 0x7f1d980d3550 2024-03-29T09:10:57.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9f400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980cb8e0 con 0x7f1d98160770 2024-03-29T09:10:57.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9f400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d981f3c20 con 0x7f1d980d0d60 2024-03-29T09:10:57.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d981f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d97e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d981f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33506/0 (socket says 172.21.15.99:33506) 2024-03-29T09:10:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d97e00700 1 -- 172.21.15.99:0/451567431 learned_addr learned my addr 172.21.15.99:0/451567431 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9ca00700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d980d0d60 0x7f1d980ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.630+0000 7f1d9d400700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d98160770 0x7f1d981f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d97e00700 1 -- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d980d0d60 msgr2=0x7f1d980ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d97e00700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d980d0d60 0x7f1d980ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d97e00700 1 -- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d98160770 msgr2=0x7f1d981f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d97e00700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d98160770 0x7f1d981f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d97e00700 1 -- 172.21.15.99:0/451567431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90009a70 con 0x7f1d980d3550 2024-03-29T09:10:57.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d97e00700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d981f2650 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400c700 tx=0x7f1d8400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:57.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d94c00700 1 -- 172.21.15.99:0/451567431 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84005d70 con 0x7f1d980d3550 2024-03-29T09:10:57.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d94c00700 1 -- 172.21.15.99:0/451567431 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d840070d0 con 0x7f1d980d3550 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d980ca840 con 0x7f1d980d3550 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d94c00700 1 -- 172.21.15.99:0/451567431 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d84007460 con 0x7f1d980d3550 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.631+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d980cae00 con 0x7f1d980d3550 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.632+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d980678b0 con 0x7f1d980d3550 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.633+0000 7f1d94c00700 1 -- 172.21.15.99:0/451567431 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1d840075c0 con 0x7f1d980d3550 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.634+0000 7f1d94c00700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1d7806ac90 0x7f1d7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.634+0000 7f1d94c00700 1 -- 172.21.15.99:0/451567431 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f1d84088b10 con 0x7f1d980d3550 2024-03-29T09:10:57.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.634+0000 7f1d9ca00700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1d7806ac90 0x7f1d7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.635+0000 7f1d9ca00700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1d7806ac90 0x7f1d7806d150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1d980b8660 tx=0x7f1d8c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:57.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.638+0000 7f1d94c00700 1 -- 172.21.15.99:0/451567431 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1d84053980 con 0x7f1d980d3550 2024-03-29T09:10:57.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.845+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d980c9af0 con 0x7f1d980d3550 2024-03-29T09:10:57.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.846+0000 7f1d94c00700 1 -- 172.21.15.99:0/451567431 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d84017020 con 0x7f1d980d3550 2024-03-29T09:10:57.845 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:10:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1d7806ac90 msgr2=0x7f1d7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1d7806ac90 0x7f1d7806d150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1d980b8660 tx=0x7f1d8c009210 comp rx=0 tx=0).stop 2024-03-29T09:10:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 msgr2=0x7f1d981f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d981f2650 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400c700 tx=0x7f1d8400ca10 comp rx=0 tx=0).stop 2024-03-29T09:10:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 shutdown_connections 2024-03-29T09:10:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1d7806ac90 0x7f1d7806d150 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d98160770 0x7f1d981f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d980d0d60 0x7f1d980ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 --2- 172.21.15.99:0/451567431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1d980d3550 0x7f1d981f2650 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 >> 172.21.15.99:0/451567431 conn(0x7f1d9808afd0 msgr2=0x7f1d980b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 shutdown_connections 2024-03-29T09:10:57.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:57.848+0000 7f1d9f400700 1 -- 172.21.15.99:0/451567431 wait complete. 2024-03-29T09:10:57.865 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-03-29T09:10:58.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.061+0000 7f43b6a00700 1 Processor -- start 2024-03-29T09:10:58.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.061+0000 7f43b6a00700 1 -- start start 2024-03-29T09:10:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.062+0000 7f43b6a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.062+0000 7f43b6a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43b00d3550 0x7f43b00ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.062+0000 7f43b6a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b00d1520 con 0x7f43b00d0d60 2024-03-29T09:10:58.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.062+0000 7f43b6a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b00d1800 con 0x7f43b00d3550 2024-03-29T09:10:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.062+0000 7f43b6a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00d1ae0 con 0x7f43b0160770 2024-03-29T09:10:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.062+0000 7f43b4a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43b00d3550 0x7f43b00ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43b4a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f43b00d3550 0x7f43b00ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35674/0 (socket says 172.21.15.99:35674) 2024-03-29T09:10:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43b4a00700 1 -- 172.21.15.99:0/1879097001 learned_addr learned my addr 172.21.15.99:0/1879097001 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43afe00700 1 --2- 172.21.15.99:0/1879097001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43b4a00700 1 -- 172.21.15.99:0/1879097001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 msgr2=0x7f43b00b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:58.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43b4a00700 1 --2- 172.21.15.99:0/1879097001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43b4a00700 1 -- 172.21.15.99:0/1879097001 >> v1:172.21.15.133:6789/0 conn(0x7f43b00d0d60 legacy=0x7f43b00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43b4a00700 1 -- 172.21.15.99:0/1879097001 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00d1dc0 con 0x7f43b00d3550 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.063+0000 7f43b4a00700 1 --2- 172.21.15.99:0/1879097001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ce080 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009e50 tx=0x7f43a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c328a9d63a28fe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43aea00700 1 -- 172.21.15.99:0/1879097001 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800e070 con 0x7f43b00d3550 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43aea00700 1 -- 172.21.15.99:0/1879097001 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800b9c0 con 0x7f43b00d3550 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43aea00700 1 -- 172.21.15.99:0/1879097001 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800bd70 con 0x7f43b00d3550 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 -- 172.21.15.99:0/1879097001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 msgr2=0x7f43b00ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 --2- 172.21.15.99:0/1879097001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ce080 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009e50 tx=0x7f43a800b330 comp rx=0 tx=0).stop 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 -- 172.21.15.99:0/1879097001 shutdown_connections 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 --2- 172.21.15.99:0/1879097001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ce080 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 --2- 172.21.15.99:0/1879097001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 -- 172.21.15.99:0/1879097001 >> 172.21.15.99:0/1879097001 conn(0x7f43b008afd0 msgr2=0x7f43b009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 -- 172.21.15.99:0/1879097001 shutdown_connections 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.064+0000 7f43b6a00700 1 -- 172.21.15.99:0/1879097001 wait complete. 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.066+0000 7f43b6a00700 1 Processor -- start 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.066+0000 7f43b6a00700 1 -- start start 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43b6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b00d0d60 0x7f43b015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43b6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43b6a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43b6a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00cc3d0 con 0x7f43b00d3550 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43b6a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00b5df0 con 0x7f43b00d0d60 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43b6a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00b60d0 con 0x7f43b0160770 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43af400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43af400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35686/0 (socket says 172.21.15.99:35686) 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43afe00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b00d0d60 0x7f43b015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43af400700 1 -- 172.21.15.99:0/2883412855 learned_addr learned my addr 172.21.15.99:0/2883412855 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.067+0000 7f43b4a00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43af400700 1 -- 172.21.15.99:0/2883412855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 msgr2=0x7f43b00ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43af400700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43af400700 1 -- 172.21.15.99:0/2883412855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b00d0d60 msgr2=0x7f43b015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43af400700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b00d0d60 0x7f43b015b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43af400700 1 -- 172.21.15.99:0/2883412855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009b30 con 0x7f43b00d3550 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43af400700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ca260 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c4e0 tx=0x7f439c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43a7e00700 1 -- 172.21.15.99:0/2883412855 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0052d0 con 0x7f43b00d3550 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43a7e00700 1 -- 172.21.15.99:0/2883412855 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c005430 con 0x7f43b00d3550 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43a7e00700 1 -- 172.21.15.99:0/2883412855 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0038c0 con 0x7f43b00d3550 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b00b63b0 con 0x7f43b00d3550 2024-03-29T09:10:58.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.068+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b015b9c0 con 0x7f43b00d3550 2024-03-29T09:10:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.069+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b00678b0 con 0x7f43b00d3550 2024-03-29T09:10:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.070+0000 7f43a7e00700 1 -- 172.21.15.99:0/2883412855 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f439c003a20 con 0x7f43b00d3550 2024-03-29T09:10:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.071+0000 7f43a7e00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f439006ac90 0x7f439006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.071+0000 7f43a7e00700 1 -- 172.21.15.99:0/2883412855 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f439c088a30 con 0x7f43b00d3550 2024-03-29T09:10:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.071+0000 7f43afe00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f439006ac90 0x7f439006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.072+0000 7f43afe00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f439006ac90 0x7f439006d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43a0007a30 tx=0x7f43a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:58.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.076+0000 7f43a7e00700 1 -- 172.21.15.99:0/2883412855 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f439c0538a0 con 0x7f43b00d3550 2024-03-29T09:10:58.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.276+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f43b0072c80 con 0x7f43b00d3550 2024-03-29T09:10:58.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.276+0000 7f43a7e00700 1 -- 172.21.15.99:0/2883412855 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7f439c0170a0 con 0x7f43b00d3550 2024-03-29T09:10:58.276 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f439006ac90 msgr2=0x7f439006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f439006ac90 0x7f439006d150 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f43a0007a30 tx=0x7f43a0008040 comp rx=0 tx=0).stop 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 msgr2=0x7f43b00ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ca260 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f439c00c4e0 tx=0x7f439c00c7f0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 shutdown_connections 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f439006ac90 0x7f439006d150 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b0160770 0x7f43b00ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b00d0d60 0x7f43b015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 --2- 172.21.15.99:0/2883412855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f43b00d3550 0x7f43b00ca260 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 >> 172.21.15.99:0/2883412855 conn(0x7f43b008afd0 msgr2=0x7f43b006cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:58.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 shutdown_connections 2024-03-29T09:10:58.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.279+0000 7f43b6a00700 1 -- 172.21.15.99:0/2883412855 wait complete. 2024-03-29T09:10:58.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:10:58.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.487+0000 7fa620a00700 1 Processor -- start 2024-03-29T09:10:58.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.487+0000 7fa620a00700 1 -- start start 2024-03-29T09:10:58.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa620a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa61c0d1b20 0x7fa61c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa620a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c160cc0 0x7fa61c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa620a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c0cddc0 con 0x7fa61c1608e0 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa620a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61c0ce0a0 con 0x7fa61c0d1b20 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa620a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0ce380 con 0x7fa61c160cc0 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa613e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa61c0d1b20 0x7fa61c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa613e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa61c0d1b20 0x7fa61c0d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35698/0 (socket says 172.21.15.99:35698) 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa613e00700 1 -- 172.21.15.99:0/3480249349 learned_addr learned my addr 172.21.15.99:0/3480249349 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa61a800700 1 --2- 172.21.15.99:0/3480249349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c160cc0 0x7fa61c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa613e00700 1 -- 172.21.15.99:0/3480249349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c160cc0 msgr2=0x7fa61c0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa613e00700 1 --2- 172.21.15.99:0/3480249349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c160cc0 0x7fa61c0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa613e00700 1 -- 172.21.15.99:0/3480249349 >> v1:172.21.15.133:6789/0 conn(0x7fa61c1608e0 legacy=0x7fa61c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:58.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.489+0000 7fa613e00700 1 -- 172.21.15.99:0/3480249349 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0ce660 con 0x7fa61c0d1b20 2024-03-29T09:10:58.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa613e00700 1 --2- 172.21.15.99:0/3480249349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c0d1b20 0x7fa61c0d0150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa6040098f0 tx=0x7fa604009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3298399d857cf739 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:58.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa619400700 1 -- 172.21.15.99:0/3480249349 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604004230 con 0x7fa61c0d1b20 2024-03-29T09:10:58.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa619400700 1 -- 172.21.15.99:0/3480249349 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604004390 con 0x7fa61c0d1b20 2024-03-29T09:10:58.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa619400700 1 -- 172.21.15.99:0/3480249349 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040036a0 con 0x7fa61c0d1b20 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa620a00700 1 -- 172.21.15.99:0/3480249349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c0d1b20 msgr2=0x7fa61c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa620a00700 1 --2- 172.21.15.99:0/3480249349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c0d1b20 0x7fa61c0d0150 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa6040098f0 tx=0x7fa604009c00 comp rx=0 tx=0).stop 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa620a00700 1 -- 172.21.15.99:0/3480249349 shutdown_connections 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa620a00700 1 --2- 172.21.15.99:0/3480249349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c160cc0 0x7fa61c0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa620a00700 1 --2- 172.21.15.99:0/3480249349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c0d1b20 0x7fa61c0d0150 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa620a00700 1 -- 172.21.15.99:0/3480249349 >> 172.21.15.99:0/3480249349 conn(0x7fa61c09c490 msgr2=0x7fa61c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.490+0000 7fa620a00700 1 -- 172.21.15.99:0/3480249349 shutdown_connections 2024-03-29T09:10:58.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.491+0000 7fa620a00700 1 -- 172.21.15.99:0/3480249349 wait complete. 2024-03-29T09:10:58.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.492+0000 7fa620a00700 1 Processor -- start 2024-03-29T09:10:58.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.492+0000 7fa620a00700 1 -- start start 2024-03-29T09:10:58.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa620a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c0d1b20 0x7fa61c0b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa620a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c1608e0 0x7fa61c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa620a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa61c160cc0 0x7fa61c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa620a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c1f4280 con 0x7fa61c1608e0 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa620a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c1f3fa0 con 0x7fa61c160cc0 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa620a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c1f4580 con 0x7fa61c0d1b20 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa613e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c1608e0 0x7fa61c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa613e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c1608e0 0x7fa61c06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35710/0 (socket says 172.21.15.99:35710) 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa613e00700 1 -- 172.21.15.99:0/2392611220 learned_addr learned my addr 172.21.15.99:0/2392611220 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.493+0000 7fa619e00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c0d1b20 0x7fa61c0b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa61a800700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa61c160cc0 0x7fa61c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa613e00700 1 -- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c0d1b20 msgr2=0x7fa61c0b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa613e00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c0d1b20 0x7fa61c0b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa613e00700 1 -- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa61c160cc0 msgr2=0x7fa61c06b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa613e00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa61c160cc0 0x7fa61c06b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa613e00700 1 -- 172.21.15.99:0/2392611220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604009550 con 0x7fa61c1608e0 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa61a800700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa61c160cc0 0x7fa61c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa613e00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c1608e0 0x7fa61c06b440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa604009130 tx=0x7fa604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa612000700 1 -- 172.21.15.99:0/2392611220 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604003b50 con 0x7fa61c1608e0 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa612000700 1 -- 172.21.15.99:0/2392611220 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60401f070 con 0x7fa61c1608e0 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c1f4860 con 0x7fa61c1608e0 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa612000700 1 -- 172.21.15.99:0/2392611220 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60400b660 con 0x7fa61c1608e0 2024-03-29T09:10:58.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.494+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c0b8930 con 0x7fa61c1608e0 2024-03-29T09:10:58.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.495+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c0b4a10 con 0x7fa61c1608e0 2024-03-29T09:10:58.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.496+0000 7fa612000700 1 -- 172.21.15.99:0/2392611220 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa604021030 con 0x7fa61c1608e0 2024-03-29T09:10:58.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.497+0000 7fa612000700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa5f806ac90 0x7fa5f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:58.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.497+0000 7fa612000700 1 -- 172.21.15.99:0/2392611220 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa60408acb0 con 0x7fa61c1608e0 2024-03-29T09:10:58.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.497+0000 7fa619e00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa5f806ac90 0x7fa5f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:58.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.498+0000 7fa619e00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa5f806ac90 0x7fa5f806d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0d1ac0 tx=0x7fa60c007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:58.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.501+0000 7fa612000700 1 -- 172.21.15.99:0/2392611220 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa604055aa0 con 0x7fa61c1608e0 2024-03-29T09:10:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.719+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa61c0b4a10 con 0x7fa61c1608e0 2024-03-29T09:10:58.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.720+0000 7fa612000700 1 -- 172.21.15.99:0/2392611220 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa6040590c0 con 0x7fa61c1608e0 2024-03-29T09:10:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa5f806ac90 msgr2=0x7fa5f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa5f806ac90 0x7fa5f806d150 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0d1ac0 tx=0x7fa60c007ec0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c1608e0 msgr2=0x7fa61c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c1608e0 0x7fa61c06b440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa604009130 tx=0x7fa604004750 comp rx=0 tx=0).stop 2024-03-29T09:10:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 shutdown_connections 2024-03-29T09:10:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa5f806ac90 0x7fa5f806d150 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa61c160cc0 0x7fa61c06b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa61c0d1b20 0x7fa61c0b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 --2- 172.21.15.99:0/2392611220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa61c1608e0 0x7fa61c06b440 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 >> 172.21.15.99:0/2392611220 conn(0x7fa61c09c490 msgr2=0x7fa61c0cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 shutdown_connections 2024-03-29T09:10:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:58.722+0000 7fa620a00700 1 -- 172.21.15.99:0/2392611220 wait complete. 2024-03-29T09:10:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:10:58.739 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:10:58.739 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:10:58.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.013+0000 7f5c27200700 1 Processor -- start 2024-03-29T09:10:59.012 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.013+0000 7f5c27200700 1 -- start start 2024-03-29T09:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c27200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5c200b1260 0x7f5c2015b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c27200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c200b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c27200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c2015c220 con 0x7f5c2015ae70 2024-03-29T09:10:59.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c27200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200b40c0 con 0x7f5c200b1260 2024-03-29T09:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c27200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2015b8f0 con 0x7f5c200b36f0 2024-03-29T09:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c1fe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5c2015ae70 0x7f5c200b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38768/0 (socket says 172.21.15.99:38768) 2024-03-29T09:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c1fe00700 1 -- 172.21.15.99:0/2113643939 learned_addr learned my addr 172.21.15.99:0/2113643939 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.014+0000 7f5c1f400700 1 --2- 172.21.15.99:0/2113643939 >> v2:172.21.15.99:3300/0 conn(0x7f5c200b1260 0x7f5c2015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c25200700 1 --2- 172.21.15.99:0/2113643939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c200b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1ea00700 1 -- 172.21.15.99:0/2113643939 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506704982 0 0) 0x7f5c2015c220 con 0x7f5c2015ae70 2024-03-29T09:10:59.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1ea00700 1 -- 172.21.15.99:0/2113643939 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c040035b0 con 0x7f5c2015ae70 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1f400700 1 -- 172.21.15.99:0/2113643939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 msgr2=0x7f5c200b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1f400700 1 --2- 172.21.15.99:0/2113643939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c200b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1f400700 1 -- 172.21.15.99:0/2113643939 >> v1:172.21.15.99:6789/0 conn(0x7f5c2015ae70 legacy=0x7f5c200b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1f400700 1 -- 172.21.15.99:0/2113643939 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2015bbd0 con 0x7f5c200b1260 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1f400700 1 --2- 172.21.15.99:0/2113643939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2015b250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009840 tx=0x7f5c0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466e8a816cd59cc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1ea00700 1 -- 172.21.15.99:0/2113643939 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c011830 con 0x7f5c200b1260 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1ea00700 1 -- 172.21.15.99:0/2113643939 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c011990 con 0x7f5c200b1260 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.015+0000 7f5c1ea00700 1 -- 172.21.15.99:0/2113643939 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c011d20 con 0x7f5c200b1260 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 -- 172.21.15.99:0/2113643939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 msgr2=0x7f5c2015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 --2- 172.21.15.99:0/2113643939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2015b250 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009840 tx=0x7f5c0c009b50 comp rx=0 tx=0).stop 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 -- 172.21.15.99:0/2113643939 shutdown_connections 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 --2- 172.21.15.99:0/2113643939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c200b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 --2- 172.21.15.99:0/2113643939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2015b250 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 -- 172.21.15.99:0/2113643939 >> 172.21.15.99:0/2113643939 conn(0x7f5c20066540 msgr2=0x7f5c20066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:59.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 -- 172.21.15.99:0/2113643939 shutdown_connections 2024-03-29T09:10:59.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.016+0000 7f5c27200700 1 -- 172.21.15.99:0/2113643939 wait complete. 2024-03-29T09:10:59.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.018+0000 7f5c27200700 1 Processor -- start 2024-03-29T09:10:59.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.018+0000 7f5c27200700 1 -- start start 2024-03-29T09:10:59.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c27200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2014e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c27200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c2014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c27200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c2015ae70 0x7f5c201501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c27200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c20152510 con 0x7f5c200b1260 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c27200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c201527f0 con 0x7f5c2015ae70 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c27200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2014f160 con 0x7f5c200b36f0 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c1fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2014e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c1fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2014e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35734/0 (socket says 172.21.15.99:35734) 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c1fe00700 1 -- 172.21.15.99:0/2576109207 learned_addr learned my addr 172.21.15.99:0/2576109207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c1f400700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c2014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c25200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c2015ae70 0x7f5c201501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c1fe00700 1 -- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 msgr2=0x7f5c2014d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.019+0000 7f5c1fe00700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c2014d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c1fe00700 1 -- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c2015ae70 msgr2=0x7f5c201501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c1fe00700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c2015ae70 0x7f5c201501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c1fe00700 1 -- 172.21.15.99:0/2576109207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0094a0 con 0x7f5c200b1260 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c25200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c2015ae70 0x7f5c201501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c1fe00700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2014e930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000c4d0 tx=0x7f5c1000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c17e00700 1 -- 172.21.15.99:0/2576109207 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10005d70 con 0x7f5c200b1260 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c17e00700 1 -- 172.21.15.99:0/2576109207 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c100070d0 con 0x7f5c200b1260 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c17e00700 1 -- 172.21.15.99:0/2576109207 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10007460 con 0x7f5c200b1260 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2014f440 con 0x7f5c200b1260 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.020+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2014ee70 con 0x7f5c200b1260 2024-03-29T09:10:59.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.021+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c200677a0 con 0x7f5c200b1260 2024-03-29T09:10:59.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.022+0000 7f5c17e00700 1 -- 172.21.15.99:0/2576109207 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5c100075c0 con 0x7f5c200b1260 2024-03-29T09:10:59.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.023+0000 7f5c17e00700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c0006ac90 0x7f5c0006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.023+0000 7f5c17e00700 1 -- 172.21.15.99:0/2576109207 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f5c10088a10 con 0x7f5c200b1260 2024-03-29T09:10:59.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.023+0000 7f5c1f400700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c0006ac90 0x7f5c0006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.024+0000 7f5c1f400700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c0006ac90 0x7f5c0006d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c000c00 tx=0x7f5c0c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.028+0000 7f5c17e00700 1 -- 172.21.15.99:0/2576109207 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5c100539b0 con 0x7f5c200b1260 2024-03-29T09:10:59.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.230+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c200b1a90 con 0x7f5c200b1260 2024-03-29T09:10:59.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.232+0000 7f5c17e00700 1 -- 172.21.15.99:0/2576109207 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7f5c10017090 con 0x7f5c200b1260 2024-03-29T09:10:59.231 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:10:59.231 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:10:59.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c0006ac90 msgr2=0x7f5c0006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c0006ac90 0x7f5c0006d150 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c000c00 tx=0x7f5c0c008030 comp rx=0 tx=0).stop 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 msgr2=0x7f5c2014e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2014e930 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5c1000c4d0 tx=0x7f5c1000c7e0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 shutdown_connections 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c0006ac90 0x7f5c0006d150 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c2015ae70 0x7f5c201501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c200b1260 0x7f5c2014e930 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 --2- 172.21.15.99:0/2576109207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c200b36f0 0x7f5c2014d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 >> 172.21.15.99:0/2576109207 conn(0x7f5c20066540 msgr2=0x7f5c2015c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.234+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 shutdown_connections 2024-03-29T09:10:59.234 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.235+0000 7f5c27200700 1 -- 172.21.15.99:0/2576109207 wait complete. 2024-03-29T09:10:59.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:10:42.037884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:10:42.900862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-03-29T09:10:53.257132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:10:59.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:10:59.445 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.446+0000 7fb51f000700 1 Processor -- start 2024-03-29T09:10:59.446 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.446+0000 7fb51f000700 1 -- start start 2024-03-29T09:10:59.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.447+0000 7fb51f000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5180b1fd0 0x7fb518070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb51f000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51806d4e0 0x7fb51806d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb51f000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51806deb0 con 0x7fb5180b3d40 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb51f000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5180717a0 con 0x7fb5180b1fd0 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb51f000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb518159cc0 con 0x7fb51806d4e0 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb517e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5180b1fd0 0x7fb518070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb517e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb5180b1fd0 0x7fb518070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35740/0 (socket says 172.21.15.99:35740) 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb517e00700 1 -- 172.21.15.99:0/2457523903 learned_addr learned my addr 172.21.15.99:0/2457523903 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb51d000700 1 --2- 172.21.15.99:0/2457523903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51806d4e0 0x7fb51806d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb517e00700 1 -- 172.21.15.99:0/2457523903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51806d4e0 msgr2=0x7fb51806d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb517e00700 1 --2- 172.21.15.99:0/2457523903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51806d4e0 0x7fb51806d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb517e00700 1 -- 172.21.15.99:0/2457523903 >> v1:172.21.15.133:6789/0 conn(0x7fb5180b3d40 legacy=0x7fb5180abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb517e00700 1 -- 172.21.15.99:0/2457523903 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51806c720 con 0x7fb5180b1fd0 2024-03-29T09:10:59.448 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.448+0000 7fb516a00700 1 -- 172.21.15.99:0/2457523903 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079609212 0 0) 0x7fb51806deb0 con 0x7fb5180b3d40 2024-03-29T09:10:59.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb517e00700 1 --2- 172.21.15.99:0/2457523903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb518070ee0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb50800bb90 tx=0x7fb50800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbe9ef8ba7472a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb516a00700 1 -- 172.21.15.99:0/2457523903 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800dd30 con 0x7fb5180b1fd0 2024-03-29T09:10:59.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb516a00700 1 -- 172.21.15.99:0/2457523903 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800de90 con 0x7fb5180b1fd0 2024-03-29T09:10:59.449 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb516a00700 1 -- 172.21.15.99:0/2457523903 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508010690 con 0x7fb5180b1fd0 2024-03-29T09:10:59.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb51f000700 1 -- 172.21.15.99:0/2457523903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 msgr2=0x7fb518070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb51f000700 1 --2- 172.21.15.99:0/2457523903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb518070ee0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb50800bb90 tx=0x7fb50800bea0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb51f000700 1 -- 172.21.15.99:0/2457523903 shutdown_connections 2024-03-29T09:10:59.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb51f000700 1 --2- 172.21.15.99:0/2457523903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb51806d4e0 0x7fb51806d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.450 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.449+0000 7fb51f000700 1 --2- 172.21.15.99:0/2457523903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb518070ee0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.450+0000 7fb51f000700 1 -- 172.21.15.99:0/2457523903 >> 172.21.15.99:0/2457523903 conn(0x7fb518066540 msgr2=0x7fb518066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.450+0000 7fb51f000700 1 -- 172.21.15.99:0/2457523903 shutdown_connections 2024-03-29T09:10:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.450+0000 7fb51f000700 1 -- 172.21.15.99:0/2457523903 wait complete. 2024-03-29T09:10:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.451+0000 7fb51f000700 1 Processor -- start 2024-03-29T09:10:59.451 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.451+0000 7fb51f000700 1 -- start start 2024-03-29T09:10:59.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.452+0000 7fb51f000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51806d4e0 0x7fb51809a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.452+0000 7fb51f000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb51814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.452+0000 7fb51f000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5180b3d40 0x7fb5180b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.452+0000 7fb51f000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51809a700 con 0x7fb5180b1fd0 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.452+0000 7fb51f000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51814b550 con 0x7fb51806d4e0 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.452+0000 7fb51f000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51814b830 con 0x7fb5180b3d40 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb51814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb51814acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35752/0 (socket says 172.21.15.99:35752) 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 -- 172.21.15.99:0/2612740366 learned_addr learned my addr 172.21.15.99:0/2612740366 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb51d000700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5180b3d40 0x7fb5180b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517e00700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51806d4e0 0x7fb51809a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 -- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5180b3d40 msgr2=0x7fb5180b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5180b3d40 0x7fb5180b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 -- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51806d4e0 msgr2=0x7fb51809a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51806d4e0 0x7fb51809a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 -- 172.21.15.99:0/2612740366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50800b7f0 con 0x7fb5180b1fd0 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb517400700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb51814acb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c550 tx=0x7fb50400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb50fe00700 1 -- 172.21.15.99:0/2612740366 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5040054e0 con 0x7fb5180b1fd0 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb50fe00700 1 -- 172.21.15.99:0/2612740366 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb504003680 con 0x7fb5180b1fd0 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51814bb10 con 0x7fb5180b1fd0 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.453+0000 7fb50fe00700 1 -- 172.21.15.99:0/2612740366 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504003a30 con 0x7fb5180b1fd0 2024-03-29T09:10:59.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.454+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51814c0d0 con 0x7fb5180b1fd0 2024-03-29T09:10:59.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.455+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5180c8d00 con 0x7fb5180b1fd0 2024-03-29T09:10:59.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.455+0000 7fb50fe00700 1 -- 172.21.15.99:0/2612740366 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb504003b90 con 0x7fb5180b1fd0 2024-03-29T09:10:59.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.456+0000 7fb50fe00700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4f806ac90 0x7fb4f806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.456+0000 7fb50fe00700 1 -- 172.21.15.99:0/2612740366 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb504088aa0 con 0x7fb5180b1fd0 2024-03-29T09:10:59.455 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.456+0000 7fb517e00700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4f806ac90 0x7fb4f806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.457+0000 7fb517e00700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4f806ac90 0x7fb4f806d150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb508000c00 tx=0x7fb508013000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.461+0000 7fb50fe00700 1 -- 172.21.15.99:0/2612740366 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb5180c8d00 con 0x7fb5180b1fd0 2024-03-29T09:10:59.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.662+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5180c8d00 con 0x7fb5180b1fd0 2024-03-29T09:10:59.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.663+0000 7fb50fe00700 1 -- 172.21.15.99:0/2612740366 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5180c8d00 con 0x7fb5180b1fd0 2024-03-29T09:10:59.662 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4f806ac90 msgr2=0x7fb4f806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4f806ac90 0x7fb4f806d150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb508000c00 tx=0x7fb508013000 comp rx=0 tx=0).stop 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 msgr2=0x7fb51814acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb51814acb0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c550 tx=0x7fb50400c910 comp rx=0 tx=0).stop 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 shutdown_connections 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4f806ac90 0x7fb4f806d150 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb5180b3d40 0x7fb5180b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb51806d4e0 0x7fb51809a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 --2- 172.21.15.99:0/2612740366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb5180b1fd0 0x7fb51814acb0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.665+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 >> 172.21.15.99:0/2612740366 conn(0x7fb518066540 msgr2=0x7fb5180ad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.666+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 shutdown_connections 2024-03-29T09:10:59.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.666+0000 7fb51f000700 1 -- 172.21.15.99:0/2612740366 wait complete. 2024-03-29T09:10:59.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:10:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.877+0000 7fd491600700 1 Processor -- start 2024-03-29T09:10:59.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.877+0000 7fd491600700 1 -- start start 2024-03-29T09:10:59.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.878+0000 7fd491600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd48c160770 0x7fd48c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.878+0000 7fd491600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd48c0d0d60 0x7fd48c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd491600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48c0d3810 con 0x7fd48c0d3380 2024-03-29T09:10:59.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd491600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0d1520 con 0x7fd48c160770 2024-03-29T09:10:59.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd491600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48c0d1800 con 0x7fd48c0d0d60 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd48c160770 0x7fd48c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48b400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd48c0d3380 0x7fd48c0cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38772/0 (socket says 172.21.15.99:38772) 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48b400700 1 -- 172.21.15.99:0/1439792956 learned_addr learned my addr 172.21.15.99:0/1439792956 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48a000700 1 --2- 172.21.15.99:0/1439792956 >> v2:172.21.15.133:3300/0 conn(0x7fd48c0d0d60 0x7fd48c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48aa00700 1 -- 172.21.15.99:0/1439792956 >> v2:172.21.15.133:3300/0 conn(0x7fd48c0d0d60 msgr2=0x7fd48c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48aa00700 1 --2- 172.21.15.99:0/1439792956 >> v2:172.21.15.133:3300/0 conn(0x7fd48c0d0d60 0x7fd48c0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48aa00700 1 -- 172.21.15.99:0/1439792956 >> v1:172.21.15.99:6789/0 conn(0x7fd48c0d3380 legacy=0x7fd48c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd48aa00700 1 -- 172.21.15.99:0/1439792956 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c0d1ae0 con 0x7fd48c160770 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.879+0000 7fd489600700 1 -- 172.21.15.99:0/1439792956 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428470776 0 0) 0x7fd48c0d3810 con 0x7fd48c0d3380 2024-03-29T09:10:59.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd48aa00700 1 --2- 172.21.15.99:0/1439792956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b3cc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0b5cf0 tx=0x7fd47c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae2d2e7577ed13e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd489600700 1 -- 172.21.15.99:0/1439792956 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c011830 con 0x7fd48c160770 2024-03-29T09:10:59.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd489600700 1 -- 172.21.15.99:0/1439792956 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c011990 con 0x7fd48c160770 2024-03-29T09:10:59.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd489600700 1 -- 172.21.15.99:0/1439792956 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c011d20 con 0x7fd48c160770 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd491600700 1 -- 172.21.15.99:0/1439792956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 msgr2=0x7fd48c0b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd491600700 1 --2- 172.21.15.99:0/1439792956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b3cc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0b5cf0 tx=0x7fd47c009b20 comp rx=0 tx=0).stop 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd491600700 1 -- 172.21.15.99:0/1439792956 shutdown_connections 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd491600700 1 --2- 172.21.15.99:0/1439792956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b3cc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd491600700 1 --2- 172.21.15.99:0/1439792956 >> v2:172.21.15.133:3300/0 conn(0x7fd48c0d0d60 0x7fd48c0d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd491600700 1 -- 172.21.15.99:0/1439792956 >> 172.21.15.99:0/1439792956 conn(0x7fd48c08afd0 msgr2=0x7fd48c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.880+0000 7fd491600700 1 -- 172.21.15.99:0/1439792956 shutdown_connections 2024-03-29T09:10:59.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.881+0000 7fd491600700 1 -- 172.21.15.99:0/1439792956 wait complete. 2024-03-29T09:10:59.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.882+0000 7fd491600700 1 Processor -- start 2024-03-29T09:10:59.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.882+0000 7fd491600700 1 -- start start 2024-03-29T09:10:59.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd491600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c0d0d60 0x7fd48c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd491600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c0d3380 0x7fd48c0b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd491600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd491600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1f6fa0 con 0x7fd48c160770 2024-03-29T09:10:59.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd491600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1f3fa0 con 0x7fd48c0d0d60 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd491600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c1f4280 con 0x7fd48c0d3380 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd48b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd48b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35770/0 (socket says 172.21.15.99:35770) 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd48b400700 1 -- 172.21.15.99:0/3461896981 learned_addr learned my addr 172.21.15.99:0/3461896981 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd48aa00700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c0d0d60 0x7fd48c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.883+0000 7fd48a000700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c0d3380 0x7fd48c0b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd48b400700 1 -- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c0d3380 msgr2=0x7fd48c0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd48b400700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c0d3380 0x7fd48c0b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd48b400700 1 -- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c0d0d60 msgr2=0x7fd48c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd48b400700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c0d0d60 0x7fd48c15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd48b400700 1 -- 172.21.15.99:0/3461896981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c0094a0 con 0x7fd48c160770 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd48b400700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b51a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd484002230 tx=0x7fd48400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd482a00700 1 -- 172.21.15.99:0/3461896981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48400aac0 con 0x7fd48c160770 2024-03-29T09:10:59.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd482a00700 1 -- 172.21.15.99:0/3461896981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48400ac40 con 0x7fd48c160770 2024-03-29T09:10:59.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd482a00700 1 -- 172.21.15.99:0/3461896981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840070d0 con 0x7fd48c160770 2024-03-29T09:10:59.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c1f4560 con 0x7fd48c160770 2024-03-29T09:10:59.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.884+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c1f64c0 con 0x7fd48c160770 2024-03-29T09:10:59.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.885+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c1f3e00 con 0x7fd48c160770 2024-03-29T09:10:59.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.886+0000 7fd482a00700 1 -- 172.21.15.99:0/3461896981 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd484007250 con 0x7fd48c160770 2024-03-29T09:10:59.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.887+0000 7fd482a00700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd46c06ac90 0x7fd46c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:10:59.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.887+0000 7fd482a00700 1 -- 172.21.15.99:0/3461896981 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fd4840890f0 con 0x7fd48c160770 2024-03-29T09:10:59.886 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.887+0000 7fd48aa00700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd46c06ac90 0x7fd46c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:10:59.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.888+0000 7fd48aa00700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd46c06ac90 0x7fd46c06d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0090b0 tx=0x7fd47c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:10:59.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:10:59.892+0000 7fd482a00700 1 -- 172.21.15.99:0/3461896981 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd48c1f3e00 con 0x7fd48c160770 2024-03-29T09:11:00.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.100+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd48c1f3e00 con 0x7fd48c160770 2024-03-29T09:11:00.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.101+0000 7fd482a00700 1 -- 172.21.15.99:0/3461896981 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd484018020 con 0x7fd48c160770 2024-03-29T09:11:00.100 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:00.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd46c06ac90 msgr2=0x7fd46c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd46c06ac90 0x7fd46c06d150 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0090b0 tx=0x7fd47c01c040 comp rx=0 tx=0).stop 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 msgr2=0x7fd48c0b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b51a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd484002230 tx=0x7fd48400cf50 comp rx=0 tx=0).stop 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 shutdown_connections 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd46c06ac90 0x7fd46c06d150 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd48c160770 0x7fd48c0b51a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd48c0d0d60 0x7fd48c15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 --2- 172.21.15.99:0/3461896981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd48c0d3380 0x7fd48c0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 >> 172.21.15.99:0/3461896981 conn(0x7fd48c08afd0 msgr2=0x7fd48c0b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 shutdown_connections 2024-03-29T09:11:00.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.103+0000 7fd491600700 1 -- 172.21.15.99:0/3461896981 wait complete. 2024-03-29T09:11:00.120 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:11:00.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.312+0000 7f48a7a00700 1 Processor -- start 2024-03-29T09:11:00.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.313+0000 7f48a7a00700 1 -- start start 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.314+0000 7f48a7a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a0167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.314+0000 7f48a7a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a00d1520 con 0x7f48a0160770 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.314+0000 7f48a7a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a00d1800 con 0x7f48a00b3cc0 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.314+0000 7f48a7a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00b4140 con 0x7f48a00cf090 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.314+0000 7f48a5000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f48a0160770 0x7f48a00d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38774/0 (socket says 172.21.15.99:38774) 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.314+0000 7f48a5000700 1 -- 172.21.15.99:0/1114775578 learned_addr learned my addr 172.21.15.99:0/1114775578 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.314+0000 7f48a5a00700 1 --2- 172.21.15.99:0/1114775578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a0167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454052224 0 0) 0x7f48a00d1520 con 0x7f48a0160770 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48880035b0 con 0x7f48a0160770 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818516579 0 0) 0x7f48a00d1800 con 0x7f48a00b3cc0 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a00d1520 con 0x7f48a00b3cc0 2024-03-29T09:11:00.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f48a5a00700 1 -- 172.21.15.99:0/1114775578 >> v1:172.21.15.133:6789/0 conn(0x7f48a00b3cc0 legacy=0x7f48a00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2608244045 0 0) 0x7f48880035b0 con 0x7f48a0160770 2024-03-29T09:11:00.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f48a5a00700 1 -- 172.21.15.99:0/1114775578 >> v1:172.21.15.99:6789/0 conn(0x7f48a0160770 legacy=0x7f48a00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f48a5a00700 1 -- 172.21.15.99:0/1114775578 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a00d1ae0 con 0x7f48a00cf090 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f48a5a00700 1 --2- 172.21.15.99:0/1114775578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a0167f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4898009dc0 tx=0x7f489800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1570c58967aecde4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.315+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800e070 con 0x7f48a00cf090 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898003a20 con 0x7f48a00cf090 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f489f400700 1 -- 172.21.15.99:0/1114775578 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894001670 con 0x7f48a00cf090 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f48a7a00700 1 -- 172.21.15.99:0/1114775578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 msgr2=0x7f48a0167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f48a7a00700 1 --2- 172.21.15.99:0/1114775578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a0167f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4898009dc0 tx=0x7f489800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f48a7a00700 1 -- 172.21.15.99:0/1114775578 shutdown_connections 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f48a7a00700 1 --2- 172.21.15.99:0/1114775578 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a0167f40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f48a7a00700 1 -- 172.21.15.99:0/1114775578 >> 172.21.15.99:0/1114775578 conn(0x7f48a008afd0 msgr2=0x7f48a009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:00.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f48a7a00700 1 -- 172.21.15.99:0/1114775578 shutdown_connections 2024-03-29T09:11:00.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.316+0000 7f48a7a00700 1 -- 172.21.15.99:0/1114775578 wait complete. 2024-03-29T09:11:00.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.318+0000 7f48a7a00700 1 Processor -- start 2024-03-29T09:11:00.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.318+0000 7f48a7a00700 1 -- start start 2024-03-29T09:11:00.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a7a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a00b3cc0 0x7f48a0155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a7a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a7a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a0160770 0x7f48a015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a7a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a01f38a0 con 0x7f48a00b3cc0 2024-03-29T09:11:00.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a7a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a015b7d0 con 0x7f48a0160770 2024-03-29T09:11:00.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a7a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a015bab0 con 0x7f48a00cf090 2024-03-29T09:11:00.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a00b3cc0 0x7f48a0155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a00b3cc0 0x7f48a0155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35772/0 (socket says 172.21.15.99:35772) 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a5000700 1 -- 172.21.15.99:0/502062725 learned_addr learned my addr 172.21.15.99:0/502062725 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f489fe00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.319+0000 7f48a5a00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a0160770 0x7f48a015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a5000700 1 -- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 msgr2=0x7f48a015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a5000700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a015ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a5000700 1 -- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a0160770 msgr2=0x7f48a015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a5000700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a0160770 0x7f48a015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a5000700 1 -- 172.21.15.99:0/502062725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898009a70 con 0x7f48a00b3cc0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a5000700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a00b3cc0 0x7f48a0155d70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f489400c4d0 tx=0x7f489400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f489cc00700 1 -- 172.21.15.99:0/502062725 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894004c80 con 0x7f48a00b3cc0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f489cc00700 1 -- 172.21.15.99:0/502062725 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4894004e00 con 0x7f48a00b3cc0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a015bd90 con 0x7f48a00b3cc0 2024-03-29T09:11:00.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f489cc00700 1 -- 172.21.15.99:0/502062725 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4894007310 con 0x7f48a00b3cc0 2024-03-29T09:11:00.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.320+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a01562b0 con 0x7f48a00b3cc0 2024-03-29T09:11:00.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.321+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a015c2b0 con 0x7f48a00b3cc0 2024-03-29T09:11:00.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.322+0000 7f489cc00700 1 -- 172.21.15.99:0/502062725 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4894007470 con 0x7f48a00b3cc0 2024-03-29T09:11:00.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.323+0000 7f489cc00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f488006ac90 0x7f488006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.323+0000 7f489cc00700 1 -- 172.21.15.99:0/502062725 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f48940893a0 con 0x7f48a00b3cc0 2024-03-29T09:11:00.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.323+0000 7f489fe00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f488006ac90 0x7f488006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.324+0000 7f489fe00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f488006ac90 0x7f488006d150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48a006c8e0 tx=0x7f488c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:00.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.327+0000 7f489cc00700 1 -- 172.21.15.99:0/502062725 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f48a015c2b0 con 0x7f48a00b3cc0 2024-03-29T09:11:00.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.532+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48a015c2b0 con 0x7f48a00b3cc0 2024-03-29T09:11:00.532 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.533+0000 7f489cc00700 1 -- 172.21.15.99:0/502062725 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48a015c2b0 con 0x7f48a00b3cc0 2024-03-29T09:11:00.532 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:11:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f488006ac90 msgr2=0x7f488006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f488006ac90 0x7f488006d150 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f48a006c8e0 tx=0x7f488c009040 comp rx=0 tx=0).stop 2024-03-29T09:11:00.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a00b3cc0 msgr2=0x7f48a0155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a00b3cc0 0x7f48a0155d70 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f489400c4d0 tx=0x7f489400c890 comp rx=0 tx=0).stop 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 shutdown_connections 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f488006ac90 0x7f488006d150 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a0160770 0x7f48a015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48a00b3cc0 0x7f48a0155d70 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 --2- 172.21.15.99:0/502062725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a00cf090 0x7f48a015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 >> 172.21.15.99:0/502062725 conn(0x7f48a008afd0 msgr2=0x7f48a006cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 shutdown_connections 2024-03-29T09:11:00.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.535+0000 7f48a7a00700 1 -- 172.21.15.99:0/502062725 wait complete. 2024-03-29T09:11:00.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-03-29T09:11:00.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.744+0000 7f6b1c200700 1 Processor -- start 2024-03-29T09:11:00.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.744+0000 7f6b1c200700 1 -- start start 2024-03-29T09:11:00.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b1c200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b140d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b1c200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6b140d3380 0x7f6b140ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b1c200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b140d3810 con 0x7f6b14160770 2024-03-29T09:11:00.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b1c200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b140b4140 con 0x7f6b140d3380 2024-03-29T09:11:00.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b1c200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140d1520 con 0x7f6b140b3cc0 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b19800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6b14160770 0x7f6b140d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38780/0 (socket says 172.21.15.99:38780) 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b19800700 1 -- 172.21.15.99:0/2399169688 learned_addr learned my addr 172.21.15.99:0/2399169688 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.745+0000 7f6b1a200700 1 --2- 172.21.15.99:0/2399169688 >> v2:172.21.15.99:3300/0 conn(0x7f6b140d3380 0x7f6b140ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b18e00700 1 --2- 172.21.15.99:0/2399169688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b140d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b13e00700 1 -- 172.21.15.99:0/2399169688 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294875275 0 0) 0x7f6b140d3810 con 0x7f6b14160770 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b13e00700 1 -- 172.21.15.99:0/2399169688 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6afc0035b0 con 0x7f6b14160770 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b1a200700 1 -- 172.21.15.99:0/2399169688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 msgr2=0x7f6b140d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b1a200700 1 --2- 172.21.15.99:0/2399169688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b140d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b1a200700 1 -- 172.21.15.99:0/2399169688 >> v1:172.21.15.99:6789/0 conn(0x7f6b14160770 legacy=0x7f6b140d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b1a200700 1 -- 172.21.15.99:0/2399169688 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b140d1800 con 0x7f6b140d3380 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b1a200700 1 --2- 172.21.15.99:0/2399169688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140ce080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009d90 tx=0x7f6b0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf5c82a647d7165 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b13e00700 1 -- 172.21.15.99:0/2399169688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00e070 con 0x7f6b140d3380 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b13e00700 1 -- 172.21.15.99:0/2399169688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0c00bb20 con 0x7f6b140d3380 2024-03-29T09:11:00.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.746+0000 7f6b13e00700 1 -- 172.21.15.99:0/2399169688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c012430 con 0x7f6b140d3380 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 -- 172.21.15.99:0/2399169688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 msgr2=0x7f6b140ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 --2- 172.21.15.99:0/2399169688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140ce080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009d90 tx=0x7f6b0c00b330 comp rx=0 tx=0).stop 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 -- 172.21.15.99:0/2399169688 shutdown_connections 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 --2- 172.21.15.99:0/2399169688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140ce080 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 --2- 172.21.15.99:0/2399169688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b140d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 -- 172.21.15.99:0/2399169688 >> 172.21.15.99:0/2399169688 conn(0x7f6b1408afd0 msgr2=0x7f6b1409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 -- 172.21.15.99:0/2399169688 shutdown_connections 2024-03-29T09:11:00.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.747+0000 7f6b1c200700 1 -- 172.21.15.99:0/2399169688 wait complete. 2024-03-29T09:11:00.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.749+0000 7f6b1c200700 1 Processor -- start 2024-03-29T09:11:00.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.749+0000 7f6b1c200700 1 -- start start 2024-03-29T09:11:00.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b1c200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b1415e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b1c200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b1c200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b14160770 0x7f6b140a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b1c200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140a0f50 con 0x7f6b140d3380 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b1c200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1415e8d0 con 0x7f6b14160770 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b1c200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b141e5b20 con 0x7f6b140b3cc0 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b18e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b18e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35790/0 (socket says 172.21.15.99:35790) 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b18e00700 1 -- 172.21.15.99:0/1696810155 learned_addr learned my addr 172.21.15.99:0/1696810155 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b19800700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b1415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.750+0000 7f6b1a200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b14160770 0x7f6b140a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b18e00700 1 -- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 msgr2=0x7f6b1415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b18e00700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b1415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b18e00700 1 -- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b14160770 msgr2=0x7f6b140a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b18e00700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b14160770 0x7f6b140a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b18e00700 1 -- 172.21.15.99:0/1696810155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c009a70 con 0x7f6b140d3380 2024-03-29T09:11:00.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b1a200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b14160770 0x7f6b140a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:00.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b18e00700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140a00e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000e9e0 tx=0x7f6b0000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:00.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b11600700 1 -- 172.21.15.99:0/1696810155 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00009710 con 0x7f6b140d3380 2024-03-29T09:11:00.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b11600700 1 -- 172.21.15.99:0/1696810155 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0000c850 con 0x7f6b140d3380 2024-03-29T09:11:00.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b11600700 1 -- 172.21.15.99:0/1696810155 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000cbc0 con 0x7f6b140d3380 2024-03-29T09:11:00.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b140cbad0 con 0x7f6b140d3380 2024-03-29T09:11:00.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.751+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b140cb480 con 0x7f6b140d3380 2024-03-29T09:11:00.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.752+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b140cb2e0 con 0x7f6b140d3380 2024-03-29T09:11:00.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.753+0000 7f6b11600700 1 -- 172.21.15.99:0/1696810155 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b0000cd20 con 0x7f6b140d3380 2024-03-29T09:11:00.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.755+0000 7f6b11600700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6af406ac90 0x7f6af406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:00.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.755+0000 7f6b11600700 1 -- 172.21.15.99:0/1696810155 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f6b00089a20 con 0x7f6b140d3380 2024-03-29T09:11:00.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.755+0000 7f6b19800700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6af406ac90 0x7f6af406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:00.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.755+0000 7f6b19800700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6af406ac90 0x7f6af406d150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b08007fd0 tx=0x7f6b08007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:00.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.759+0000 7f6b11600700 1 -- 172.21.15.99:0/1696810155 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6b140cb2e0 con 0x7f6b140d3380 2024-03-29T09:11:00.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.960+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f6b140cb2e0 con 0x7f6b140d3380 2024-03-29T09:11:00.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.961+0000 7f6b11600700 1 -- 172.21.15.99:0/1696810155 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7f6b140cb2e0 con 0x7f6b140d3380 2024-03-29T09:11:00.960 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:00.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6af406ac90 msgr2=0x7f6af406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6af406ac90 0x7f6af406d150 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b08007fd0 tx=0x7f6b08007dc0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 msgr2=0x7f6b140a00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140a00e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000e9e0 tx=0x7f6b0000ecf0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 shutdown_connections 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6af406ac90 0x7f6af406d150 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b14160770 0x7f6b140a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b140b3cc0 0x7f6b1415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 --2- 172.21.15.99:0/1696810155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b140d3380 0x7f6b140a00e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 >> 172.21.15.99:0/1696810155 conn(0x7f6b1408afd0 msgr2=0x7f6b140b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 shutdown_connections 2024-03-29T09:11:00.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:00.963+0000 7f6b1c200700 1 -- 172.21.15.99:0/1696810155 wait complete. 2024-03-29T09:11:00.981 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:11:01.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.173+0000 7fa833c00700 1 Processor -- start 2024-03-29T09:11:01.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.173+0000 7fa833c00700 1 -- start start 2024-03-29T09:11:01.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa833c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa82c1608e0 0x7fa82c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:01.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa833c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa82c160cc0 0x7fa82c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa833c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c0d3a70 0x7fa82c0d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa833c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0cddc0 con 0x7fa82c160cc0 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa833c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0ce0a0 con 0x7fa82c1608e0 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa833c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0ce380 con 0x7fa82c0d3a70 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa82c160cc0 0x7fa82c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa82c160cc0 0x7fa82c0ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35796/0 (socket says 172.21.15.99:35796) 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 -- 172.21.15.99:0/1676964739 learned_addr learned my addr 172.21.15.99:0/1676964739 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa831200700 1 --2- 172.21.15.99:0/1676964739 >> v2:172.21.15.133:3300/0 conn(0x7fa82c1608e0 0x7fa82c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa831c00700 1 --2- 172.21.15.99:0/1676964739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c0d3a70 0x7fa82c0d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 -- 172.21.15.99:0/1676964739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c0d3a70 msgr2=0x7fa82c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 --2- 172.21.15.99:0/1676964739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c0d3a70 0x7fa82c0d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 -- 172.21.15.99:0/1676964739 >> v2:172.21.15.133:3300/0 conn(0x7fa82c1608e0 msgr2=0x7fa82c0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 --2- 172.21.15.99:0/1676964739 >> v2:172.21.15.133:3300/0 conn(0x7fa82c1608e0 0x7fa82c0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa82be00700 1 -- 172.21.15.99:0/1676964739 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0ce660 con 0x7fa82c160cc0 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa831200700 1 --2- 172.21.15.99:0/1676964739 >> v2:172.21.15.133:3300/0 conn(0x7fa82c1608e0 0x7fa82c0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.174+0000 7fa831c00700 1 --2- 172.21.15.99:0/1676964739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c0d3a70 0x7fa82c0d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.175+0000 7fa82be00700 1 --2- 172.21.15.99:0/1676964739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c160cc0 0x7fa82c0ceb80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa82c08d790 tx=0x7fa818009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b5f3216e04dfa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.175+0000 7fa82b400700 1 -- 172.21.15.99:0/1676964739 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81800b870 con 0x7fa82c160cc0 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.175+0000 7fa82b400700 1 -- 172.21.15.99:0/1676964739 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81800b9d0 con 0x7fa82c160cc0 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.175+0000 7fa82b400700 1 -- 172.21.15.99:0/1676964739 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81800bd60 con 0x7fa82c160cc0 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.175+0000 7fa833c00700 1 -- 172.21.15.99:0/1676964739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c160cc0 msgr2=0x7fa82c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.175+0000 7fa833c00700 1 --2- 172.21.15.99:0/1676964739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c160cc0 0x7fa82c0ceb80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa82c08d790 tx=0x7fa818009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.176+0000 7fa833c00700 1 -- 172.21.15.99:0/1676964739 shutdown_connections 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.176+0000 7fa833c00700 1 --2- 172.21.15.99:0/1676964739 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c0d3a70 0x7fa82c0d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.176+0000 7fa833c00700 1 --2- 172.21.15.99:0/1676964739 >> v2:172.21.15.133:3300/0 conn(0x7fa82c1608e0 0x7fa82c0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.176+0000 7fa833c00700 1 --2- 172.21.15.99:0/1676964739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c160cc0 0x7fa82c0ceb80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.176+0000 7fa833c00700 1 -- 172.21.15.99:0/1676964739 >> 172.21.15.99:0/1676964739 conn(0x7fa82c09c490 msgr2=0x7fa82c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.176+0000 7fa833c00700 1 -- 172.21.15.99:0/1676964739 shutdown_connections 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.176+0000 7fa833c00700 1 -- 172.21.15.99:0/1676964739 wait complete. 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.177+0000 7fa833c00700 1 Processor -- start 2024-03-29T09:11:01.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.177+0000 7fa833c00700 1 -- start start 2024-03-29T09:11:01.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.178+0000 7fa833c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c0d3a70 0x7fa82c15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:01.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.178+0000 7fa833c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c1608e0 0x7fa82c1f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.178+0000 7fa833c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa82c160cc0 0x7fa82c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.178+0000 7fa833c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15ac00 con 0x7fa82c0d3a70 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.178+0000 7fa833c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1f3440 con 0x7fa82c160cc0 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.178+0000 7fa833c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1f3670 con 0x7fa82c1608e0 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c0d3a70 0x7fa82c15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c0d3a70 0x7fa82c15a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35812/0 (socket says 172.21.15.99:35812) 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 -- 172.21.15.99:0/1085077814 learned_addr learned my addr 172.21.15.99:0/1085077814 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa82c160cc0 0x7fa82c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa82be00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c1608e0 0x7fa82c1f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 -- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c1608e0 msgr2=0x7fa82c1f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:01.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c1608e0 0x7fa82c1f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 -- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa82c160cc0 msgr2=0x7fa82c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa82c160cc0 0x7fa82c153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 -- 172.21.15.99:0/1085077814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa818009550 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa831200700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c0d3a70 0x7fa82c15a6c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c750 tx=0x7fa82000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa828c00700 1 -- 172.21.15.99:0/1085077814 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820011800 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.179+0000 7fa828c00700 1 -- 172.21.15.99:0/1085077814 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa820011960 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.180+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c1f3950 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.180+0000 7fa828c00700 1 -- 172.21.15.99:0/1085077814 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820011cd0 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.180+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c1f6670 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.180+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c1f3e70 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.181+0000 7fa828c00700 1 -- 172.21.15.99:0/1085077814 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa820021350 con 0x7fa82c0d3a70 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.182+0000 7fa828c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa80c06ac90 0x7fa80c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:01.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.182+0000 7fa828c00700 1 -- 172.21.15.99:0/1085077814 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa820089450 con 0x7fa82c0d3a70 2024-03-29T09:11:01.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.183+0000 7fa82be00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa80c06ac90 0x7fa80c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:01.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.183+0000 7fa82be00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa80c06ac90 0x7fa80c06d150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa82c161130 tx=0x7fa818008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:01.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.187+0000 7fa828c00700 1 -- 172.21.15.99:0/1085077814 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa8200542c0 con 0x7fa82c0d3a70 2024-03-29T09:11:01.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.403+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa82c0d2300 con 0x7fa82c0d3a70 2024-03-29T09:11:01.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.404+0000 7fa828c00700 1 -- 172.21.15.99:0/1085077814 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa8200578e0 con 0x7fa82c0d3a70 2024-03-29T09:11:01.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa80c06ac90 msgr2=0x7fa80c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:01.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa80c06ac90 0x7fa80c06d150 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa82c161130 tx=0x7fa818008030 comp rx=0 tx=0).stop 2024-03-29T09:11:01.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c0d3a70 msgr2=0x7fa82c15a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:01.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c0d3a70 0x7fa82c15a6c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c750 tx=0x7fa82000ca60 comp rx=0 tx=0).stop 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 shutdown_connections 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa80c06ac90 0x7fa80c06d150 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa82c160cc0 0x7fa82c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa82c0d3a70 0x7fa82c15a6c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 --2- 172.21.15.99:0/1085077814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa82c1608e0 0x7fa82c1f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 >> 172.21.15.99:0/1085077814 conn(0x7fa82c09c490 msgr2=0x7fa82c15e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 shutdown_connections 2024-03-29T09:11:01.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:01.406+0000 7fa833c00700 1 -- 172.21.15.99:0/1085077814 wait complete. 2024-03-29T09:11:01.407 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:11:01.423 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:11:01.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:01.423 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:01.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[60135]: fuse finished with error 0 and tester_r 0 2024-03-29T09:11:01.488 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:11:07.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:11:07.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:07.591 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:11:07.638 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:11:07.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.833+0000 7fb025a00700 1 Processor -- start 2024-03-29T09:11:07.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.833+0000 7fb025a00700 1 -- start start 2024-03-29T09:11:07.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb025a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb02015ae70 0x7fb02015b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:07.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb025a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0200b0b50 0x7fb0200b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:07.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb025a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0200b14f0 con 0x7fb0200b36f0 2024-03-29T09:11:07.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb025a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02015c220 con 0x7fb02015ae70 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb025a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02015b8f0 con 0x7fb0200b0b50 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01ea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb02015ae70 0x7fb02015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01ea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb02015ae70 0x7fb02015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35822/0 (socket says 172.21.15.99:35822) 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01f400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb0200b36f0 0x7fb0200b3b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:38788/0 (socket says 172.21.15.99:38788) 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01ea00700 1 -- 172.21.15.99:0/1370575238 learned_addr learned my addr 172.21.15.99:0/1370575238 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01e000700 1 --2- 172.21.15.99:0/1370575238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0200b0b50 0x7fb0200b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01ea00700 1 -- 172.21.15.99:0/1370575238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0200b0b50 msgr2=0x7fb0200b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01ea00700 1 --2- 172.21.15.99:0/1370575238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0200b0b50 0x7fb0200b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01ea00700 1 -- 172.21.15.99:0/1370575238 >> v1:172.21.15.99:6789/0 conn(0x7fb0200b36f0 legacy=0x7fb0200b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01ea00700 1 -- 172.21.15.99:0/1370575238 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02015bbd0 con 0x7fb02015ae70 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.835+0000 7fb01d600700 1 -- 172.21.15.99:0/1370575238 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515439821 0 0) 0x7fb0200b14f0 con 0x7fb0200b36f0 2024-03-29T09:11:07.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb01ea00700 1 --2- 172.21.15.99:0/1370575238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02015ae70 0x7fb02015b250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb0100098f0 tx=0x7fb010009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e8ef4091c6f98c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:07.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb01d600700 1 -- 172.21.15.99:0/1370575238 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01000b8a0 con 0x7fb02015ae70 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb01d600700 1 -- 172.21.15.99:0/1370575238 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01000ba00 con 0x7fb02015ae70 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb01d600700 1 -- 172.21.15.99:0/1370575238 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01000bd90 con 0x7fb02015ae70 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb025a00700 1 -- 172.21.15.99:0/1370575238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02015ae70 msgr2=0x7fb02015b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb025a00700 1 --2- 172.21.15.99:0/1370575238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02015ae70 0x7fb02015b250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb0100098f0 tx=0x7fb010009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb025a00700 1 -- 172.21.15.99:0/1370575238 shutdown_connections 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.836+0000 7fb025a00700 1 --2- 172.21.15.99:0/1370575238 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb02015ae70 0x7fb02015b250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.837+0000 7fb025a00700 1 --2- 172.21.15.99:0/1370575238 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb0200b0b50 0x7fb0200b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.837+0000 7fb025a00700 1 -- 172.21.15.99:0/1370575238 >> 172.21.15.99:0/1370575238 conn(0x7fb020066540 msgr2=0x7fb020066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.837+0000 7fb025a00700 1 -- 172.21.15.99:0/1370575238 shutdown_connections 2024-03-29T09:11:07.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.837+0000 7fb025a00700 1 -- 172.21.15.99:0/1370575238 wait complete. 2024-03-29T09:11:07.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.838+0000 7fb025a00700 1 Processor -- start 2024-03-29T09:11:07.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.838+0000 7fb025a00700 1 -- start start 2024-03-29T09:11:07.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.839+0000 7fb025a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0200b0b50 0x7fb0201527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:07.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.839+0000 7fb025a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0200b36f0 0x7fb0200ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.839+0000 7fb025a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb02015ae70 0x7fb020156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.839+0000 7fb025a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0200ae280 con 0x7fb0200b36f0 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.839+0000 7fb025a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0200ad1e0 con 0x7fb0200b0b50 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.839+0000 7fb025a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0200ad4c0 con 0x7fb02015ae70 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.839+0000 7fb01e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0200b36f0 0x7fb0200ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0200b36f0 0x7fb0200ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35836/0 (socket says 172.21.15.99:35836) 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 -- 172.21.15.99:0/2626969023 learned_addr learned my addr 172.21.15.99:0/2626969023 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01f400700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb02015ae70 0x7fb020156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:07.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01ea00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0200b0b50 0x7fb0201527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 -- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb02015ae70 msgr2=0x7fb020156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb02015ae70 0x7fb020156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 -- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0200b0b50 msgr2=0x7fb0201527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0200b0b50 0x7fb0201527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 -- 172.21.15.99:0/2626969023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010009550 con 0x7fb0200b36f0 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb01e000700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0200b36f0 0x7fb0200ac8b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e810 tx=0x7fb00c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.840+0000 7fb016a00700 1 -- 172.21.15.99:0/2626969023 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00c070 con 0x7fb0200b36f0 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.841+0000 7fb016a00700 1 -- 172.21.15.99:0/2626969023 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c013610 con 0x7fb0200b36f0 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.841+0000 7fb016a00700 1 -- 172.21.15.99:0/2626969023 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0139c0 con 0x7fb0200b36f0 2024-03-29T09:11:07.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.841+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0200ad7a0 con 0x7fb0200b36f0 2024-03-29T09:11:07.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.841+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0200add60 con 0x7fb0200b36f0 2024-03-29T09:11:07.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.842+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0200677a0 con 0x7fb0200b36f0 2024-03-29T09:11:07.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.843+0000 7fb016a00700 1 -- 172.21.15.99:0/2626969023 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb00c013b20 con 0x7fb0200b36f0 2024-03-29T09:11:07.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.843+0000 7fb016a00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb00006ac90 0x7fb00006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:07.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.844+0000 7fb016a00700 1 -- 172.21.15.99:0/2626969023 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb00c089420 con 0x7fb0200b36f0 2024-03-29T09:11:07.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.844+0000 7fb01ea00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb00006ac90 0x7fb00006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:07.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.844+0000 7fb01ea00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb00006ac90 0x7fb00006d150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb010014040 tx=0x7fb010008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:07.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:07.848+0000 7fb016a00700 1 -- 172.21.15.99:0/2626969023 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb00c0542a0 con 0x7fb0200b36f0 2024-03-29T09:11:08.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.100+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb0200b1a90 con 0x7fb0200b36f0 2024-03-29T09:11:08.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.100+0000 7fb016a00700 1 -- 172.21.15.99:0/2626969023 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+82 (secure 0 0 0) 0x7fb00c0578c0 con 0x7fb0200b36f0 2024-03-29T09:11:08.099 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-03-29T09:11:08.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb00006ac90 msgr2=0x7fb00006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:08.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb00006ac90 0x7fb00006d150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb010014040 tx=0x7fb010008030 comp rx=0 tx=0).stop 2024-03-29T09:11:08.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0200b36f0 msgr2=0x7fb0200ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:08.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0200b36f0 0x7fb0200ac8b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00e810 tx=0x7fb00c00eb20 comp rx=0 tx=0).stop 2024-03-29T09:11:08.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 shutdown_connections 2024-03-29T09:11:08.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb00006ac90 0x7fb00006d150 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:08.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb02015ae70 0x7fb020156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:08.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0200b0b50 0x7fb0201527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:08.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 --2- 172.21.15.99:0/2626969023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb0200b36f0 0x7fb0200ac8b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:08.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 >> 172.21.15.99:0/2626969023 conn(0x7fb020066540 msgr2=0x7fb0200cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:08.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 shutdown_connections 2024-03-29T09:11:08.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:08.103+0000 7fb025a00700 1 -- 172.21.15.99:0/2626969023 wait complete. 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:11:08.120 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:08.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:08.121 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:08.163 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42120sec preferred_lft 42120sec 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:11:08.164 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::3caa:acff:feac:fccd/64 scope link 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5832:eeff:fe0b:ee92/64 scope link 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether fe:d0:b6:f6:cb:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::fcd0:b6ff:fef6:cba2/64 scope link 2024-03-29T09:11:08.165 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:08.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:08.166 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:11:08.266 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:11:08.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:08.269 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:11:08.313 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:11:08.313 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:11:08.313 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:11:08.313 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:11:08.313 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:11:08.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:08.314 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:11:08.314 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:11:08.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:11:08.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:11:08.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:11:08.314 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:11:08.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:08.515 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:11:08.556 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:11:08.556 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T09:11:08.557 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:08.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:08.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:11:08.663 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T09:11:08.663 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:11:08.663 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:08.705 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:08.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:11:08.705 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:08.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:11:08.891 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:11:08.891 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:11:08.954 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:11:08.956 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:11:08.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:08.957 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:11:08.999 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:11:08.999 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T09:11:09.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:11:09.108 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:11:09.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:11:09.172 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:11:09.175 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:11:09.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:09.176 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:11:09.206 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-29T09:11:09.206+0000 7f951736a5c0 -1 init, newargv = 0x558ac8bc58e0 newargc=15 2024-03-29T09:11:09.206 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[61557]: starting ceph client 2024-03-29T09:11:09.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[61557]: starting fuse 2024-03-29T09:11:09.220 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:11:09.220 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:11:09.220 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:11:09.220 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:11:09.220 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:11:09.220 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:11:09.220 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:11:09.221 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:11:09.222 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:11:09.222 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:11:09.222 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:11:09.222 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:11:09.222 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:09.222 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-29T09:11:09.222 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:11:09.398 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.61557.asok 2024-03-29T09:11:09.577 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.61557.asok 2024-03-29T09:11:09.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:09.577 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.61557.asok status 2024-03-29T09:11:09.795 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:11:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:11:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:11:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:11:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-29T09:11:09.796 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:11:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T09:11:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "61557", 2024-03-29T09:11:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:11:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:11:09.797 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:11:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5754, 2024-03-29T09:11:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:11:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:11:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:11:09.798 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5754 2024-03-29T09:11:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:11:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:11:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:11:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4082152046 2024-03-29T09:11:09.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 4082152046 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5754 192.168.0.1:0/4082152046", 2024-03-29T09:11:09.800 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/4082152046", 2024-03-29T09:11:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:11:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 17, 2024-03-29T09:11:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 42, 2024-03-29T09:11:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:11:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:11:09.801 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:11:09.801 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:11:09.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:09.810 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:11:09.810 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:09.852 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:11:09.853 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:09.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:09.853 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:09.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:09.957 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-03-29T09:11:10.000 INFO:teuthology.orchestra.run.smithi099.stderr:setfattr: layoutfile: Permission denied 2024-03-29T09:11:10.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:11:10.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:10.001 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-03-29T09:11:10.084 INFO:teuthology.orchestra.run.smithi099.stderr:setfattr: layoutdir: Permission denied 2024-03-29T09:11:10.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:11:10.085 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:10.086 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:10.167 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:10.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.396+0000 7f56b5800700 1 Processor -- start 2024-03-29T09:11:10.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.396+0000 7f56b5800700 1 -- start start 2024-03-29T09:11:10.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.397+0000 7f56b5800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56b015ae70 0x7f56b015b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.397+0000 7f56b5800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.397+0000 7f56b5800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b00b14f0 con 0x7f56b00b0b50 2024-03-29T09:11:10.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.397+0000 7f56b5800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b015c220 con 0x7f56b015ae70 2024-03-29T09:11:10.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.397+0000 7f56b5800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b015b8f0 con 0x7f56b00b3e00 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.398+0000 7f56af400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.398+0000 7f56aea00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56b015ae70 0x7f56b015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.398+0000 7f56af400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:53544/0 (socket says 172.21.15.99:53544) 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.398+0000 7f56aea00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56b015ae70 0x7f56b015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:36348/0 (socket says 172.21.15.99:36348) 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.398+0000 7f56af400700 1 -- 172.21.15.99:0/3712193912 learned_addr learned my addr 172.21.15.99:0/3712193912 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56af400700 1 -- 172.21.15.99:0/3712193912 >> v2:172.21.15.133:3300/0 conn(0x7f56b015ae70 msgr2=0x7f56b015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56af400700 1 --2- 172.21.15.99:0/3712193912 >> v2:172.21.15.133:3300/0 conn(0x7f56b015ae70 0x7f56b015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56af400700 1 -- 172.21.15.99:0/3712193912 >> v1:172.21.15.133:6789/0 conn(0x7f56b00b0b50 legacy=0x7f56b00b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56af400700 1 -- 172.21.15.99:0/3712193912 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b015bbd0 con 0x7f56b00b3e00 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56af400700 1 --2- 172.21.15.99:0/3712193912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00b0fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009e50 tx=0x7f56a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127e8dfb1b3cd02d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56ad600700 1 -- 172.21.15.99:0/3712193912 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a800e070 con 0x7f56b00b3e00 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56ad600700 1 -- 172.21.15.99:0/3712193912 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a800b910 con 0x7f56b00b3e00 2024-03-29T09:11:10.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.399+0000 7f56ad600700 1 -- 172.21.15.99:0/3712193912 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a800bcc0 con 0x7f56b00b3e00 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 -- 172.21.15.99:0/3712193912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 msgr2=0x7f56b00b0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 --2- 172.21.15.99:0/3712193912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00b0fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009e50 tx=0x7f56a800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 -- 172.21.15.99:0/3712193912 shutdown_connections 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 --2- 172.21.15.99:0/3712193912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00b0fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 --2- 172.21.15.99:0/3712193912 >> v2:172.21.15.133:3300/0 conn(0x7f56b015ae70 0x7f56b015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 -- 172.21.15.99:0/3712193912 >> 172.21.15.99:0/3712193912 conn(0x7f56b0066540 msgr2=0x7f56b0066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 -- 172.21.15.99:0/3712193912 shutdown_connections 2024-03-29T09:11:10.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.400+0000 7f56b5800700 1 -- 172.21.15.99:0/3712193912 wait complete. 2024-03-29T09:11:10.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.402+0000 7f56b5800700 1 Processor -- start 2024-03-29T09:11:10.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.402+0000 7f56b5800700 1 -- start start 2024-03-29T09:11:10.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56b5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56b00b0b50 0x7f56b00ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56b5800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56b5800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56b015ae70 0x7f56b01f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56b5800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0071650 con 0x7f56b00b0b50 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56b5800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b00ade80 con 0x7f56b015ae70 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56b5800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0150670 con 0x7f56b00b3e00 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56aea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56b00b0b50 0x7f56b00ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56aea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56b00b0b50 0x7f56b00ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35488/0 (socket says 172.21.15.99:35488) 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.403+0000 7f56aea00700 1 -- 172.21.15.99:0/1370016343 learned_addr learned my addr 172.21.15.99:0/1370016343 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56af400700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56b015ae70 0x7f56b01f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56ae000700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56aea00700 1 -- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 msgr2=0x7f56b00ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56aea00700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56aea00700 1 -- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56b015ae70 msgr2=0x7f56b01f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56aea00700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56b015ae70 0x7f56b01f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56aea00700 1 -- 172.21.15.99:0/1370016343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a8009b30 con 0x7f56b00b0b50 2024-03-29T09:11:10.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56aea00700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56b00b0b50 0x7f56b00ad2c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f56a000e870 tx=0x7f56a000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:10.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56a6a00700 1 -- 172.21.15.99:0/1370016343 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0009590 con 0x7f56b00b0b50 2024-03-29T09:11:10.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56a6a00700 1 -- 172.21.15.99:0/1370016343 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a0013680 con 0x7f56b00b0b50 2024-03-29T09:11:10.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b0150950 con 0x7f56b00b0b50 2024-03-29T09:11:10.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.404+0000 7f56a6a00700 1 -- 172.21.15.99:0/1370016343 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00139f0 con 0x7f56b00b0b50 2024-03-29T09:11:10.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.405+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b00ae160 con 0x7f56b00b0b50 2024-03-29T09:11:10.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.406+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b00677a0 con 0x7f56b00b0b50 2024-03-29T09:11:10.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.406+0000 7f56a6a00700 1 -- 172.21.15.99:0/1370016343 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f56a0013b50 con 0x7f56b00b0b50 2024-03-29T09:11:10.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.407+0000 7f56a6a00700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f569006ac90 0x7f569006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.407+0000 7f56a6a00700 1 -- 172.21.15.99:0/1370016343 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f56a0089460 con 0x7f56b00b0b50 2024-03-29T09:11:10.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.408+0000 7f56ae000700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f569006ac90 0x7f569006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.408+0000 7f56ae000700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f569006ac90 0x7f569006d150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f569c007fd0 tx=0x7f569c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:10.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.412+0000 7f56a6a00700 1 -- 172.21.15.99:0/1370016343 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f56a0054360 con 0x7f56b00b0b50 2024-03-29T09:11:10.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.619+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56b00b1a90 con 0x7f56b00b0b50 2024-03-29T09:11:10.620 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.620+0000 7f56a6a00700 1 -- 172.21.15.99:0/1370016343 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7f56a0057980 con 0x7f56b00b0b50 2024-03-29T09:11:10.621 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:10.621 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:10.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f569006ac90 msgr2=0x7f569006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f569006ac90 0x7f569006d150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f569c007fd0 tx=0x7f569c009040 comp rx=0 tx=0).stop 2024-03-29T09:11:10.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56b00b0b50 msgr2=0x7f56b00ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56b00b0b50 0x7f56b00ad2c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f56a000e870 tx=0x7f56a000eb80 comp rx=0 tx=0).stop 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 shutdown_connections 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f569006ac90 0x7f569006d150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56b015ae70 0x7f56b01f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56b00b0b50 0x7f56b00ad2c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 --2- 172.21.15.99:0/1370016343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56b00b3e00 0x7f56b00ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 >> 172.21.15.99:0/1370016343 conn(0x7f56b0066540 msgr2=0x7f56b015ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 shutdown_connections 2024-03-29T09:11:10.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.623+0000 7f56b5800700 1 -- 172.21.15.99:0/1370016343 wait complete. 2024-03-29T09:11:10.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:10:42.037884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:10:42.900862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-03-29T09:10:53.257132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:11:10.642 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:11:10.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.833+0000 7f7c81400700 1 Processor -- start 2024-03-29T09:11:10.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.833+0000 7f7c81400700 1 -- start start 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.834+0000 7f7c81400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c7c160770 0x7f7c7c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c81400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c81400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c7c0d1520 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c81400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7c0d1800 con 0x7f7c7c160770 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c81400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0d1ae0 con 0x7f7c7c0d3550 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c7b200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c7a800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c7c160770 0x7f7c7c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c7a800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c7c160770 0x7f7c7c0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:36376/0 (socket says 172.21.15.99:36376) 2024-03-29T09:11:10.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c7a800700 1 -- 172.21.15.99:0/2232262780 learned_addr learned my addr 172.21.15.99:0/2232262780 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:10.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.835+0000 7f7c7b200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:53554/0 (socket says 172.21.15.99:53554) 2024-03-29T09:11:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c7b200700 1 -- 172.21.15.99:0/2232262780 >> v2:172.21.15.133:3300/0 conn(0x7f7c7c160770 msgr2=0x7f7c7c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c7b200700 1 --2- 172.21.15.99:0/2232262780 >> v2:172.21.15.133:3300/0 conn(0x7f7c7c160770 0x7f7c7c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c7b200700 1 -- 172.21.15.99:0/2232262780 >> v1:172.21.15.133:6789/0 conn(0x7f7c7c0d0d60 legacy=0x7f7c7c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c7b200700 1 -- 172.21.15.99:0/2232262780 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c0d1dc0 con 0x7f7c7c0d3550 2024-03-29T09:11:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c7a800700 1 --2- 172.21.15.99:0/2232262780 >> v2:172.21.15.133:3300/0 conn(0x7f7c7c160770 0x7f7c7c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:10.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c7b200700 1 --2- 172.21.15.99:0/2232262780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ce080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009d90 tx=0x7f7c7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1510ffe249db363c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c79e00700 1 -- 172.21.15.99:0/2232262780 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400e070 con 0x7f7c7c0d3550 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c79e00700 1 -- 172.21.15.99:0/2232262780 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7400bb20 con 0x7f7c7c0d3550 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.836+0000 7f7c79e00700 1 -- 172.21.15.99:0/2232262780 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74012430 con 0x7f7c7c0d3550 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 -- 172.21.15.99:0/2232262780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 msgr2=0x7f7c7c0ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 --2- 172.21.15.99:0/2232262780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ce080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009d90 tx=0x7f7c7400b330 comp rx=0 tx=0).stop 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 -- 172.21.15.99:0/2232262780 shutdown_connections 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 --2- 172.21.15.99:0/2232262780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ce080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 --2- 172.21.15.99:0/2232262780 >> v2:172.21.15.133:3300/0 conn(0x7f7c7c160770 0x7f7c7c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 -- 172.21.15.99:0/2232262780 >> 172.21.15.99:0/2232262780 conn(0x7f7c7c08afd0 msgr2=0x7f7c7c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 -- 172.21.15.99:0/2232262780 shutdown_connections 2024-03-29T09:11:10.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.837+0000 7f7c81400700 1 -- 172.21.15.99:0/2232262780 wait complete. 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.838+0000 7f7c81400700 1 Processor -- start 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.839+0000 7f7c81400700 1 -- start start 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.839+0000 7f7c81400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c7c0d0d60 0x7f7c7c0cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.839+0000 7f7c81400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.839+0000 7f7c81400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c7c160770 0x7f7c7c0b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.839+0000 7f7c81400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0cd520 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.839+0000 7f7c81400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0cc3c0 con 0x7f7c7c160770 2024-03-29T09:11:10.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.839+0000 7f7c81400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7c0a0e90 con 0x7f7c7c0d3550 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c7c0d0d60 0x7f7c7c0cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c7c0d0d60 0x7f7c7c0cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35506/0 (socket says 172.21.15.99:35506) 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 -- 172.21.15.99:0/3949361186 learned_addr learned my addr 172.21.15.99:0/3949361186 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7b200700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c7c160770 0x7f7c7c0b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c73e00700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 -- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 msgr2=0x7f7c7c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 -- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c7c160770 msgr2=0x7f7c7c0b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c7c160770 0x7f7c7c0b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c7a800700 1 -- 172.21.15.99:0/3949361186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c009550 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.840+0000 7f7c73e00700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:10.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.841+0000 7f7c7a800700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c7c0d0d60 0x7f7c7c0cc850 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00e9b0 tx=0x7f7c6c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.841+0000 7f7c72a00700 1 -- 172.21.15.99:0/3949361186 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00c7f0 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.841+0000 7f7c72a00700 1 -- 172.21.15.99:0/3949361186 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c00c950 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.841+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c74009a70 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.841+0000 7f7c72a00700 1 -- 172.21.15.99:0/3949361186 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c00ccc0 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.841+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c7c0cba90 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.842+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7c0678b0 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.843+0000 7f7c72a00700 1 -- 172.21.15.99:0/3949361186 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7c6c013470 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.843+0000 7f7c72a00700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c5806ac90 0x7f7c5806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.844+0000 7f7c72a00700 1 -- 172.21.15.99:0/3949361186 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f7c6c089ad0 con 0x7f7c7c0d0d60 2024-03-29T09:11:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.844+0000 7f7c73e00700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c5806ac90 0x7f7c5806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:10.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.844+0000 7f7c73e00700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c5806ac90 0x7f7c5806d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0c9b00 tx=0x7f7c64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:10.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:10.848+0000 7f7c72a00700 1 -- 172.21.15.99:0/3949361186 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7c6c054940 con 0x7f7c7c0d0d60 2024-03-29T09:11:11.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.051+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c7c073640 con 0x7f7c7c0d0d60 2024-03-29T09:11:11.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.051+0000 7f7c72a00700 1 -- 172.21.15.99:0/3949361186 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c6c0170a0 con 0x7f7c7c0d0d60 2024-03-29T09:11:11.051 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c5806ac90 msgr2=0x7f7c5806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c5806ac90 0x7f7c5806d150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0c9b00 tx=0x7f7c64009040 comp rx=0 tx=0).stop 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c7c0d0d60 msgr2=0x7f7c7c0cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c7c0d0d60 0x7f7c7c0cc850 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00e9b0 tx=0x7f7c6c00ed70 comp rx=0 tx=0).stop 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 shutdown_connections 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c5806ac90 0x7f7c5806d150 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c7c160770 0x7f7c7c0b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c7c0d0d60 0x7f7c7c0cc850 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 --2- 172.21.15.99:0/3949361186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c7c0d3550 0x7f7c7c0ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 >> 172.21.15.99:0/3949361186 conn(0x7f7c7c08afd0 msgr2=0x7f7c7c0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:11.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 shutdown_connections 2024-03-29T09:11:11.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.054+0000 7f7c81400700 1 -- 172.21.15.99:0/3949361186 wait complete. 2024-03-29T09:11:11.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:11:11.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.265+0000 7fa01e400700 1 Processor -- start 2024-03-29T09:11:11.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.265+0000 7fa01e400700 1 -- start start 2024-03-29T09:11:11.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa01e400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa01e400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa0180d3380 0x7fa0180ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa01e400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0180d3810 con 0x7fa018160770 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa01e400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0180b4140 con 0x7fa0180d3380 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa01e400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180d1520 con 0x7fa0180b3cc0 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa017400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa018160770 0x7fa0180d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39800/0 (socket says 172.21.15.99:39800) 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa017400700 1 -- 172.21.15.99:0/1104861751 learned_addr learned my addr 172.21.15.99:0/1104861751 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa017e00700 1 --2- 172.21.15.99:0/1104861751 >> v2:172.21.15.133:3300/0 conn(0x7fa0180d3380 0x7fa0180ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa016a00700 1 --2- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa016000700 1 -- 172.21.15.99:0/1104861751 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233876976 0 0) 0x7fa0180d3810 con 0x7fa018160770 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.266+0000 7fa016000700 1 -- 172.21.15.99:0/1104861751 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ffc0035b0 con 0x7fa018160770 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa017e00700 1 -- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 msgr2=0x7fa0180d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa016000700 1 -- 172.21.15.99:0/1104861751 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809124049 0 0) 0x7f9ffc0035b0 con 0x7fa018160770 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa017e00700 1 --2- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa017e00700 1 -- 172.21.15.99:0/1104861751 >> v1:172.21.15.99:6789/0 conn(0x7fa018160770 legacy=0x7fa0180d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa017e00700 1 -- 172.21.15.99:0/1104861751 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180d1800 con 0x7fa0180d3380 2024-03-29T09:11:11.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa016a00700 1 --2- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa017e00700 1 --2- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 0x7fa0180ce080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa010009d90 tx=0x7fa01000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f784b6c4776e03 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa016000700 1 -- 172.21.15.99:0/1104861751 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000e070 con 0x7fa0180d3380 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa016000700 1 -- 172.21.15.99:0/1104861751 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010003a20 con 0x7fa0180d3380 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.267+0000 7fa016000700 1 -- 172.21.15.99:0/1104861751 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008001670 con 0x7fa0180d3380 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 -- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 msgr2=0x7fa0180ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 --2- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 0x7fa0180ce080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa010009d90 tx=0x7fa01000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 -- 172.21.15.99:0/1104861751 shutdown_connections 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 --2- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 0x7fa0180ce080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 --2- 172.21.15.99:0/1104861751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 -- 172.21.15.99:0/1104861751 >> 172.21.15.99:0/1104861751 conn(0x7fa01808afd0 msgr2=0x7fa01809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:11.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 -- 172.21.15.99:0/1104861751 shutdown_connections 2024-03-29T09:11:11.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.268+0000 7fa01e400700 1 -- 172.21.15.99:0/1104861751 wait complete. 2024-03-29T09:11:11.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.270+0000 7fa01e400700 1 Processor -- start 2024-03-29T09:11:11.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.270+0000 7fa01e400700 1 -- start start 2024-03-29T09:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa01e400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa01e400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 0x7fa0180b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa01e400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018160770 0x7fa0180cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa01e400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180b6820 con 0x7fa018160770 2024-03-29T09:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa01e400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180b8990 con 0x7fa0180d3380 2024-03-29T09:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa01e400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180b5e80 con 0x7fa0180b3cc0 2024-03-29T09:11:11.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa017e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018160770 0x7fa0180cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa017e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018160770 0x7fa0180cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35512/0 (socket says 172.21.15.99:35512) 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa017e00700 1 -- 172.21.15.99:0/1694945187 learned_addr learned my addr 172.21.15.99:0/1694945187 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa016a00700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 0x7fa0180b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa017400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa017e00700 1 -- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 msgr2=0x7fa0180b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.271+0000 7fa017e00700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa017e00700 1 -- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 msgr2=0x7fa0180b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa017e00700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 0x7fa0180b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa017e00700 1 -- 172.21.15.99:0/1694945187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040094a0 con 0x7fa018160770 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa017400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa017e00700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018160770 0x7fa0180cf7f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa01000b680 tx=0x7fa010004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa00f400700 1 -- 172.21.15.99:0/1694945187 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100044c0 con 0x7fa018160770 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa00f400700 1 -- 172.21.15.99:0/1694945187 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010004620 con 0x7fa018160770 2024-03-29T09:11:11.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa00f400700 1 -- 172.21.15.99:0/1694945187 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010012430 con 0x7fa018160770 2024-03-29T09:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa010009a70 con 0x7fa018160770 2024-03-29T09:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.272+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0180b6440 con 0x7fa018160770 2024-03-29T09:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.273+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0180b6600 con 0x7fa018160770 2024-03-29T09:11:11.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.274+0000 7fa00f400700 1 -- 172.21.15.99:0/1694945187 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa010012590 con 0x7fa018160770 2024-03-29T09:11:11.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.275+0000 7fa00f400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ff806ac90 0x7f9ff806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.275+0000 7fa00f400700 1 -- 172.21.15.99:0/1694945187 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa01008b7f0 con 0x7fa018160770 2024-03-29T09:11:11.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.275+0000 7fa017400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ff806ac90 0x7f9ff806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.276+0000 7fa017400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ff806ac90 0x7f9ff806d150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa0080011e0 tx=0x7fa008007ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:11.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.279+0000 7fa00f400700 1 -- 172.21.15.99:0/1694945187 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa010056670 con 0x7fa018160770 2024-03-29T09:11:11.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.484+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0180b6960 con 0x7fa018160770 2024-03-29T09:11:11.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.485+0000 7fa00f400700 1 -- 172.21.15.99:0/1694945187 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0180b6960 con 0x7fa018160770 2024-03-29T09:11:11.485 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ff806ac90 msgr2=0x7f9ff806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ff806ac90 0x7f9ff806d150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa0080011e0 tx=0x7fa008007ac0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018160770 msgr2=0x7fa0180cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018160770 0x7fa0180cf7f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa01000b680 tx=0x7fa010004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 shutdown_connections 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9ff806ac90 0x7f9ff806d150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa018160770 0x7fa0180cf7f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0180b3cc0 0x7fa0180b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 --2- 172.21.15.99:0/1694945187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0180d3380 0x7fa0180b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 >> 172.21.15.99:0/1694945187 conn(0x7fa01808afd0 msgr2=0x7fa01815f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 shutdown_connections 2024-03-29T09:11:11.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.487+0000 7fa01e400700 1 -- 172.21.15.99:0/1694945187 wait complete. 2024-03-29T09:11:11.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:11:11.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.699+0000 7f6527000700 1 Processor -- start 2024-03-29T09:11:11.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.699+0000 7f6527000700 1 -- start start 2024-03-29T09:11:11.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.700+0000 7f6527000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f65200cf090 0x7f6520167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.700+0000 7f6527000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65200d1520 con 0x7f65200b3cc0 2024-03-29T09:11:11.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.700+0000 7f6527000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65200d1800 con 0x7f6520160770 2024-03-29T09:11:11.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.700+0000 7f6527000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65200b4140 con 0x7f65200cf090 2024-03-29T09:11:11.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f6525000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f65200cf090 0x7f6520167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f651f400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f65200b3cc0 0x7f65200d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39810/0 (socket says 172.21.15.99:39810) 2024-03-29T09:11:11.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f651f400700 1 -- 172.21.15.99:0/1031101877 learned_addr learned my addr 172.21.15.99:0/1031101877 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:11.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f651ea00700 1 -- 172.21.15.99:0/1031101877 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934993342 0 0) 0x7f65200d1520 con 0x7f65200b3cc0 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f651ea00700 1 -- 172.21.15.99:0/1031101877 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65040035b0 con 0x7f65200b3cc0 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f6525000700 1 -- 172.21.15.99:0/1031101877 >> v1:172.21.15.133:6789/0 conn(0x7f6520160770 legacy=0x7f65200d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f6525000700 1 -- 172.21.15.99:0/1031101877 >> v1:172.21.15.99:6789/0 conn(0x7f65200b3cc0 legacy=0x7f65200d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.701+0000 7f6525000700 1 -- 172.21.15.99:0/1031101877 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200d1ae0 con 0x7f65200cf090 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f6525000700 1 --2- 172.21.15.99:0/1031101877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65200cf090 0x7f6520167f40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6518009d90 tx=0x7f651800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293a3e1cb89f3753 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f651ea00700 1 -- 172.21.15.99:0/1031101877 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800e070 con 0x7f65200cf090 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f651ea00700 1 -- 172.21.15.99:0/1031101877 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651800bb20 con 0x7f65200cf090 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f651ea00700 1 -- 172.21.15.99:0/1031101877 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518012430 con 0x7f65200cf090 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f6527000700 1 -- 172.21.15.99:0/1031101877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65200cf090 msgr2=0x7f6520167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f6527000700 1 --2- 172.21.15.99:0/1031101877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65200cf090 0x7f6520167f40 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6518009d90 tx=0x7f651800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f6527000700 1 -- 172.21.15.99:0/1031101877 shutdown_connections 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f6527000700 1 --2- 172.21.15.99:0/1031101877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65200cf090 0x7f6520167f40 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f6527000700 1 -- 172.21.15.99:0/1031101877 >> 172.21.15.99:0/1031101877 conn(0x7f652008afd0 msgr2=0x7f652009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:11.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.702+0000 7f6527000700 1 -- 172.21.15.99:0/1031101877 shutdown_connections 2024-03-29T09:11:11.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.703+0000 7f6527000700 1 -- 172.21.15.99:0/1031101877 wait complete. 2024-03-29T09:11:11.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.705+0000 7f6527000700 1 Processor -- start 2024-03-29T09:11:11.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.705+0000 7f6527000700 1 -- start start 2024-03-29T09:11:11.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6527000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65200b3cc0 0x7f65201f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6527000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65200cf090 0x7f65201f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6527000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6520160770 0x7f65201593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6527000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6520159930 con 0x7f6520160770 2024-03-29T09:11:11.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6527000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cc370 con 0x7f65200cf090 2024-03-29T09:11:11.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6527000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652015f630 con 0x7f65200b3cc0 2024-03-29T09:11:11.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6525000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6520160770 0x7f65201593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6525000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6520160770 0x7f65201593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35522/0 (socket says 172.21.15.99:35522) 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.706+0000 7f6525000700 1 -- 172.21.15.99:0/3071836427 learned_addr learned my addr 172.21.15.99:0/3071836427 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f651f400700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65200cf090 0x7f65201f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6525000700 1 -- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65200b3cc0 msgr2=0x7f65201f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f651fe00700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65200b3cc0 0x7f65201f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6525000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65200b3cc0 0x7f65201f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6525000700 1 -- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65200cf090 msgr2=0x7f65201f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6525000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65200cf090 0x7f65201f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6525000700 1 -- 172.21.15.99:0/3071836427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6518009a70 con 0x7f6520160770 2024-03-29T09:11:11.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f651f400700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65200cf090 0x7f65201f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:11.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f651fe00700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65200b3cc0 0x7f65201f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:11.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6525000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6520160770 0x7f65201593f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6518013000 tx=0x7f6518004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:11.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6517e00700 1 -- 172.21.15.99:0/3071836427 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518012430 con 0x7f6520160770 2024-03-29T09:11:11.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6517e00700 1 -- 172.21.15.99:0/3071836427 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6518020070 con 0x7f6520160770 2024-03-29T09:11:11.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.707+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65201f64d0 con 0x7f6520160770 2024-03-29T09:11:11.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.708+0000 7f6517e00700 1 -- 172.21.15.99:0/3071836427 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518004280 con 0x7f6520160770 2024-03-29T09:11:11.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.708+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65201f6a90 con 0x7f6520160770 2024-03-29T09:11:11.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.709+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65200cc590 con 0x7f6520160770 2024-03-29T09:11:11.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.709+0000 7f6517e00700 1 -- 172.21.15.99:0/3071836427 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f65180043e0 con 0x7f6520160770 2024-03-29T09:11:11.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.710+0000 7f6517e00700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650006ac90 0x7f650006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:11.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.710+0000 7f6517e00700 1 -- 172.21.15.99:0/3071836427 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f6518022070 con 0x7f6520160770 2024-03-29T09:11:11.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.710+0000 7f651fe00700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650006ac90 0x7f650006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:11.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.711+0000 7f651fe00700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650006ac90 0x7f650006d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6510000f60 tx=0x7f6510009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:11.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.715+0000 7f6517e00700 1 -- 172.21.15.99:0/3071836427 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f65200cc590 con 0x7f6520160770 2024-03-29T09:11:11.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.918+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65200cc590 con 0x7f6520160770 2024-03-29T09:11:11.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.918+0000 7f6517e00700 1 -- 172.21.15.99:0/3071836427 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65200cc590 con 0x7f6520160770 2024-03-29T09:11:11.919 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:11:11.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650006ac90 msgr2=0x7f650006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650006ac90 0x7f650006d150 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6510000f60 tx=0x7f6510009040 comp rx=0 tx=0).stop 2024-03-29T09:11:11.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6520160770 msgr2=0x7f65201593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:11.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6520160770 0x7f65201593f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6518013000 tx=0x7f6518004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 shutdown_connections 2024-03-29T09:11:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650006ac90 0x7f650006d150 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6520160770 0x7f65201593f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65200b3cc0 0x7f65201f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 --2- 172.21.15.99:0/3071836427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65200cf090 0x7f65201f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 >> 172.21.15.99:0/3071836427 conn(0x7f652008afd0 msgr2=0x7f65200cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 shutdown_connections 2024-03-29T09:11:11.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:11.921+0000 7f6527000700 1 -- 172.21.15.99:0/3071836427 wait complete. 2024-03-29T09:11:11.939 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-03-29T09:11:12.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.134+0000 7fa189000700 1 Processor -- start 2024-03-29T09:11:12.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.134+0000 7fa189000700 1 -- start start 2024-03-29T09:11:12.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.135+0000 7fa189000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa184160770 0x7fa1840b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.135+0000 7fa189000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d3550 0x7fa1840ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa189000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1840d1520 con 0x7fa1840d0d60 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa189000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1840d1800 con 0x7fa184160770 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa189000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840d1ae0 con 0x7fa1840d3550 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa184160770 0x7fa1840b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa184160770 0x7fa1840b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35536/0 (socket says 172.21.15.99:35536) 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 -- 172.21.15.99:0/3018065194 learned_addr learned my addr 172.21.15.99:0/3018065194 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 -- 172.21.15.99:0/3018065194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d3550 msgr2=0x7fa1840ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 --2- 172.21.15.99:0/3018065194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d3550 0x7fa1840ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 -- 172.21.15.99:0/3018065194 >> v1:172.21.15.133:6789/0 conn(0x7fa1840d0d60 legacy=0x7fa1840d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 -- 172.21.15.99:0/3018065194 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1840d1dc0 con 0x7fa184160770 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.136+0000 7fa181e00700 1 --2- 172.21.15.99:0/3018065194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa1840b3cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa1740098f0 tx=0x7fa174009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69b10648239e93b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa180a00700 1 -- 172.21.15.99:0/3018065194 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174004230 con 0x7fa184160770 2024-03-29T09:11:12.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa180a00700 1 -- 172.21.15.99:0/3018065194 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174004390 con 0x7fa184160770 2024-03-29T09:11:12.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa180a00700 1 -- 172.21.15.99:0/3018065194 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1740036a0 con 0x7fa184160770 2024-03-29T09:11:12.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 -- 172.21.15.99:0/3018065194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 msgr2=0x7fa1840b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 --2- 172.21.15.99:0/3018065194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa1840b3cc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa1740098f0 tx=0x7fa174009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 -- 172.21.15.99:0/3018065194 shutdown_connections 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 --2- 172.21.15.99:0/3018065194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d3550 0x7fa1840ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 --2- 172.21.15.99:0/3018065194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa1840b3cc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 -- 172.21.15.99:0/3018065194 >> 172.21.15.99:0/3018065194 conn(0x7fa18408afd0 msgr2=0x7fa18409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 -- 172.21.15.99:0/3018065194 shutdown_connections 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.137+0000 7fa189000700 1 -- 172.21.15.99:0/3018065194 wait complete. 2024-03-29T09:11:12.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.139+0000 7fa189000700 1 Processor -- start 2024-03-29T09:11:12.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.139+0000 7fa189000700 1 -- start start 2024-03-29T09:11:12.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa189000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d0d60 0x7fa1840b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa189000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1840d3550 0x7fa18414b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa189000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa18414b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa189000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840cbe20 con 0x7fa184160770 2024-03-29T09:11:12.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa189000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840cb2a0 con 0x7fa1840d3550 2024-03-29T09:11:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa189000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1840cb580 con 0x7fa1840d0d60 2024-03-29T09:11:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa182800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa18414b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa181e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d0d60 0x7fa1840b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa181400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1840d3550 0x7fa18414b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.140+0000 7fa182800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa18414b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35550/0 (socket says 172.21.15.99:35550) 2024-03-29T09:11:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa181e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d0d60 0x7fa1840b86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:53632/0 (socket says 172.21.15.99:53632) 2024-03-29T09:11:12.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa182800700 1 -- 172.21.15.99:0/2112921257 learned_addr learned my addr 172.21.15.99:0/2112921257 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa182800700 1 -- 172.21.15.99:0/2112921257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d0d60 msgr2=0x7fa1840b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa182800700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d0d60 0x7fa1840b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa182800700 1 -- 172.21.15.99:0/2112921257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1840d3550 msgr2=0x7fa18414b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa182800700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1840d3550 0x7fa18414b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa182800700 1 -- 172.21.15.99:0/2112921257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009550 con 0x7fa184160770 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa182800700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa18414b810 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b9e0 tx=0x7fa17c00bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa17a000700 1 -- 172.21.15.99:0/2112921257 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00d8e0 con 0x7fa184160770 2024-03-29T09:11:12.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa17a000700 1 -- 172.21.15.99:0/2112921257 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17c00da80 con 0x7fa184160770 2024-03-29T09:11:12.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1840cb860 con 0x7fa184160770 2024-03-29T09:11:12.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.141+0000 7fa17a000700 1 -- 172.21.15.99:0/2112921257 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00de30 con 0x7fa184160770 2024-03-29T09:11:12.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.142+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1840b8c00 con 0x7fa184160770 2024-03-29T09:11:12.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.143+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1840b5e80 con 0x7fa184160770 2024-03-29T09:11:12.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.143+0000 7fa17a000700 1 -- 172.21.15.99:0/2112921257 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa17c015420 con 0x7fa184160770 2024-03-29T09:11:12.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.144+0000 7fa17a000700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa16406ac90 0x7fa16406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.145+0000 7fa17a000700 1 -- 172.21.15.99:0/2112921257 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa17c011070 con 0x7fa184160770 2024-03-29T09:11:12.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.145+0000 7fa181e00700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa16406ac90 0x7fa16406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.145 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.145+0000 7fa181e00700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa16406ac90 0x7fa16406d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa174009130 tx=0x7fa174008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:12.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.148+0000 7fa17a000700 1 -- 172.21.15.99:0/2112921257 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa17c04f580 con 0x7fa184160770 2024-03-29T09:11:12.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.351+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fa1840b8dc0 con 0x7fa184160770 2024-03-29T09:11:12.351 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.351+0000 7fa17a000700 1 -- 172.21.15.99:0/2112921257 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7fa1840b5e80 con 0x7fa184160770 2024-03-29T09:11:12.351 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa16406ac90 msgr2=0x7fa16406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa16406ac90 0x7fa16406d150 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa174009130 tx=0x7fa174008030 comp rx=0 tx=0).stop 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 msgr2=0x7fa18414b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa18414b810 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b9e0 tx=0x7fa17c00bcf0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 shutdown_connections 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa16406ac90 0x7fa16406d150 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa184160770 0x7fa18414b810 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1840d0d60 0x7fa1840b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 --2- 172.21.15.99:0/2112921257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1840d3550 0x7fa18414b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 >> 172.21.15.99:0/2112921257 conn(0x7fa18408afd0 msgr2=0x7fa18406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:12.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 shutdown_connections 2024-03-29T09:11:12.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.354+0000 7fa189000700 1 -- 172.21.15.99:0/2112921257 wait complete. 2024-03-29T09:11:12.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:11:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.563+0000 7efe28c00700 1 Processor -- start 2024-03-29T09:11:12.563 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.563+0000 7efe28c00700 1 -- start start 2024-03-29T09:11:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.564+0000 7efe28c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe240b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.564+0000 7efe28c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe240cd820 con 0x7efe2415db00 2024-03-29T09:11:12.564 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.564+0000 7efe28c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe2415e530 con 0x7efe24143940 2024-03-29T09:11:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.564+0000 7efe28c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe24143d20 con 0x7efe240ce9d0 2024-03-29T09:11:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe21400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7efe2415db00 0x7efe240ce2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39818/0 (socket says 172.21.15.99:39818) 2024-03-29T09:11:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe21400700 1 -- 172.21.15.99:0/3658417517 learned_addr learned my addr 172.21.15.99:0/3658417517 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:12.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe22800700 1 --2- 172.21.15.99:0/3658417517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe240b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe20a00700 1 -- 172.21.15.99:0/3658417517 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983279588 0 0) 0x7efe240cd820 con 0x7efe2415db00 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe20a00700 1 -- 172.21.15.99:0/3658417517 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe080035b0 con 0x7efe2415db00 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe20a00700 1 -- 172.21.15.99:0/3658417517 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190843231 0 0) 0x7efe2415e530 con 0x7efe24143940 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe20a00700 1 -- 172.21.15.99:0/3658417517 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe240cd820 con 0x7efe24143940 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe22800700 1 -- 172.21.15.99:0/3658417517 >> v1:172.21.15.133:6789/0 conn(0x7efe24143940 legacy=0x7efe24070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe22800700 1 -- 172.21.15.99:0/3658417517 >> v1:172.21.15.99:6789/0 conn(0x7efe2415db00 legacy=0x7efe240ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe22800700 1 -- 172.21.15.99:0/3658417517 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2415e810 con 0x7efe240ce9d0 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.565+0000 7efe22800700 1 --2- 172.21.15.99:0/3658417517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe240b0b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009dc0 tx=0x7efe1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a7eab705aa82e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.566+0000 7efe20a00700 1 -- 172.21.15.99:0/3658417517 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c00e070 con 0x7efe240ce9d0 2024-03-29T09:11:12.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.566+0000 7efe20a00700 1 -- 172.21.15.99:0/3658417517 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c00bb20 con 0x7efe240ce9d0 2024-03-29T09:11:12.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.566+0000 7efe20a00700 1 -- 172.21.15.99:0/3658417517 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c012430 con 0x7efe240ce9d0 2024-03-29T09:11:12.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.567+0000 7efe28c00700 1 -- 172.21.15.99:0/3658417517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 msgr2=0x7efe240b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.567+0000 7efe28c00700 1 --2- 172.21.15.99:0/3658417517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe240b0b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009dc0 tx=0x7efe1c00b330 comp rx=0 tx=0).stop 2024-03-29T09:11:12.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.567+0000 7efe28c00700 1 -- 172.21.15.99:0/3658417517 shutdown_connections 2024-03-29T09:11:12.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.567+0000 7efe28c00700 1 --2- 172.21.15.99:0/3658417517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe240b0b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.567+0000 7efe28c00700 1 -- 172.21.15.99:0/3658417517 >> 172.21.15.99:0/3658417517 conn(0x7efe24066540 msgr2=0x7efe24066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:12.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.567+0000 7efe28c00700 1 -- 172.21.15.99:0/3658417517 shutdown_connections 2024-03-29T09:11:12.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.567+0000 7efe28c00700 1 -- 172.21.15.99:0/3658417517 wait complete. 2024-03-29T09:11:12.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.568+0000 7efe28c00700 1 Processor -- start 2024-03-29T09:11:12.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.568+0000 7efe28c00700 1 -- start start 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.569+0000 7efe28c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe24136770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.569+0000 7efe28c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe24143940 0x7efe2413a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.569+0000 7efe28c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe2415db00 0x7efe24139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.569+0000 7efe28c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe240aff40 con 0x7efe24143940 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.569+0000 7efe28c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe24136cb0 con 0x7efe2415db00 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.569+0000 7efe28c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe240af330 con 0x7efe240ce9d0 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe24143940 0x7efe2413a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe24143940 0x7efe2413a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35566/0 (socket says 172.21.15.99:35566) 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 -- 172.21.15.99:0/3413370679 learned_addr learned my addr 172.21.15.99:0/3413370679 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:12.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21e00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe24136770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe22800700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe2415db00 0x7efe24139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 -- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 msgr2=0x7efe24136770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe24136770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 -- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe2415db00 msgr2=0x7efe24139750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe2415db00 0x7efe24139750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 -- 172.21.15.99:0/3413370679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c009a70 con 0x7efe24143940 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe22800700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe2415db00 0x7efe24139750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.570+0000 7efe21400700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe24143940 0x7efe2413a430 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efe10001eb0 tx=0x7efe1000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.571+0000 7efe1a000700 1 -- 172.21.15.99:0/3413370679 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10005d70 con 0x7efe24143940 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.571+0000 7efe1a000700 1 -- 172.21.15.99:0/3413370679 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe100070d0 con 0x7efe24143940 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.571+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe240af610 con 0x7efe24143940 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.571+0000 7efe1a000700 1 -- 172.21.15.99:0/3413370679 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10007460 con 0x7efe24143940 2024-03-29T09:11:12.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.571+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe240afbd0 con 0x7efe24143940 2024-03-29T09:11:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.572+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe240cd1d0 con 0x7efe24143940 2024-03-29T09:11:12.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.573+0000 7efe1a000700 1 -- 172.21.15.99:0/3413370679 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efe100075c0 con 0x7efe24143940 2024-03-29T09:11:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.575+0000 7efe1a000700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe0406ac90 0x7efe0406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:12.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.575+0000 7efe1a000700 1 -- 172.21.15.99:0/3413370679 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7efe10088aa0 con 0x7efe24143940 2024-03-29T09:11:12.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.575+0000 7efe21e00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe0406ac90 0x7efe0406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:12.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.575+0000 7efe21e00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe0406ac90 0x7efe0406d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efe140011e0 tx=0x7efe14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:12.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.579+0000 7efe1a000700 1 -- 172.21.15.99:0/3413370679 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efe100539b0 con 0x7efe24143940 2024-03-29T09:11:12.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.796+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efe240677a0 con 0x7efe24143940 2024-03-29T09:11:12.796 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.796+0000 7efe1a000700 1 -- 172.21.15.99:0/3413370679 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7efe10017080 con 0x7efe24143940 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe0406ac90 msgr2=0x7efe0406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe0406ac90 0x7efe0406d150 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efe140011e0 tx=0x7efe14009040 comp rx=0 tx=0).stop 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe24143940 msgr2=0x7efe2413a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe24143940 0x7efe2413a430 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efe10001eb0 tx=0x7efe1000c7b0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 shutdown_connections 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe0406ac90 0x7efe0406d150 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe2415db00 0x7efe24139750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe240ce9d0 0x7efe24136770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.799 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 --2- 172.21.15.99:0/3413370679 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe24143940 0x7efe2413a430 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:12.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 >> 172.21.15.99:0/3413370679 conn(0x7efe24066540 msgr2=0x7efe2413e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:12.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 shutdown_connections 2024-03-29T09:11:12.800 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:12.799+0000 7efe28c00700 1 -- 172.21.15.99:0/3413370679 wait complete. 2024-03-29T09:11:12.800 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:11:12.817 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:12.817 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:12.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.090+0000 7fb74ea00700 1 Processor -- start 2024-03-29T09:11:13.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.090+0000 7fb74ea00700 1 -- start start 2024-03-29T09:11:13.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.091+0000 7fb74ea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1780 0x7fb748168350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.091+0000 7fb74ea00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7480d1a50 con 0x7fb74815d980 2024-03-29T09:11:13.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.091+0000 7fb74ea00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7480d2260 con 0x7fb7480cfd70 2024-03-29T09:11:13.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.091+0000 7fb74ea00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7480d1ca0 con 0x7fb7480b1780 2024-03-29T09:11:13.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb74ca00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb7480b1780 0x7fb748168350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb747400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb74815d980 0x7fb7480cdd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39826/0 (socket says 172.21.15.99:39826) 2024-03-29T09:11:13.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb747400700 1 -- 172.21.15.99:0/81754590 learned_addr learned my addr 172.21.15.99:0/81754590 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:13.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb74ca00700 1 -- 172.21.15.99:0/81754590 >> v1:172.21.15.133:6789/0 conn(0x7fb7480cfd70 legacy=0x7fb74815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb746a00700 1 -- 172.21.15.99:0/81754590 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439049193 0 0) 0x7fb7480d1a50 con 0x7fb74815d980 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb74ca00700 1 -- 172.21.15.99:0/81754590 >> v1:172.21.15.99:6789/0 conn(0x7fb74815d980 legacy=0x7fb7480cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb74ca00700 1 -- 172.21.15.99:0/81754590 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480d1f80 con 0x7fb7480b1780 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb74ca00700 1 --2- 172.21.15.99:0/81754590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7480b1780 0x7fb748168350 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb740009d90 tx=0x7fb74000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e241ea59221e9af server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb746a00700 1 -- 172.21.15.99:0/81754590 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74000e070 con 0x7fb7480b1780 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb746a00700 1 -- 172.21.15.99:0/81754590 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74000bbd0 con 0x7fb7480b1780 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.092+0000 7fb746a00700 1 -- 172.21.15.99:0/81754590 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb740012430 con 0x7fb7480b1780 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.093+0000 7fb74ea00700 1 -- 172.21.15.99:0/81754590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7480b1780 msgr2=0x7fb748168350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.093+0000 7fb74ea00700 1 --2- 172.21.15.99:0/81754590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7480b1780 0x7fb748168350 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb740009d90 tx=0x7fb74000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:13.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.093+0000 7fb74ea00700 1 -- 172.21.15.99:0/81754590 shutdown_connections 2024-03-29T09:11:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.093+0000 7fb74ea00700 1 --2- 172.21.15.99:0/81754590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7480b1780 0x7fb748168350 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.093+0000 7fb74ea00700 1 -- 172.21.15.99:0/81754590 >> 172.21.15.99:0/81754590 conn(0x7fb74809c470 msgr2=0x7fb7480d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.093+0000 7fb74ea00700 1 -- 172.21.15.99:0/81754590 shutdown_connections 2024-03-29T09:11:13.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.093+0000 7fb74ea00700 1 -- 172.21.15.99:0/81754590 wait complete. 2024-03-29T09:11:13.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.095+0000 7fb74ea00700 1 Processor -- start 2024-03-29T09:11:13.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.095+0000 7fb74ea00700 1 -- start start 2024-03-29T09:11:13.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ea00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7480b1780 0x7fb7480c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ea00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7480cfd70 0x7fb7481f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74815d980 0x7fb7480c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ea00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480b7510 con 0x7fb74815d980 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ea00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480b77f0 con 0x7fb7480b1780 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ea00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7480b7ad0 con 0x7fb7480cfd70 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74815d980 0x7fb7480c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ca00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74815d980 0x7fb7480c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35582/0 (socket says 172.21.15.99:35582) 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb74ca00700 1 -- 172.21.15.99:0/1732293883 learned_addr learned my addr 172.21.15.99:0/1732293883 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb747e00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7480b1780 0x7fb7480c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.096+0000 7fb747400700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7480cfd70 0x7fb7481f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ca00700 1 -- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7480cfd70 msgr2=0x7fb7481f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ca00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7480cfd70 0x7fb7481f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ca00700 1 -- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7480b1780 msgr2=0x7fb7480c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ca00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7480b1780 0x7fb7480c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ca00700 1 -- 172.21.15.99:0/1732293883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740009a70 con 0x7fb74815d980 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ca00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74815d980 0x7fb7480c9670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb740014040 tx=0x7fb740004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb73fe00700 1 -- 172.21.15.99:0/1732293883 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74001b810 con 0x7fb74815d980 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb73fe00700 1 -- 172.21.15.99:0/1732293883 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb740020070 con 0x7fb74815d980 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7480b7db0 con 0x7fb74815d980 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb73fe00700 1 -- 172.21.15.99:0/1732293883 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74001bbb0 con 0x7fb74815d980 2024-03-29T09:11:13.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.097+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7480c9bb0 con 0x7fb74815d980 2024-03-29T09:11:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.098+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7480b4a30 con 0x7fb74815d980 2024-03-29T09:11:13.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.099+0000 7fb73fe00700 1 -- 172.21.15.99:0/1732293883 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb74001bd10 con 0x7fb74815d980 2024-03-29T09:11:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.100+0000 7fb73fe00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72806ac90 0x7fb72806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.101+0000 7fb73fe00700 1 -- 172.21.15.99:0/1732293883 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb740022070 con 0x7fb74815d980 2024-03-29T09:11:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.101+0000 7fb747e00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72806ac90 0x7fb72806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.101+0000 7fb747e00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72806ac90 0x7fb72806d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb738007fd0 tx=0x7fb738009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.105+0000 7fb73fe00700 1 -- 172.21.15.99:0/1732293883 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb740056540 con 0x7fb74815d980 2024-03-29T09:11:13.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.310+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7480c9e30 con 0x7fb74815d980 2024-03-29T09:11:13.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.312+0000 7fb73fe00700 1 -- 172.21.15.99:0/1732293883 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7fb7480c9e30 con 0x7fb74815d980 2024-03-29T09:11:13.312 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:13.312 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:13.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72806ac90 msgr2=0x7fb72806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72806ac90 0x7fb72806d150 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb738007fd0 tx=0x7fb738009040 comp rx=0 tx=0).stop 2024-03-29T09:11:13.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74815d980 msgr2=0x7fb7480c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74815d980 0x7fb7480c9670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb740014040 tx=0x7fb740004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 shutdown_connections 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72806ac90 0x7fb72806d150 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74815d980 0x7fb7480c9670 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7480b1780 0x7fb7480c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 --2- 172.21.15.99:0/1732293883 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7480cfd70 0x7fb7481f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 >> 172.21.15.99:0/1732293883 conn(0x7fb74809c470 msgr2=0x7fb7480b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 shutdown_connections 2024-03-29T09:11:13.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.314+0000 7fb74ea00700 1 -- 172.21.15.99:0/1732293883 wait complete. 2024-03-29T09:11:13.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:10:42.037884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:10:42.900862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-03-29T09:10:53.257132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:11:13.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:11:13.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.525+0000 7fa752600700 1 Processor -- start 2024-03-29T09:11:13.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.525+0000 7fa752600700 1 -- start start 2024-03-29T09:11:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.526+0000 7fa752600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa74c0b3cc0 0x7fa74c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.526+0000 7fa752600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0d3380 0x7fa74c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.526+0000 7fa752600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74c0d3810 con 0x7fa74c160770 2024-03-29T09:11:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.526+0000 7fa752600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74c0b4140 con 0x7fa74c0b3cc0 2024-03-29T09:11:13.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.526+0000 7fa752600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0d1520 con 0x7fa74c0d3380 2024-03-29T09:11:13.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0d3380 0x7fa74c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74aa00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa74c0b3cc0 0x7fa74c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74b400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa74c160770 0x7fa74c0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:51636/0 (socket says 172.21.15.99:51636) 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74b400700 1 -- 172.21.15.99:0/553010059 learned_addr learned my addr 172.21.15.99:0/553010059 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74a000700 1 -- 172.21.15.99:0/553010059 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024242977 0 0) 0x7fa74c0d3810 con 0x7fa74c160770 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74aa00700 1 -- 172.21.15.99:0/553010059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0d3380 msgr2=0x7fa74c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74aa00700 1 --2- 172.21.15.99:0/553010059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0d3380 0x7fa74c0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74aa00700 1 -- 172.21.15.99:0/553010059 >> v1:172.21.15.133:6789/0 conn(0x7fa74c160770 legacy=0x7fa74c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.527+0000 7fa74aa00700 1 -- 172.21.15.99:0/553010059 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c0d1800 con 0x7fa74c0b3cc0 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa74aa00700 1 --2- 172.21.15.99:0/553010059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c0b3cc0 0x7fa74c0d2e40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa738009840 tx=0x7fa738009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e66a3b54f73a0fd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa74a000700 1 -- 172.21.15.99:0/553010059 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738011830 con 0x7fa74c0b3cc0 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa74a000700 1 -- 172.21.15.99:0/553010059 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa738011990 con 0x7fa74c0b3cc0 2024-03-29T09:11:13.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa74a000700 1 -- 172.21.15.99:0/553010059 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738011d20 con 0x7fa74c0b3cc0 2024-03-29T09:11:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa752600700 1 -- 172.21.15.99:0/553010059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c0b3cc0 msgr2=0x7fa74c0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa752600700 1 --2- 172.21.15.99:0/553010059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c0b3cc0 0x7fa74c0d2e40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa738009840 tx=0x7fa738009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:13.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa752600700 1 -- 172.21.15.99:0/553010059 shutdown_connections 2024-03-29T09:11:13.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa752600700 1 --2- 172.21.15.99:0/553010059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0d3380 0x7fa74c0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa752600700 1 --2- 172.21.15.99:0/553010059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c0b3cc0 0x7fa74c0d2e40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.528+0000 7fa752600700 1 -- 172.21.15.99:0/553010059 >> 172.21.15.99:0/553010059 conn(0x7fa74c08afd0 msgr2=0x7fa74c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:13.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.529+0000 7fa752600700 1 -- 172.21.15.99:0/553010059 shutdown_connections 2024-03-29T09:11:13.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.529+0000 7fa752600700 1 -- 172.21.15.99:0/553010059 wait complete. 2024-03-29T09:11:13.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.531+0000 7fa752600700 1 Processor -- start 2024-03-29T09:11:13.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.531+0000 7fa752600700 1 -- start start 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa752600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0b3cc0 0x7fa74c0a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa752600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa74c0d3380 0x7fa74c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa752600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c160770 0x7fa74c0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa752600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0cb250 con 0x7fa74c0d3380 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa752600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0a06d0 con 0x7fa74c160770 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa752600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c09fda0 con 0x7fa74c0b3cc0 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa74aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa74c0d3380 0x7fa74c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.532+0000 7fa74aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa74c0d3380 0x7fa74c1545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35586/0 (socket says 172.21.15.99:35586) 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74aa00700 1 -- 172.21.15.99:0/540599195 learned_addr learned my addr 172.21.15.99:0/540599195 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:13.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74be00700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c160770 0x7fa74c0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74b400700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0b3cc0 0x7fa74c0a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74aa00700 1 -- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0b3cc0 msgr2=0x7fa74c0a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74aa00700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0b3cc0 0x7fa74c0a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74aa00700 1 -- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c160770 msgr2=0x7fa74c0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74aa00700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c160770 0x7fa74c0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74aa00700 1 -- 172.21.15.99:0/540599195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7380094a0 con 0x7fa74c0d3380 2024-03-29T09:11:13.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa74aa00700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa74c0d3380 0x7fa74c1545d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa7380090b0 tx=0x7fa738004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.533+0000 7fa743400700 1 -- 172.21.15.99:0/540599195 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738004550 con 0x7fa74c0d3380 2024-03-29T09:11:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.534+0000 7fa743400700 1 -- 172.21.15.99:0/540599195 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73801f070 con 0x7fa74c0d3380 2024-03-29T09:11:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.534+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c0a0080 con 0x7fa74c0d3380 2024-03-29T09:11:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.534+0000 7fa743400700 1 -- 172.21.15.99:0/540599195 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380038e0 con 0x7fa74c0d3380 2024-03-29T09:11:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.534+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c154b10 con 0x7fa74c0d3380 2024-03-29T09:11:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.535+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c154cd0 con 0x7fa74c0d3380 2024-03-29T09:11:13.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.536+0000 7fa743400700 1 -- 172.21.15.99:0/540599195 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa738003a40 con 0x7fa74c0d3380 2024-03-29T09:11:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.536+0000 7fa743400700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa72c06ac90 0x7fa72c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.536+0000 7fa743400700 1 -- 172.21.15.99:0/540599195 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa738021070 con 0x7fa74c0d3380 2024-03-29T09:11:13.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.537+0000 7fa74b400700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa72c06ac90 0x7fa72c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.537+0000 7fa74b400700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa72c06ac90 0x7fa72c06d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa73c000f60 tx=0x7fa73c007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.541+0000 7fa743400700 1 -- 172.21.15.99:0/540599195 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa738091080 con 0x7fa74c0d3380 2024-03-29T09:11:13.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.743+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa74c0678b0 con 0x7fa74c0d3380 2024-03-29T09:11:13.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.743+0000 7fa743400700 1 -- 172.21.15.99:0/540599195 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7380150a0 con 0x7fa74c0d3380 2024-03-29T09:11:13.743 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa72c06ac90 msgr2=0x7fa72c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa72c06ac90 0x7fa72c06d150 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa73c000f60 tx=0x7fa73c007e70 comp rx=0 tx=0).stop 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa74c0d3380 msgr2=0x7fa74c1545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa74c0d3380 0x7fa74c1545d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa7380090b0 tx=0x7fa738004750 comp rx=0 tx=0).stop 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 shutdown_connections 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa72c06ac90 0x7fa72c06d150 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa74c160770 0x7fa74c0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa74c0b3cc0 0x7fa74c0a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 --2- 172.21.15.99:0/540599195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa74c0d3380 0x7fa74c1545d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.745+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 >> 172.21.15.99:0/540599195 conn(0x7fa74c08afd0 msgr2=0x7fa74c06bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:13.746 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.746+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 shutdown_connections 2024-03-29T09:11:13.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.746+0000 7fa752600700 1 -- 172.21.15.99:0/540599195 wait complete. 2024-03-29T09:11:13.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:11:13.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.956+0000 7fe90de00700 1 Processor -- start 2024-03-29T09:11:13.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.956+0000 7fe90de00700 1 -- start start 2024-03-29T09:11:13.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe90de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9080d1760 0x7fe9080d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe90de00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080cdcb0 0x7fe9080d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe90de00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe9080d12b0 0x7fe9080b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe90de00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe908161ae0 con 0x7fe9080d1760 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe90de00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe908160f10 con 0x7fe9080d12b0 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe90de00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9080d2080 con 0x7fe9080cdcb0 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe906a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9080d1760 0x7fe9080d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe906a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe9080d1760 0x7fe9080d1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35602/0 (socket says 172.21.15.99:35602) 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe906a00700 1 -- 172.21.15.99:0/1461080418 learned_addr learned my addr 172.21.15.99:0/1461080418 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe906000700 1 --2- 172.21.15.99:0/1461080418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080cdcb0 0x7fe9080d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.957+0000 7fe907400700 1 --2- 172.21.15.99:0/1461080418 >> v2:172.21.15.133:3300/0 conn(0x7fe9080d12b0 0x7fe9080b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe906a00700 1 -- 172.21.15.99:0/1461080418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080cdcb0 msgr2=0x7fe9080d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:13.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe906a00700 1 --2- 172.21.15.99:0/1461080418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080cdcb0 0x7fe9080d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe906a00700 1 -- 172.21.15.99:0/1461080418 >> v2:172.21.15.133:3300/0 conn(0x7fe9080d12b0 msgr2=0x7fe9080b32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe906a00700 1 --2- 172.21.15.99:0/1461080418 >> v2:172.21.15.133:3300/0 conn(0x7fe9080d12b0 0x7fe9080b32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe906a00700 1 -- 172.21.15.99:0/1461080418 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9081611b0 con 0x7fe9080d1760 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe906000700 1 --2- 172.21.15.99:0/1461080418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080cdcb0 0x7fe9080d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe907400700 1 --2- 172.21.15.99:0/1461080418 >> v2:172.21.15.133:3300/0 conn(0x7fe9080d12b0 0x7fe9080b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe906a00700 1 --2- 172.21.15.99:0/1461080418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080d1760 0x7fe9080d1b40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe9080b5d70 tx=0x7fe8f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8613f9d6facb5ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe905600700 1 -- 172.21.15.99:0/1461080418 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800b870 con 0x7fe9080d1760 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe905600700 1 -- 172.21.15.99:0/1461080418 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f800b9d0 con 0x7fe9080d1760 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.958+0000 7fe905600700 1 -- 172.21.15.99:0/1461080418 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800bd60 con 0x7fe9080d1760 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 -- 172.21.15.99:0/1461080418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080d1760 msgr2=0x7fe9080d1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 --2- 172.21.15.99:0/1461080418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080d1760 0x7fe9080d1b40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe9080b5d70 tx=0x7fe8f8009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 -- 172.21.15.99:0/1461080418 shutdown_connections 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 --2- 172.21.15.99:0/1461080418 >> v2:172.21.15.133:3300/0 conn(0x7fe9080d12b0 0x7fe9080b32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 --2- 172.21.15.99:0/1461080418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080d1760 0x7fe9080d1b40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 --2- 172.21.15.99:0/1461080418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080cdcb0 0x7fe9080d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 -- 172.21.15.99:0/1461080418 >> 172.21.15.99:0/1461080418 conn(0x7fe90808afd0 msgr2=0x7fe90809c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:13.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 -- 172.21.15.99:0/1461080418 shutdown_connections 2024-03-29T09:11:13.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.959+0000 7fe90de00700 1 -- 172.21.15.99:0/1461080418 wait complete. 2024-03-29T09:11:13.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.961+0000 7fe90de00700 1 Processor -- start 2024-03-29T09:11:13.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.961+0000 7fe90de00700 1 -- start start 2024-03-29T09:11:13.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe90de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080cdcb0 0x7fe90815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe90de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080d12b0 0x7fe9080c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe90de00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080d1760 0x7fe9081f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe90de00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9080c9c50 con 0x7fe9080cdcb0 2024-03-29T09:11:13.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe90de00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9080b1870 con 0x7fe9080d12b0 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe90de00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9080b0860 con 0x7fe9080d1760 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080cdcb0 0x7fe90815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080cdcb0 0x7fe90815d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35618/0 (socket says 172.21.15.99:35618) 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 -- 172.21.15.99:0/3664694956 learned_addr learned my addr 172.21.15.99:0/3664694956 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 -- 172.21.15.99:0/3664694956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080d1760 msgr2=0x7fe9081f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080d1760 0x7fe9081f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 -- 172.21.15.99:0/3664694956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080d12b0 msgr2=0x7fe9080c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080d12b0 0x7fe9080c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 -- 172.21.15.99:0/3664694956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8009550 con 0x7fe9080cdcb0 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.962+0000 7fe906a00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080cdcb0 0x7fe90815d5b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8014040 tx=0x7fe8f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.963+0000 7fe8ff400700 1 -- 172.21.15.99:0/3664694956 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8004520 con 0x7fe9080cdcb0 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.963+0000 7fe8ff400700 1 -- 172.21.15.99:0/3664694956 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f801e070 con 0x7fe9080cdcb0 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.963+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9080b0ab0 con 0x7fe9080cdcb0 2024-03-29T09:11:13.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.963+0000 7fe8ff400700 1 -- 172.21.15.99:0/3664694956 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80038c0 con 0x7fe9080cdcb0 2024-03-29T09:11:13.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.963+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9080b1070 con 0x7fe9080cdcb0 2024-03-29T09:11:13.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.964+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9080678b0 con 0x7fe9080cdcb0 2024-03-29T09:11:13.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.965+0000 7fe8ff400700 1 -- 172.21.15.99:0/3664694956 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe8f8003a20 con 0x7fe9080cdcb0 2024-03-29T09:11:13.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.966+0000 7fe8ff400700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe8e806ac90 0x7fe8e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:13.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.966+0000 7fe8ff400700 1 -- 172.21.15.99:0/3664694956 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fe8f8020070 con 0x7fe9080cdcb0 2024-03-29T09:11:13.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.966+0000 7fe906000700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe8e806ac90 0x7fe8e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:13.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.967+0000 7fe906000700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe8e806ac90 0x7fe8e806d150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe908155a20 tx=0x7fe8f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:13.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:13.970+0000 7fe8ff400700 1 -- 172.21.15.99:0/3664694956 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe8f8055bc0 con 0x7fe9080cdcb0 2024-03-29T09:11:14.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.177+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9080b2db0 con 0x7fe9080cdcb0 2024-03-29T09:11:14.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.177+0000 7fe8ff400700 1 -- 172.21.15.99:0/3664694956 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8f80150a0 con 0x7fe9080cdcb0 2024-03-29T09:11:14.177 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe8e806ac90 msgr2=0x7fe8e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe8e806ac90 0x7fe8e806d150 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe908155a20 tx=0x7fe8f4008040 comp rx=0 tx=0).stop 2024-03-29T09:11:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080cdcb0 msgr2=0x7fe90815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080cdcb0 0x7fe90815d5b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8014040 tx=0x7fe8f8004750 comp rx=0 tx=0).stop 2024-03-29T09:11:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 shutdown_connections 2024-03-29T09:11:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe8e806ac90 0x7fe8e806d150 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9080d1760 0x7fe9081f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9080cdcb0 0x7fe90815d5b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 --2- 172.21.15.99:0/3664694956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9080d12b0 0x7fe9080c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 >> 172.21.15.99:0/3664694956 conn(0x7fe90808afd0 msgr2=0x7fe90806cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:14.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 shutdown_connections 2024-03-29T09:11:14.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.180+0000 7fe90de00700 1 -- 172.21.15.99:0/3664694956 wait complete. 2024-03-29T09:11:14.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:11:14.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.390+0000 7fab67800700 1 Processor -- start 2024-03-29T09:11:14.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.391+0000 7fab67800700 1 -- start start 2024-03-29T09:11:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab67800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab60160770 0x7fab600b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab67800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fab600d3550 0x7fab600ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab67800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab600d1520 con 0x7fab600d0d60 2024-03-29T09:11:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab67800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600d1800 con 0x7fab600d3550 2024-03-29T09:11:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab67800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab600d1ae0 con 0x7fab60160770 2024-03-29T09:11:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab65800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fab600d3550 0x7fab600ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab5fe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fab600d0d60 0x7fab600d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39830/0 (socket says 172.21.15.99:39830) 2024-03-29T09:11:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab5fe00700 1 -- 172.21.15.99:0/334025671 learned_addr learned my addr 172.21.15.99:0/334025671 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab64e00700 1 --2- 172.21.15.99:0/334025671 >> v2:172.21.15.133:3300/0 conn(0x7fab60160770 0x7fab600b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab5f400700 1 -- 172.21.15.99:0/334025671 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282488335 0 0) 0x7fab600d1520 con 0x7fab600d0d60 2024-03-29T09:11:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab5f400700 1 -- 172.21.15.99:0/334025671 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab480035b0 con 0x7fab600d0d60 2024-03-29T09:11:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab65800700 1 -- 172.21.15.99:0/334025671 >> v2:172.21.15.133:3300/0 conn(0x7fab60160770 msgr2=0x7fab600b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.392+0000 7fab65800700 1 --2- 172.21.15.99:0/334025671 >> v2:172.21.15.133:3300/0 conn(0x7fab60160770 0x7fab600b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.393+0000 7fab65800700 1 -- 172.21.15.99:0/334025671 >> v1:172.21.15.99:6789/0 conn(0x7fab600d0d60 legacy=0x7fab600d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.393+0000 7fab65800700 1 -- 172.21.15.99:0/334025671 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600d1dc0 con 0x7fab600d3550 2024-03-29T09:11:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.393+0000 7fab64e00700 1 --2- 172.21.15.99:0/334025671 >> v2:172.21.15.133:3300/0 conn(0x7fab60160770 0x7fab600b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.393+0000 7fab65800700 1 --2- 172.21.15.99:0/334025671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600ce080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fab600b5d20 tx=0x7fab5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7bf9bdc2a37d4d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.393+0000 7fab5f400700 1 -- 172.21.15.99:0/334025671 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5800e070 con 0x7fab600d3550 2024-03-29T09:11:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.393+0000 7fab5f400700 1 -- 172.21.15.99:0/334025671 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5800bb20 con 0x7fab600d3550 2024-03-29T09:11:14.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.393+0000 7fab5f400700 1 -- 172.21.15.99:0/334025671 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58012430 con 0x7fab600d3550 2024-03-29T09:11:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 -- 172.21.15.99:0/334025671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 msgr2=0x7fab600ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 --2- 172.21.15.99:0/334025671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600ce080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fab600b5d20 tx=0x7fab5800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 -- 172.21.15.99:0/334025671 shutdown_connections 2024-03-29T09:11:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 --2- 172.21.15.99:0/334025671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600ce080 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 --2- 172.21.15.99:0/334025671 >> v2:172.21.15.133:3300/0 conn(0x7fab60160770 0x7fab600b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 -- 172.21.15.99:0/334025671 >> 172.21.15.99:0/334025671 conn(0x7fab6008afd0 msgr2=0x7fab6009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:14.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 -- 172.21.15.99:0/334025671 shutdown_connections 2024-03-29T09:11:14.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.394+0000 7fab67800700 1 -- 172.21.15.99:0/334025671 wait complete. 2024-03-29T09:11:14.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.395+0000 7fab67800700 1 Processor -- start 2024-03-29T09:11:14.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.396+0000 7fab67800700 1 -- start start 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.396+0000 7fab67800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab600d0d60 0x7fab6015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab67800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab67800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab60160770 0x7fab6015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab67800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab600b7b50 con 0x7fab600d3550 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab67800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6009e040 con 0x7fab600d0d60 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab67800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab600b6ed0 con 0x7fab60160770 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600b85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35640/0 (socket says 172.21.15.99:35640) 2024-03-29T09:11:14.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 -- 172.21.15.99:0/2054968491 learned_addr learned my addr 172.21.15.99:0/2054968491 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab64e00700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab600d0d60 0x7fab6015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab65800700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab60160770 0x7fab6015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 -- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab60160770 msgr2=0x7fab6015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab60160770 0x7fab6015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 -- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab600d0d60 msgr2=0x7fab6015c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab600d0d60 0x7fab6015c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.397+0000 7fab5fe00700 1 -- 172.21.15.99:0/2054968491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab540094a0 con 0x7fab600d3550 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.398+0000 7fab5fe00700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600b85a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c700 tx=0x7fab4c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.398+0000 7fab5cc00700 1 -- 172.21.15.99:0/2054968491 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c005d70 con 0x7fab600d3550 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.398+0000 7fab5cc00700 1 -- 172.21.15.99:0/2054968491 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4c0070d0 con 0x7fab600d3550 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.398+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab58009a70 con 0x7fab600d3550 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.398+0000 7fab5cc00700 1 -- 172.21.15.99:0/2054968491 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c007460 con 0x7fab600d3550 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.398+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab600b7490 con 0x7fab600d3550 2024-03-29T09:11:14.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.399+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab600b7c90 con 0x7fab600d3550 2024-03-29T09:11:14.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.400+0000 7fab5cc00700 1 -- 172.21.15.99:0/2054968491 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fab4c0075c0 con 0x7fab600d3550 2024-03-29T09:11:14.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.401+0000 7fab5cc00700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab4006ac90 0x7fab4006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.401+0000 7fab5cc00700 1 -- 172.21.15.99:0/2054968491 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fab4c0889f0 con 0x7fab600d3550 2024-03-29T09:11:14.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.401+0000 7fab64e00700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab4006ac90 0x7fab4006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.401+0000 7fab64e00700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab4006ac90 0x7fab4006d150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fab54009cd0 tx=0x7fab54009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:14.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.405+0000 7fab5cc00700 1 -- 172.21.15.99:0/2054968491 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fab4c0538f0 con 0x7fab600d3550 2024-03-29T09:11:14.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.608+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab600678b0 con 0x7fab600d3550 2024-03-29T09:11:14.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.608+0000 7fab5cc00700 1 -- 172.21.15.99:0/2054968491 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab4c0170a0 con 0x7fab600d3550 2024-03-29T09:11:14.609 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:11:14.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.610+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab4006ac90 msgr2=0x7fab4006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.611 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab4006ac90 0x7fab4006d150 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fab54009cd0 tx=0x7fab54009210 comp rx=0 tx=0).stop 2024-03-29T09:11:14.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 msgr2=0x7fab600b85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600b85a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00c700 tx=0x7fab4c00ca10 comp rx=0 tx=0).stop 2024-03-29T09:11:14.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 shutdown_connections 2024-03-29T09:11:14.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fab4006ac90 0x7fab4006d150 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab60160770 0x7fab6015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab600d0d60 0x7fab6015c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 --2- 172.21.15.99:0/2054968491 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab600d3550 0x7fab600b85a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 >> 172.21.15.99:0/2054968491 conn(0x7fab6008afd0 msgr2=0x7fab600ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:14.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 shutdown_connections 2024-03-29T09:11:14.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.611+0000 7fab67800700 1 -- 172.21.15.99:0/2054968491 wait complete. 2024-03-29T09:11:14.629 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-03-29T09:11:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.822+0000 7f0cecc00700 1 Processor -- start 2024-03-29T09:11:14.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.822+0000 7f0cecc00700 1 -- start start 2024-03-29T09:11:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0cecc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce8160770 0x7f0ce80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0cecc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0ce80d3550 0x7f0ce80ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0cecc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce80d1520 con 0x7f0ce80d0d60 2024-03-29T09:11:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0cecc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce80d1800 con 0x7f0ce80d3550 2024-03-29T09:11:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0cecc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce80d1ae0 con 0x7f0ce8160770 2024-03-29T09:11:14.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0ce6800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0ce80d3550 0x7f0ce80ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0ce5e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce8160770 0x7f0ce80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0ce6800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0ce80d3550 0x7f0ce80ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:36522/0 (socket says 172.21.15.99:36522) 2024-03-29T09:11:14.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.823+0000 7f0ce6800700 1 -- 172.21.15.99:0/474150428 learned_addr learned my addr 172.21.15.99:0/474150428 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce6800700 1 -- 172.21.15.99:0/474150428 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce8160770 msgr2=0x7f0ce80b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce6800700 1 --2- 172.21.15.99:0/474150428 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce8160770 0x7f0ce80b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce6800700 1 -- 172.21.15.99:0/474150428 >> v1:172.21.15.133:6789/0 conn(0x7f0ce80d0d60 legacy=0x7f0ce80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce6800700 1 -- 172.21.15.99:0/474150428 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80d1dc0 con 0x7f0ce80d3550 2024-03-29T09:11:14.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce4a00700 1 -- 172.21.15.99:0/474150428 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776397063 0 0) 0x7f0ce80d1520 con 0x7f0ce80d0d60 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce6800700 1 --2- 172.21.15.99:0/474150428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d3550 0x7f0ce80ce080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009e50 tx=0x7f0ce000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7fc3b27c165ed7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce4a00700 1 -- 172.21.15.99:0/474150428 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000e070 con 0x7f0ce80d3550 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce4a00700 1 -- 172.21.15.99:0/474150428 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce000b910 con 0x7f0ce80d3550 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.824+0000 7f0ce4a00700 1 -- 172.21.15.99:0/474150428 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000bcc0 con 0x7f0ce80d3550 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 -- 172.21.15.99:0/474150428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d3550 msgr2=0x7f0ce80ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 --2- 172.21.15.99:0/474150428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d3550 0x7f0ce80ce080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009e50 tx=0x7f0ce000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 -- 172.21.15.99:0/474150428 shutdown_connections 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 --2- 172.21.15.99:0/474150428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d3550 0x7f0ce80ce080 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 --2- 172.21.15.99:0/474150428 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce8160770 0x7f0ce80b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 -- 172.21.15.99:0/474150428 >> 172.21.15.99:0/474150428 conn(0x7f0ce808afd0 msgr2=0x7f0ce809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 -- 172.21.15.99:0/474150428 shutdown_connections 2024-03-29T09:11:14.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.825+0000 7f0cecc00700 1 -- 172.21.15.99:0/474150428 wait complete. 2024-03-29T09:11:14.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.827+0000 7f0cecc00700 1 Processor -- start 2024-03-29T09:11:14.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.827+0000 7f0cecc00700 1 -- start start 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0cecc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d0d60 0x7f0ce8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0cecc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce80d3550 0x7f0ce81f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0cecc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ce8160770 0x7f0ce80b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0cecc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce815a410 con 0x7f0ce8160770 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0cecc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce81f3a90 con 0x7f0ce80d0d60 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0cecc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce814c010 con 0x7f0ce80d3550 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0ce6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ce8160770 0x7f0ce80b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0ce6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ce8160770 0x7f0ce80b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35654/0 (socket says 172.21.15.99:35654) 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0ce6800700 1 -- 172.21.15.99:0/2447758339 learned_addr learned my addr 172.21.15.99:0/2447758339 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:14.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0ce5400700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce80d3550 0x7f0ce81f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.828+0000 7f0ce5e00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d0d60 0x7f0ce8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0ce6800700 1 -- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce80d3550 msgr2=0x7f0ce81f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0ce6800700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce80d3550 0x7f0ce81f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0ce6800700 1 -- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d0d60 msgr2=0x7f0ce8159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0ce6800700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d0d60 0x7f0ce8159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0ce6800700 1 -- 172.21.15.99:0/2447758339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0009b30 con 0x7f0ce8160770 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0ce6800700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ce8160770 0x7f0ce80b8200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000b680 tx=0x7f0ce0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0cde000700 1 -- 172.21.15.99:0/2447758339 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0004870 con 0x7f0ce8160770 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0cde000700 1 -- 172.21.15.99:0/2447758339 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce00092e0 con 0x7f0ce8160770 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce814c2f0 con 0x7f0ce8160770 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0cde000700 1 -- 172.21.15.99:0/2447758339 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce001c660 con 0x7f0ce8160770 2024-03-29T09:11:14.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.829+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce814c8b0 con 0x7f0ce8160770 2024-03-29T09:11:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.830+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce814cdd0 con 0x7f0ce8160770 2024-03-29T09:11:14.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.831+0000 7f0cde000700 1 -- 172.21.15.99:0/2447758339 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0ce001c7c0 con 0x7f0ce8160770 2024-03-29T09:11:14.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.832+0000 7f0cde000700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0cc806ac90 0x7f0cc806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:14.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.833+0000 7f0cde000700 1 -- 172.21.15.99:0/2447758339 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f0ce008ae80 con 0x7f0ce8160770 2024-03-29T09:11:14.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.833+0000 7f0ce5e00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0cc806ac90 0x7f0cc806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:14.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.833+0000 7f0ce5e00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0cc806ac90 0x7f0cc806d150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009b40 tx=0x7f0cd8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:14.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:14.836+0000 7f0cde000700 1 -- 172.21.15.99:0/2447758339 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0ce814cdd0 con 0x7f0ce8160770 2024-03-29T09:11:15.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.042+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f0ce814cdd0 con 0x7f0ce8160770 2024-03-29T09:11:15.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.042+0000 7f0cde000700 1 -- 172.21.15.99:0/2447758339 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7f0ce814cdd0 con 0x7f0ce8160770 2024-03-29T09:11:15.042 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0cc806ac90 msgr2=0x7f0cc806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0cc806ac90 0x7f0cc806d150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009b40 tx=0x7f0cd8009210 comp rx=0 tx=0).stop 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ce8160770 msgr2=0x7f0ce80b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ce8160770 0x7f0ce80b8200 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000b680 tx=0x7f0ce0004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 shutdown_connections 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0cc806ac90 0x7f0cc806d150 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0ce8160770 0x7f0ce80b8200 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ce80d0d60 0x7f0ce8159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 --2- 172.21.15.99:0/2447758339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ce80d3550 0x7f0ce81f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.044+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 >> 172.21.15.99:0/2447758339 conn(0x7f0ce808afd0 msgr2=0x7f0ce8073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.045+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 shutdown_connections 2024-03-29T09:11:15.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.045+0000 7f0cecc00700 1 -- 172.21.15.99:0/2447758339 wait complete. 2024-03-29T09:11:15.062 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:11:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.251+0000 7f9d38400700 1 Processor -- start 2024-03-29T09:11:15.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.252+0000 7f9d38400700 1 -- start start 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d38400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015d690 0x7f9d300cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d38400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d3015da70 0x7f9d300cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d38400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d300d1740 0x7f9d300d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d38400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d300d2130 con 0x7f9d3015da70 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d38400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d300afde0 con 0x7f9d300d1740 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d38400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300cf660 con 0x7f9d3015d690 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d3015da70 0x7f9d300cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f9d3015da70 0x7f9d300cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35660/0 (socket says 172.21.15.99:35660) 2024-03-29T09:11:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 -- 172.21.15.99:0/1163129967 learned_addr learned my addr 172.21.15.99:0/1163129967 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d36400700 1 --2- 172.21.15.99:0/1163129967 >> v2:172.21.15.133:3300/0 conn(0x7f9d300d1740 0x7f9d300d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.254 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35a00700 1 --2- 172.21.15.99:0/1163129967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015d690 0x7f9d300cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 -- 172.21.15.99:0/1163129967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015d690 msgr2=0x7f9d300cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 --2- 172.21.15.99:0/1163129967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015d690 0x7f9d300cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 -- 172.21.15.99:0/1163129967 >> v2:172.21.15.133:3300/0 conn(0x7f9d300d1740 msgr2=0x7f9d300d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 --2- 172.21.15.99:0/1163129967 >> v2:172.21.15.133:3300/0 conn(0x7f9d300d1740 0x7f9d300d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.253+0000 7f9d35000700 1 -- 172.21.15.99:0/1163129967 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d300d01d0 con 0x7f9d3015da70 2024-03-29T09:11:15.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.254+0000 7f9d35000700 1 --2- 172.21.15.99:0/1163129967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d3015da70 0x7f9d300cf120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9d300ca1b0 tx=0x7f9d1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f77f5387b3db1b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.254+0000 7f9d2fe00700 1 -- 172.21.15.99:0/1163129967 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c011830 con 0x7f9d3015da70 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.254+0000 7f9d2fe00700 1 -- 172.21.15.99:0/1163129967 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c011990 con 0x7f9d3015da70 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.254+0000 7f9d2fe00700 1 -- 172.21.15.99:0/1163129967 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c011d20 con 0x7f9d3015da70 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.254+0000 7f9d38400700 1 -- 172.21.15.99:0/1163129967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d3015da70 msgr2=0x7f9d300cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.254+0000 7f9d38400700 1 --2- 172.21.15.99:0/1163129967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d3015da70 0x7f9d300cf120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9d300ca1b0 tx=0x7f9d1c009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.255+0000 7f9d38400700 1 -- 172.21.15.99:0/1163129967 shutdown_connections 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.255+0000 7f9d38400700 1 --2- 172.21.15.99:0/1163129967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015d690 0x7f9d300cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.255+0000 7f9d38400700 1 --2- 172.21.15.99:0/1163129967 >> v2:172.21.15.133:3300/0 conn(0x7f9d300d1740 0x7f9d300d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.255+0000 7f9d38400700 1 --2- 172.21.15.99:0/1163129967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d3015da70 0x7f9d300cf120 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.255+0000 7f9d38400700 1 -- 172.21.15.99:0/1163129967 >> 172.21.15.99:0/1163129967 conn(0x7f9d3009c480 msgr2=0x7f9d300d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.255+0000 7f9d38400700 1 -- 172.21.15.99:0/1163129967 shutdown_connections 2024-03-29T09:11:15.256 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.255+0000 7f9d38400700 1 -- 172.21.15.99:0/1163129967 wait complete. 2024-03-29T09:11:15.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.256+0000 7f9d38400700 1 Processor -- start 2024-03-29T09:11:15.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.257+0000 7f9d38400700 1 -- start start 2024-03-29T09:11:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.257+0000 7f9d38400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d300d1740 0x7f9d300ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.257+0000 7f9d38400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d3015d690 0x7f9d300b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.257+0000 7f9d38400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015da70 0x7f9d3015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.257+0000 7f9d38400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300cd890 con 0x7f9d300d1740 2024-03-29T09:11:15.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.257+0000 7f9d38400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d301f4020 con 0x7f9d3015d690 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d38400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300cd2d0 con 0x7f9d3015da70 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d300d1740 0x7f9d300ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d300d1740 0x7f9d300ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35676/0 (socket says 172.21.15.99:35676) 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 -- 172.21.15.99:0/4205895523 learned_addr learned my addr 172.21.15.99:0/4205895523 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35000700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d3015d690 0x7f9d300b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 -- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015da70 msgr2=0x7f9d3015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015da70 0x7f9d3015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 -- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d3015d690 msgr2=0x7f9d300b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d3015d690 0x7f9d300b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 -- 172.21.15.99:0/4205895523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c009550 con 0x7f9d300d1740 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35000700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d3015d690 0x7f9d300b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.258+0000 7f9d35a00700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d300d1740 0x7f9d300ca220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400e960 tx=0x7f9d2400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.259+0000 7f9d2d600700 1 -- 172.21.15.99:0/4205895523 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24009710 con 0x7f9d300d1740 2024-03-29T09:11:15.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.259+0000 7f9d2d600700 1 -- 172.21.15.99:0/4205895523 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2400c850 con 0x7f9d300d1740 2024-03-29T09:11:15.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.259+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d300cd5b0 con 0x7f9d300d1740 2024-03-29T09:11:15.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.259+0000 7f9d2d600700 1 -- 172.21.15.99:0/4205895523 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2400cbc0 con 0x7f9d300d1740 2024-03-29T09:11:15.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.259+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d300ca760 con 0x7f9d300d1740 2024-03-29T09:11:15.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.260+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d300cda80 con 0x7f9d300d1740 2024-03-29T09:11:15.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.261+0000 7f9d2d600700 1 -- 172.21.15.99:0/4205895523 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9d2400cd20 con 0x7f9d300d1740 2024-03-29T09:11:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.261+0000 7f9d2d600700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9d1006ac90 0x7f9d1006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.262+0000 7f9d2d600700 1 -- 172.21.15.99:0/4205895523 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f9d24089620 con 0x7f9d300d1740 2024-03-29T09:11:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.262+0000 7f9d35000700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9d1006ac90 0x7f9d1006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.262+0000 7f9d35000700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9d1006ac90 0x7f9d1006d150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d300ca1b0 tx=0x7f9d1c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:15.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.266+0000 7f9d2d600700 1 -- 172.21.15.99:0/4205895523 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9d240545a0 con 0x7f9d300d1740 2024-03-29T09:11:15.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.486+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d300677a0 con 0x7f9d300d1740 2024-03-29T09:11:15.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.486+0000 7f9d2d600700 1 -- 172.21.15.99:0/4205895523 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9d24057bc0 con 0x7f9d300d1740 2024-03-29T09:11:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.488+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9d1006ac90 msgr2=0x7f9d1006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.488+0000 7f9d38400700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9d1006ac90 0x7f9d1006d150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d300ca1b0 tx=0x7f9d1c008030 comp rx=0 tx=0).stop 2024-03-29T09:11:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d300d1740 msgr2=0x7f9d300ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d300d1740 0x7f9d300ca220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400e960 tx=0x7f9d2400ed20 comp rx=0 tx=0).stop 2024-03-29T09:11:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 shutdown_connections 2024-03-29T09:11:15.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9d1006ac90 0x7f9d1006d150 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9d3015da70 0x7f9d3015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9d300d1740 0x7f9d300ca220 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 --2- 172.21.15.99:0/4205895523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d3015d690 0x7f9d300b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 >> 172.21.15.99:0/4205895523 conn(0x7f9d3009c480 msgr2=0x7f9d300c9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:15.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 shutdown_connections 2024-03-29T09:11:15.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.489+0000 7f9d38400700 1 -- 172.21.15.99:0/4205895523 wait complete. 2024-03-29T09:11:15.490 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:11:15.507 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:11:15.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:15.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:15.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[61557]: fuse finished with error 0 and tester_r 0 2024-03-29T09:11:15.573 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:11:15.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:11:15.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:15.595 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:11:15.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-03-29T09:11:15.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.914+0000 7efe96400700 1 Processor -- start 2024-03-29T09:11:15.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.914+0000 7efe96400700 1 -- start start 2024-03-29T09:11:15.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.915+0000 7efe96400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe900b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.915+0000 7efe96400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe900d0d60 0x7efe900d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.915+0000 7efe96400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe900d3810 con 0x7efe900d3380 2024-03-29T09:11:15.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.915+0000 7efe96400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe900d1520 con 0x7efe900d0d60 2024-03-29T09:11:15.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.915+0000 7efe96400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe900d1800 con 0x7efe90160770 2024-03-29T09:11:15.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.915+0000 7efe8ea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe900d0d60 0x7efe900d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8ea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efe900d0d60 0x7efe900d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35678/0 (socket says 172.21.15.99:35678) 2024-03-29T09:11:15.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8ea00700 1 -- 172.21.15.99:0/4169944400 learned_addr learned my addr 172.21.15.99:0/4169944400 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:15.916 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8f400700 1 --2- 172.21.15.99:0/4169944400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe900b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8ea00700 1 -- 172.21.15.99:0/4169944400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 msgr2=0x7efe900b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8ea00700 1 --2- 172.21.15.99:0/4169944400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe900b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8ea00700 1 -- 172.21.15.99:0/4169944400 >> v1:172.21.15.99:6789/0 conn(0x7efe900d3380 legacy=0x7efe900cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8ea00700 1 -- 172.21.15.99:0/4169944400 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe900d1ae0 con 0x7efe900d0d60 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8e000700 1 -- 172.21.15.99:0/4169944400 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63780708 0 0) 0x7efe900d3810 con 0x7efe900d3380 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.916+0000 7efe8ea00700 1 --2- 172.21.15.99:0/4169944400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d0d60 0x7efe900d2e40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0098f0 tx=0x7efe7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77069f62cbae02c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe8e000700 1 -- 172.21.15.99:0/4169944400 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c011830 con 0x7efe900d0d60 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe8e000700 1 -- 172.21.15.99:0/4169944400 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c011990 con 0x7efe900d0d60 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe8e000700 1 -- 172.21.15.99:0/4169944400 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c011d20 con 0x7efe900d0d60 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 -- 172.21.15.99:0/4169944400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d0d60 msgr2=0x7efe900d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 --2- 172.21.15.99:0/4169944400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d0d60 0x7efe900d2e40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0098f0 tx=0x7efe7c009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:15.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 -- 172.21.15.99:0/4169944400 shutdown_connections 2024-03-29T09:11:15.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 --2- 172.21.15.99:0/4169944400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe900b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 --2- 172.21.15.99:0/4169944400 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d0d60 0x7efe900d2e40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 -- 172.21.15.99:0/4169944400 >> 172.21.15.99:0/4169944400 conn(0x7efe9008afd0 msgr2=0x7efe9009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:15.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 -- 172.21.15.99:0/4169944400 shutdown_connections 2024-03-29T09:11:15.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.917+0000 7efe96400700 1 -- 172.21.15.99:0/4169944400 wait complete. 2024-03-29T09:11:15.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.919+0000 7efe96400700 1 Processor -- start 2024-03-29T09:11:15.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.919+0000 7efe96400700 1 -- start start 2024-03-29T09:11:15.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe96400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe900d0d60 0x7efe901f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe96400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d3380 0x7efe900d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe96400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe9014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe96400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900b7d60 con 0x7efe900d3380 2024-03-29T09:11:15.920 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe96400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900b5d50 con 0x7efe900d0d60 2024-03-29T09:11:15.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe96400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe901f6270 con 0x7efe90160770 2024-03-29T09:11:15.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d3380 0x7efe900d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe8ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d3380 0x7efe900d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35688/0 (socket says 172.21.15.99:35688) 2024-03-29T09:11:15.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe8ea00700 1 -- 172.21.15.99:0/1782649047 learned_addr learned my addr 172.21.15.99:0/1782649047 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:15.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe8fe00700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe9014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.921 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.920+0000 7efe8f400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe900d0d60 0x7efe901f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe8ea00700 1 -- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 msgr2=0x7efe9014c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe8ea00700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe9014c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe8ea00700 1 -- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe900d0d60 msgr2=0x7efe901f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe8ea00700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe900d0d60 0x7efe901f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe8ea00700 1 -- 172.21.15.99:0/1782649047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c009550 con 0x7efe900d3380 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe8ea00700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d3380 0x7efe900d0000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009c40 tx=0x7efe7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe87400700 1 -- 172.21.15.99:0/1782649047 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c004340 con 0x7efe900d3380 2024-03-29T09:11:15.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe87400700 1 -- 172.21.15.99:0/1782649047 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c01f070 con 0x7efe900d3380 2024-03-29T09:11:15.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe901f64e0 con 0x7efe900d3380 2024-03-29T09:11:15.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe87400700 1 -- 172.21.15.99:0/1782649047 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0036a0 con 0x7efe900d3380 2024-03-29T09:11:15.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.921+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe900b5f60 con 0x7efe900d3380 2024-03-29T09:11:15.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.922+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe900678b0 con 0x7efe900d3380 2024-03-29T09:11:15.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.923+0000 7efe87400700 1 -- 172.21.15.99:0/1782649047 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efe7c021030 con 0x7efe900d3380 2024-03-29T09:11:15.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.924+0000 7efe87400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe7006ac90 0x7efe7006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:15.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.924+0000 7efe87400700 1 -- 172.21.15.99:0/1782649047 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7efe7c08a9a0 con 0x7efe900d3380 2024-03-29T09:11:15.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.924+0000 7efe8f400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe7006ac90 0x7efe7006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:15.925 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.925+0000 7efe8f400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe7006ac90 0x7efe7006d150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efe901f5c90 tx=0x7efe80009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:15.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:15.929+0000 7efe87400700 1 -- 172.21.15.99:0/1782649047 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efe7c055820 con 0x7efe900d3380 2024-03-29T09:11:16.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.220+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7efe900cd4c0 con 0x7efe900d3380 2024-03-29T09:11:16.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.225+0000 7efe87400700 1 -- 172.21.15.99:0/1782649047 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v17) v1 ==== 211+0+0 (secure 0 0 0) 0x7efe7c058e40 con 0x7efe900d3380 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.227+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe7006ac90 msgr2=0x7efe7006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.227+0000 7efe96400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe7006ac90 0x7efe7006d150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efe901f5c90 tx=0x7efe80009210 comp rx=0 tx=0).stop 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.227+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d3380 msgr2=0x7efe900d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.227+0000 7efe96400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d3380 0x7efe900d0000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009c40 tx=0x7efe7c004750 comp rx=0 tx=0).stop 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.227+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 shutdown_connections 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.227+0000 7efe96400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efe7006ac90 0x7efe7006d150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.227+0000 7efe96400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe90160770 0x7efe9014c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.228+0000 7efe96400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe900d0d60 0x7efe901f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.228 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.228+0000 7efe96400700 1 --2- 172.21.15.99:0/1782649047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe900d3380 0x7efe900d0000 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.228+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 >> 172.21.15.99:0/1782649047 conn(0x7efe9008afd0 msgr2=0x7efe90073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:16.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.228+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 shutdown_connections 2024-03-29T09:11:16.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.228+0000 7efe96400700 1 -- 172.21.15.99:0/1782649047 wait complete. 2024-03-29T09:11:16.229 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:11:16.246 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:11:16.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.434+0000 7f568de00700 1 Processor -- start 2024-03-29T09:11:16.434 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.434+0000 7f568de00700 1 -- start start 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f568de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f56881608e0 0x7f56880d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f568de00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f56880d1b20 0x7f56880ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f568de00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880cddc0 con 0x7f5688160cc0 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f568de00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880ce0a0 con 0x7f56881608e0 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f568de00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880ce380 con 0x7f56880d1b20 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f5686a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f56881608e0 0x7f56880d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f5686a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f56881608e0 0x7f56880d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35700/0 (socket says 172.21.15.99:35700) 2024-03-29T09:11:16.436 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.435+0000 7f5686a00700 1 -- 172.21.15.99:0/1253167442 learned_addr learned my addr 172.21.15.99:0/1253167442 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5687400700 1 --2- 172.21.15.99:0/1253167442 >> v2:172.21.15.133:3300/0 conn(0x7f56880d1b20 0x7f56880ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5686a00700 1 -- 172.21.15.99:0/1253167442 >> v2:172.21.15.133:3300/0 conn(0x7f56880d1b20 msgr2=0x7f56880ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5686a00700 1 --2- 172.21.15.99:0/1253167442 >> v2:172.21.15.133:3300/0 conn(0x7f56880d1b20 0x7f56880ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5686a00700 1 -- 172.21.15.99:0/1253167442 >> v1:172.21.15.133:6789/0 conn(0x7f5688160cc0 legacy=0x7f56880d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5686a00700 1 -- 172.21.15.99:0/1253167442 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880ce660 con 0x7f56881608e0 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5686a00700 1 --2- 172.21.15.99:0/1253167442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880d0150 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f568809cb70 tx=0x7f5678009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee4781eb783abd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5685600700 1 -- 172.21.15.99:0/1253167442 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678004230 con 0x7f56881608e0 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5685600700 1 -- 172.21.15.99:0/1253167442 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678004390 con 0x7f56881608e0 2024-03-29T09:11:16.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.436+0000 7f5685600700 1 -- 172.21.15.99:0/1253167442 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56780036a0 con 0x7f56881608e0 2024-03-29T09:11:16.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 -- 172.21.15.99:0/1253167442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 msgr2=0x7f56880d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:16.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 --2- 172.21.15.99:0/1253167442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880d0150 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f568809cb70 tx=0x7f5678009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 -- 172.21.15.99:0/1253167442 shutdown_connections 2024-03-29T09:11:16.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 --2- 172.21.15.99:0/1253167442 >> v2:172.21.15.133:3300/0 conn(0x7f56880d1b20 0x7f56880ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 --2- 172.21.15.99:0/1253167442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880d0150 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 -- 172.21.15.99:0/1253167442 >> 172.21.15.99:0/1253167442 conn(0x7f568809c490 msgr2=0x7f56880d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:16.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 -- 172.21.15.99:0/1253167442 shutdown_connections 2024-03-29T09:11:16.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.437+0000 7f568de00700 1 -- 172.21.15.99:0/1253167442 wait complete. 2024-03-29T09:11:16.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.438+0000 7f568de00700 1 Processor -- start 2024-03-29T09:11:16.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.439+0000 7f568de00700 1 -- start start 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.439+0000 7f568de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56880d1b20 0x7f56880ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.439+0000 7f568de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f568de00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5688160cc0 0x7f5688148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f568de00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568815cc10 con 0x7f56881608e0 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f568de00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880ca800 con 0x7f56880d1b20 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f568de00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880ccbf0 con 0x7f5688160cc0 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:35714/0 (socket says 172.21.15.99:35714) 2024-03-29T09:11:16.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 -- 172.21.15.99:0/1054227842 learned_addr learned my addr 172.21.15.99:0/1054227842 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686a00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56880d1b20 0x7f56880ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5687400700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5688160cc0 0x7f5688148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 -- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5688160cc0 msgr2=0x7f5688148780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5688160cc0 0x7f5688148780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 -- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56880d1b20 msgr2=0x7f56880ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56880d1b20 0x7f56880ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 -- 172.21.15.99:0/1054227842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678009550 con 0x7f56881608e0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686a00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56880d1b20 0x7f56880ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.440+0000 7f5686000700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880cc3a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f567400b450 tx=0x7f567400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.441+0000 7f567f400700 1 -- 172.21.15.99:0/1054227842 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674011830 con 0x7f56881608e0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.441+0000 7f567f400700 1 -- 172.21.15.99:0/1054227842 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5674011990 con 0x7f56881608e0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.441+0000 7f567f400700 1 -- 172.21.15.99:0/1054227842 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674011d40 con 0x7f56881608e0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.441+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56880cced0 con 0x7f56881608e0 2024-03-29T09:11:16.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.441+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5688155a70 con 0x7f56881608e0 2024-03-29T09:11:16.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.442+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56880677a0 con 0x7f56881608e0 2024-03-29T09:11:16.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.443+0000 7f567f400700 1 -- 172.21.15.99:0/1054227842 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f567401a400 con 0x7f56881608e0 2024-03-29T09:11:16.443 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.443+0000 7f567f400700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f566806ac90 0x7f566806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:16.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.444+0000 7f567f400700 1 -- 172.21.15.99:0/1054227842 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f5674089360 con 0x7f56881608e0 2024-03-29T09:11:16.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.444+0000 7f5686a00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f566806ac90 0x7f566806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:16.444 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.444+0000 7f5686a00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f566806ac90 0x7f566806d150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5678009130 tx=0x7f5678008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:16.447 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.448+0000 7f567f400700 1 -- 172.21.15.99:0/1054227842 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f56740541d0 con 0x7f56881608e0 2024-03-29T09:11:16.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.704+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f56880d0150 con 0x7f56881608e0 2024-03-29T09:11:16.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.704+0000 7f567f400700 1 -- 172.21.15.99:0/1054227842 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+82 (secure 0 0 0) 0x7f56740577f0 con 0x7f56881608e0 2024-03-29T09:11:16.704 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f566806ac90 msgr2=0x7f566806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f566806ac90 0x7f566806d150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5678009130 tx=0x7f5678008030 comp rx=0 tx=0).stop 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 msgr2=0x7f56880cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880cc3a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f567400b450 tx=0x7f567400b810 comp rx=0 tx=0).stop 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 shutdown_connections 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f566806ac90 0x7f566806d150 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5688160cc0 0x7f5688148780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56880d1b20 0x7f56880ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 --2- 172.21.15.99:0/1054227842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f56881608e0 0x7f56880cc3a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 >> 172.21.15.99:0/1054227842 conn(0x7f568809c490 msgr2=0x7f56880c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:16.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 shutdown_connections 2024-03-29T09:11:16.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:16.707+0000 7f568de00700 1 -- 172.21.15.99:0/1054227842 wait complete. 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:11:16.725 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:16.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:16.726 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:11:16.768 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:11:16.768 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:11:16.768 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:11:16.768 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:16.768 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42111sec preferred_lft 42111sec 2024-03-29T09:11:16.769 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::3caa:acff:feac:fccd/64 scope link 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5832:eeff:fe0b:ee92/64 scope link 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:11:16.770 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether fe:d0:b6:f6:cb:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:11:16.771 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::fcd0:b6ff:fef6:cba2/64 scope link 2024-03-29T09:11:16.771 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:16.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:16.771 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:11:16.870 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:11:16.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:16.874 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:11:16.918 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:11:16.918 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:11:16.918 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:11:16.918 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:11:16.918 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:11:16.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:16.919 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:11:16.919 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:11:16.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:11:16.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:11:16.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:11:16.919 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:11:17.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:17.121 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:11:17.162 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:11:17.162 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T09:11:17.163 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:17.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:17.163 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:11:17.269 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T09:11:17.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:11:17.269 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:17.310 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:17.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:11:17.310 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:17.391 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:11:17.496 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:11:17.496 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:11:17.558 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:11:17.561 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:11:17.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:17.561 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:11:17.602 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:11:17.602 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T09:11:17.646 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:11:17.711 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:11:17.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:11:17.774 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:11:17.777 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:11:17.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:17.777 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:11:17.810 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-29T09:11:17.811+0000 7f1254fb95c0 -1 init, newargv = 0x558b4609fe30 newargc=15 2024-03-29T09:11:17.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[62772]: starting ceph client 2024-03-29T09:11:17.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[62772]: starting fuse 2024-03-29T09:11:17.824 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:11:17.825 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T09:11:17.825 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:11:17.826 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:17.827 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-29T09:11:17.827 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:11:18.003 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.62772.asok 2024-03-29T09:11:18.181 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62772.asok 2024-03-29T09:11:18.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:18.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62772.asok status 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "62772", 2024-03-29T09:11:18.399 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 5847, 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 5847 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 709223702 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:18.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 709223702 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.5847 192.168.0.1:0/709223702", 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/709223702", 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 17, 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 42, 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:11:18.401 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:11:18.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:18.413 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:11:18.413 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:18.455 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:11:18.456 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:18.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:18.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:18.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:18.561 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-03-29T09:11:18.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:18.605 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-03-29T09:11:18.693 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:18.693 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:18.734 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:18.964 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.964+0000 7fb675000700 1 Processor -- start 2024-03-29T09:11:18.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.965+0000 7fb675000700 1 -- start start 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb675000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 0x7fb67015b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb675000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6700b36f0 0x7fb6700b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb675000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67015c220 con 0x7fb67015ae70 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb675000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6700b40c0 con 0x7fb6700b36f0 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb675000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67015b8f0 con 0x7fb6700b1260 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb66de00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb67015ae70 0x7fb6700b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47374/0 (socket says 172.21.15.99:47374) 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb66de00700 1 -- 172.21.15.99:0/2425493099 learned_addr learned my addr 172.21.15.99:0/2425493099 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:18.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb667e00700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 0x7fb67015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb66e800700 1 --2- 172.21.15.99:0/2425493099 >> v2:172.21.15.133:3300/0 conn(0x7fb6700b36f0 0x7fb6700b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228322712 0 0) 0x7fb67015c220 con 0x7fb67015ae70 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.966+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6540035b0 con 0x7fb67015ae70 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1813794558 0 0) 0x7fb6540035b0 con 0x7fb67015ae70 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 msgr2=0x7fb67015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 0x7fb67015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 >> v2:172.21.15.133:3300/0 conn(0x7fb6700b36f0 msgr2=0x7fb6700b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 --2- 172.21.15.99:0/2425493099 >> v2:172.21.15.133:3300/0 conn(0x7fb6700b36f0 0x7fb6700b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67015bbd0 con 0x7fb67015ae70 2024-03-29T09:11:18.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fb660001670 con 0x7fb67015ae70 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 >> v1:172.21.15.99:6789/0 conn(0x7fb67015ae70 legacy=0x7fb6700b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb654003a00 0x7fb654003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 0x7fb654004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654004d90 0x7fb654005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700b40c0 con 0x7fb654003a00 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67015b8f0 con 0x7fb654004d90 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65400ba10 con 0x7fb6700b1260 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66de00700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb654003a00 0x7fb654003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb667e00700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 0x7fb654004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:18.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.967+0000 7fb66e800700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654004d90 0x7fb654005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66de00700 1 -- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 msgr2=0x7fb654004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66de00700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 0x7fb654004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66de00700 1 -- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654004d90 msgr2=0x7fb654005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66de00700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654004d90 0x7fb654005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66de00700 1 -- 172.21.15.99:0/2425493099 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540035b0 con 0x7fb654003a00 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66e800700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654004d90 0x7fb654005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb667e00700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6700b1260 0x7fb654004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66de00700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb654003a00 0x7fb654003e20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c320 tx=0x7fb654005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a94f1cb26cacd4f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66000ce60 con 0x7fb654003a00 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660005960 con 0x7fb654003a00 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66d400700 1 -- 172.21.15.99:0/2425493099 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660005d10 con 0x7fb654003a00 2024-03-29T09:11:18.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb675000700 1 -- 172.21.15.99:0/2425493099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb654003a00 msgr2=0x7fb654003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb675000700 1 --2- 172.21.15.99:0/2425493099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb654003a00 0x7fb654003e20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c320 tx=0x7fb654005880 comp rx=0 tx=0).stop 2024-03-29T09:11:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.968+0000 7fb66e800700 1 -- 172.21.15.99:0/2425493099 reap_dead start 2024-03-29T09:11:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.969+0000 7fb675000700 1 -- 172.21.15.99:0/2425493099 shutdown_connections 2024-03-29T09:11:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.969+0000 7fb675000700 1 -- 172.21.15.99:0/2425493099 >> 172.21.15.99:0/2425493099 conn(0x7fb670066540 msgr2=0x7fb670066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.969+0000 7fb675000700 1 -- 172.21.15.99:0/2425493099 shutdown_connections 2024-03-29T09:11:18.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.969+0000 7fb675000700 1 -- 172.21.15.99:0/2425493099 wait complete. 2024-03-29T09:11:18.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.970+0000 7fb675000700 1 Processor -- start 2024-03-29T09:11:18.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.971+0000 7fb675000700 1 -- start start 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.971+0000 7fb675000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654003a00 0x7fb67013b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.971+0000 7fb675000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6701507e0 0x7fb670159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.971+0000 7fb675000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb670159ba0 0x7fb67013d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb675000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67014f3b0 con 0x7fb670159ba0 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb675000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67013b7a0 con 0x7fb654003a00 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb675000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb670150c60 con 0x7fb6701507e0 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb670159ba0 0x7fb67013d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb670159ba0 0x7fb67013d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43754/0 (socket says 172.21.15.99:43754) 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 -- 172.21.15.99:0/2614872281 learned_addr learned my addr 172.21.15.99:0/2614872281 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:18.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 -- 172.21.15.99:0/2614872281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6701507e0 msgr2=0x7fb670159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6701507e0 0x7fb670159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 -- 172.21.15.99:0/2614872281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654003a00 msgr2=0x7fb67013b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654003a00 0x7fb67013b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 -- 172.21.15.99:0/2614872281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67013ba40 con 0x7fb670159ba0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb66e800700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb670159ba0 0x7fb67013d790 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb670070190 tx=0x7fb6680104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.972+0000 7fb666000700 1 -- 172.21.15.99:0/2614872281 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66801b070 con 0x7fb670159ba0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.973+0000 7fb666000700 1 -- 172.21.15.99:0/2614872281 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66800f040 con 0x7fb670159ba0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.973+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67013bd20 con 0x7fb670159ba0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.973+0000 7fb666000700 1 -- 172.21.15.99:0/2614872281 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800e050 con 0x7fb670159ba0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.973+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67013c2e0 con 0x7fb670159ba0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.974+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6700677a0 con 0x7fb670159ba0 2024-03-29T09:11:18.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.975+0000 7fb666000700 1 -- 172.21.15.99:0/2614872281 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb668015440 con 0x7fb670159ba0 2024-03-29T09:11:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.976+0000 7fb666000700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb64c06ac90 0x7fb64c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:18.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.976+0000 7fb666000700 1 -- 172.21.15.99:0/2614872281 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb6680896d0 con 0x7fb670159ba0 2024-03-29T09:11:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.976+0000 7fb66de00700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb64c06ac90 0x7fb64c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:18.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.977+0000 7fb66de00700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb64c06ac90 0x7fb64c06d150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb660001930 tx=0x7fb66000f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:18.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:18.981+0000 7fb666000700 1 -- 172.21.15.99:0/2614872281 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb668054720 con 0x7fb670159ba0 2024-03-29T09:11:19.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.186+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6700b2c10 con 0x7fb670159ba0 2024-03-29T09:11:19.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.187+0000 7fb666000700 1 -- 172.21.15.99:0/2614872281 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7fb668057d40 con 0x7fb670159ba0 2024-03-29T09:11:19.188 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:19.188 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:19.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb64c06ac90 msgr2=0x7fb64c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb64c06ac90 0x7fb64c06d150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb660001930 tx=0x7fb66000f000 comp rx=0 tx=0).stop 2024-03-29T09:11:19.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb670159ba0 msgr2=0x7fb67013d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.190 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb670159ba0 0x7fb67013d790 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb670070190 tx=0x7fb6680104a0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 shutdown_connections 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb64c06ac90 0x7fb64c06d150 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb670159ba0 0x7fb67013d790 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb654003a00 0x7fb67013b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 --2- 172.21.15.99:0/2614872281 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6701507e0 0x7fb670159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 >> 172.21.15.99:0/2614872281 conn(0x7fb670066540 msgr2=0x7fb6701556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 shutdown_connections 2024-03-29T09:11:19.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.190+0000 7fb675000700 1 -- 172.21.15.99:0/2614872281 wait complete. 2024-03-29T09:11:19.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:10:42.037884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:10:42.900862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-03-29T09:10:53.257132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:11:19.209 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:11:19.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.402+0000 7fd51e800700 1 Processor -- start 2024-03-29T09:11:19.402 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.402+0000 7fd51e800700 1 -- start start 2024-03-29T09:11:19.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.403+0000 7fd51e800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd5180b3cc0 0x7fd5180d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.403+0000 7fd51e800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180d3810 con 0x7fd518160770 2024-03-29T09:11:19.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.403+0000 7fd51e800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180d1520 con 0x7fd5180d3380 2024-03-29T09:11:19.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.403+0000 7fd51e800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180b4140 con 0x7fd5180b3cc0 2024-03-29T09:11:19.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.403+0000 7fd517400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd518160770 0x7fd5180d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47382/0 (socket says 172.21.15.99:47382) 2024-03-29T09:11:19.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.403+0000 7fd517400700 1 -- 172.21.15.99:0/1197816059 learned_addr learned my addr 172.21.15.99:0/1197816059 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:19.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd50fe00700 1 --2- 172.21.15.99:0/1197816059 >> v2:172.21.15.133:3300/0 conn(0x7fd5180b3cc0 0x7fd5180d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252072244 0 0) 0x7fd5180d3810 con 0x7fd518160770 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4fc0035b0 con 0x7fd518160770 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305950850 0 0) 0x7fd4fc0035b0 con 0x7fd518160770 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 >> v2:172.21.15.133:3300/0 conn(0x7fd5180b3cc0 msgr2=0x7fd5180d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 --2- 172.21.15.99:0/1197816059 >> v2:172.21.15.133:3300/0 conn(0x7fd5180b3cc0 0x7fd5180d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 >> v1:172.21.15.133:6789/0 conn(0x7fd5180d3380 legacy=0x7fd5180ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180d1800 con 0x7fd518160770 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fd508001670 con 0x7fd518160770 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 >> v1:172.21.15.99:6789/0 conn(0x7fd518160770 legacy=0x7fd5180d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4fc003a00 0x7fd4fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc004850 0x7fd4fc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4fc005210 0x7fd4fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc00c080 con 0x7fd4fc003a00 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc00c1f0 con 0x7fd4fc004850 2024-03-29T09:11:19.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.404+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc00c3b0 con 0x7fd4fc005210 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517400700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4fc003a00 0x7fd4fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd50fe00700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc004850 0x7fd4fc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517e00700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4fc005210 0x7fd4fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517400700 1 -- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4fc005210 msgr2=0x7fd4fc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517400700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4fc005210 0x7fd4fc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517400700 1 -- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc004850 msgr2=0x7fd4fc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517400700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc004850 0x7fd4fc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517400700 1 -- 172.21.15.99:0/1197816059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0035b0 con 0x7fd4fc003a00 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517e00700 1 -- 172.21.15.99:0/1197816059 reap_dead start 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd517400700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4fc003a00 0x7fd4fc003e20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c240 tx=0x7fd4fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba694303bfbb378f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508007c10 con 0x7fd4fc003a00 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50800ce00 con 0x7fd4fc003a00 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.405+0000 7fd516a00700 1 -- 172.21.15.99:0/1197816059 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508005ba0 con 0x7fd4fc003a00 2024-03-29T09:11:19.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.406+0000 7fd51e800700 1 -- 172.21.15.99:0/1197816059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4fc003a00 msgr2=0x7fd4fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.406+0000 7fd51e800700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4fc003a00 0x7fd4fc003e20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd50800c240 tx=0x7fd4fc005880 comp rx=0 tx=0).stop 2024-03-29T09:11:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.406+0000 7fd51e800700 1 -- 172.21.15.99:0/1197816059 shutdown_connections 2024-03-29T09:11:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.406+0000 7fd51e800700 1 --2- 172.21.15.99:0/1197816059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd4fc003a00 0x7fd4fc003e20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.406+0000 7fd51e800700 1 -- 172.21.15.99:0/1197816059 >> 172.21.15.99:0/1197816059 conn(0x7fd51808afd0 msgr2=0x7fd51809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.406+0000 7fd51e800700 1 -- 172.21.15.99:0/1197816059 shutdown_connections 2024-03-29T09:11:19.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.406+0000 7fd51e800700 1 -- 172.21.15.99:0/1197816059 wait complete. 2024-03-29T09:11:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.408+0000 7fd51e800700 1 Processor -- start 2024-03-29T09:11:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.408+0000 7fd51e800700 1 -- start start 2024-03-29T09:11:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd51e800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc003a00 0x7fd5180cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd51e800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5180c6cc0 0x7fd518155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd51e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5180cc640 0x7fd5180ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd51e800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180b6d20 con 0x7fd5180cc640 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd51e800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180c9640 con 0x7fd4fc003a00 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd51e800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180cd000 con 0x7fd5180c6cc0 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd517e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5180cc640 0x7fd5180ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd517e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5180cc640 0x7fd5180ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43770/0 (socket says 172.21.15.99:43770) 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd517e00700 1 -- 172.21.15.99:0/790726559 learned_addr learned my addr 172.21.15.99:0/790726559 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd517400700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc003a00 0x7fd5180cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.409+0000 7fd50fe00700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5180c6cc0 0x7fd518155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd517e00700 1 -- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5180c6cc0 msgr2=0x7fd518155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd517e00700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5180c6cc0 0x7fd518155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd517e00700 1 -- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc003a00 msgr2=0x7fd5180cfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd517e00700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc003a00 0x7fd5180cfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd517e00700 1 -- 172.21.15.99:0/790726559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180c7d40 con 0x7fd5180cc640 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd517e00700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5180cc640 0x7fd5180ccac0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd51806b4d0 tx=0x7fd51000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd50f400700 1 -- 172.21.15.99:0/790726559 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51000ee70 con 0x7fd5180cc640 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd50f400700 1 -- 172.21.15.99:0/790726559 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510003c10 con 0x7fd5180cc640 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd50f400700 1 -- 172.21.15.99:0/790726559 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100050d0 con 0x7fd5180cc640 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5180c74a0 con 0x7fd5180cc640 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.410+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5180c7a60 con 0x7fd5180cc640 2024-03-29T09:11:19.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.411+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5180b6e60 con 0x7fd5180cc640 2024-03-29T09:11:19.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.412+0000 7fd50f400700 1 -- 172.21.15.99:0/790726559 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd510005230 con 0x7fd5180cc640 2024-03-29T09:11:19.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.413+0000 7fd50f400700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4f406ac90 0x7fd4f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.413+0000 7fd50f400700 1 -- 172.21.15.99:0/790726559 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fd5100893e0 con 0x7fd5180cc640 2024-03-29T09:11:19.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.413+0000 7fd517400700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4f406ac90 0x7fd4f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.414+0000 7fd517400700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4f406ac90 0x7fd4f406d150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd508000c00 tx=0x7fd508007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:19.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.417+0000 7fd50f400700 1 -- 172.21.15.99:0/790726559 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd5180b6e60 con 0x7fd5180cc640 2024-03-29T09:11:19.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.624+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5180b6e60 con 0x7fd5180cc640 2024-03-29T09:11:19.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.624+0000 7fd50f400700 1 -- 172.21.15.99:0/790726559 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5180b6e60 con 0x7fd5180cc640 2024-03-29T09:11:19.624 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:11:19.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4f406ac90 msgr2=0x7fd4f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4f406ac90 0x7fd4f406d150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd508000c00 tx=0x7fd508007000 comp rx=0 tx=0).stop 2024-03-29T09:11:19.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5180cc640 msgr2=0x7fd5180ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5180cc640 0x7fd5180ccac0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd51806b4d0 tx=0x7fd51000ed50 comp rx=0 tx=0).stop 2024-03-29T09:11:19.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 shutdown_connections 2024-03-29T09:11:19.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd4f406ac90 0x7fd4f406d150 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd5180cc640 0x7fd5180ccac0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4fc003a00 0x7fd5180cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 --2- 172.21.15.99:0/790726559 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5180c6cc0 0x7fd518155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.626+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 >> 172.21.15.99:0/790726559 conn(0x7fd51808afd0 msgr2=0x7fd51815f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:19.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.627+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 shutdown_connections 2024-03-29T09:11:19.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.627+0000 7fd51e800700 1 -- 172.21.15.99:0/790726559 wait complete. 2024-03-29T09:11:19.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:11:19.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.838+0000 7f355f200700 1 Processor -- start 2024-03-29T09:11:19.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.838+0000 7f355f200700 1 -- start start 2024-03-29T09:11:19.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.839+0000 7f355f200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35580ae280 0x7f355806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.839+0000 7f355f200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580af8f0 con 0x7f35580ae660 2024-03-29T09:11:19.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.839+0000 7f355f200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580aed70 con 0x7f35580d07d0 2024-03-29T09:11:19.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.839+0000 7f355f200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355815d7e0 con 0x7f35580ae280 2024-03-29T09:11:19.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3557e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f35580ae280 0x7f355806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3557400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f35580ae660 0x7f35580d00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47386/0 (socket says 172.21.15.99:47386) 2024-03-29T09:11:19.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3557400700 1 -- 172.21.15.99:0/2512827131 learned_addr learned my addr 172.21.15.99:0/2512827131 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3557e00700 1 -- 172.21.15.99:0/2512827131 >> v1:172.21.15.133:6789/0 conn(0x7f35580d07d0 legacy=0x7f3558071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3557e00700 1 -- 172.21.15.99:0/2512827131 >> v1:172.21.15.99:6789/0 conn(0x7f35580ae660 legacy=0x7f35580d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3557e00700 1 -- 172.21.15.99:0/2512827131 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580af050 con 0x7f35580ae280 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3557e00700 1 --2- 172.21.15.99:0/2512827131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f355806d4d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3548009840 tx=0x7f3548009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a656157421ea8fe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.840+0000 7f3556a00700 1 -- 172.21.15.99:0/2512827131 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548011830 con 0x7f35580ae280 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f3556a00700 1 -- 172.21.15.99:0/2512827131 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548011990 con 0x7f35580ae280 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f3556a00700 1 -- 172.21.15.99:0/2512827131 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548011d20 con 0x7f35580ae280 2024-03-29T09:11:19.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f355f200700 1 -- 172.21.15.99:0/2512827131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 msgr2=0x7f355806d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f355f200700 1 --2- 172.21.15.99:0/2512827131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f355806d4d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3548009840 tx=0x7f3548009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:19.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f355f200700 1 -- 172.21.15.99:0/2512827131 shutdown_connections 2024-03-29T09:11:19.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f355f200700 1 --2- 172.21.15.99:0/2512827131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f355806d4d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f355f200700 1 -- 172.21.15.99:0/2512827131 >> 172.21.15.99:0/2512827131 conn(0x7f3558066720 msgr2=0x7f3558066b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:19.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f355f200700 1 -- 172.21.15.99:0/2512827131 shutdown_connections 2024-03-29T09:11:19.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.841+0000 7f355f200700 1 -- 172.21.15.99:0/2512827131 wait complete. 2024-03-29T09:11:19.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.843+0000 7f355f200700 1 Processor -- start 2024-03-29T09:11:19.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.843+0000 7f355f200700 1 -- start start 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f355f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f35581499e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f355f200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35580ae660 0x7f355815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f355f200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35580d07d0 0x7f355815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f355f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558149f20 con 0x7f35580ae280 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f355f200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558146230 con 0x7f35580d07d0 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f355f200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c6a40 con 0x7f35580ae660 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f3557e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f35581499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f3557e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f35581499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43780/0 (socket says 172.21.15.99:43780) 2024-03-29T09:11:19.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f3557e00700 1 -- 172.21.15.99:0/1754016201 learned_addr learned my addr 172.21.15.99:0/1754016201 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f3557400700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35580ae660 0x7f355815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.844+0000 7f355d200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35580d07d0 0x7f355815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f3557e00700 1 -- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35580ae660 msgr2=0x7f355815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f3557e00700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35580ae660 0x7f355815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f3557e00700 1 -- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35580d07d0 msgr2=0x7f355815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f3557e00700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35580d07d0 0x7f355815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f3557e00700 1 -- 172.21.15.99:0/1754016201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480094a0 con 0x7f35580ae280 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f3557400700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35580ae660 0x7f355815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f355d200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35580d07d0 0x7f355815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f3557e00700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f35581499e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f35480090b0 tx=0x7f3548004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f354fe00700 1 -- 172.21.15.99:0/1754016201 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35480038a0 con 0x7f35580ae280 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f354fe00700 1 -- 172.21.15.99:0/1754016201 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354801f070 con 0x7f35580ae280 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3558146670 con 0x7f35580ae280 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f354fe00700 1 -- 172.21.15.99:0/1754016201 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548003c60 con 0x7f35580ae280 2024-03-29T09:11:19.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.845+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3558146c30 con 0x7f35580ae280 2024-03-29T09:11:19.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.846+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35580678b0 con 0x7f35580ae280 2024-03-29T09:11:19.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.847+0000 7f354fe00700 1 -- 172.21.15.99:0/1754016201 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f354801a430 con 0x7f35580ae280 2024-03-29T09:11:19.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.848+0000 7f354fe00700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f353806ac90 0x7f353806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:19.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.848+0000 7f354fe00700 1 -- 172.21.15.99:0/1754016201 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f3548021070 con 0x7f35580ae280 2024-03-29T09:11:19.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.848+0000 7f3557400700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f353806ac90 0x7f353806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:19.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.849+0000 7f3557400700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f353806ac90 0x7f353806d150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3544007fd0 tx=0x7f3544009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:19.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:19.852+0000 7f354fe00700 1 -- 172.21.15.99:0/1754016201 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3548056050 con 0x7f35580ae280 2024-03-29T09:11:20.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.058+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f355815d2f0 con 0x7f35580ae280 2024-03-29T09:11:20.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.058+0000 7f354fe00700 1 -- 172.21.15.99:0/1754016201 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3548059670 con 0x7f35580ae280 2024-03-29T09:11:20.058 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f353806ac90 msgr2=0x7f353806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f353806ac90 0x7f353806d150 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3544007fd0 tx=0x7f3544009040 comp rx=0 tx=0).stop 2024-03-29T09:11:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 msgr2=0x7f35581499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f35581499e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f35480090b0 tx=0x7f3548004750 comp rx=0 tx=0).stop 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 shutdown_connections 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f353806ac90 0x7f353806d150 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35580d07d0 0x7f355815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35580ae280 0x7f35581499e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 --2- 172.21.15.99:0/1754016201 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35580ae660 0x7f355815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 >> 172.21.15.99:0/1754016201 conn(0x7f3558066720 msgr2=0x7f355806db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 shutdown_connections 2024-03-29T09:11:20.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.061+0000 7f355f200700 1 -- 172.21.15.99:0/1754016201 wait complete. 2024-03-29T09:11:20.080 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:11:20.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.271+0000 7ff4c7400700 1 Processor -- start 2024-03-29T09:11:20.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.271+0000 7ff4c7400700 1 -- start start 2024-03-29T09:11:20.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c7400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4c0160770 0x7ff4c00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c7400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c00d0d60 0x7ff4c00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c7400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c00d3810 con 0x7ff4c00d3380 2024-03-29T09:11:20.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c7400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c00d1520 con 0x7ff4c0160770 2024-03-29T09:11:20.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c7400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00d1800 con 0x7ff4c00d0d60 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c4a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4c0160770 0x7ff4c00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c4a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff4c0160770 0x7ff4c00b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43794/0 (socket says 172.21.15.99:43794) 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.272+0000 7ff4c4a00700 1 -- 172.21.15.99:0/712375071 learned_addr learned my addr 172.21.15.99:0/712375071 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4bfe00700 1 --2- 172.21.15.99:0/712375071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c00d0d60 0x7ff4c00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4c4a00700 1 -- 172.21.15.99:0/712375071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c00d0d60 msgr2=0x7ff4c00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4c4a00700 1 --2- 172.21.15.99:0/712375071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c00d0d60 0x7ff4c00d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4c4a00700 1 -- 172.21.15.99:0/712375071 >> v1:172.21.15.133:6789/0 conn(0x7ff4c00d3380 legacy=0x7ff4c00cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4c4a00700 1 -- 172.21.15.99:0/712375071 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c00d1ae0 con 0x7ff4c0160770 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4c4a00700 1 --2- 172.21.15.99:0/712375071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c0160770 0x7ff4c00b3cc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098f0 tx=0x7ff4b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6713e6c4fad977a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4bf400700 1 -- 172.21.15.99:0/712375071 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4004230 con 0x7ff4c0160770 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4bf400700 1 -- 172.21.15.99:0/712375071 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b4004390 con 0x7ff4c0160770 2024-03-29T09:11:20.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.273+0000 7ff4bf400700 1 -- 172.21.15.99:0/712375071 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b40036a0 con 0x7ff4c0160770 2024-03-29T09:11:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 -- 172.21.15.99:0/712375071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c0160770 msgr2=0x7ff4c00b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 --2- 172.21.15.99:0/712375071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c0160770 0x7ff4c00b3cc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098f0 tx=0x7ff4b4009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 -- 172.21.15.99:0/712375071 shutdown_connections 2024-03-29T09:11:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 --2- 172.21.15.99:0/712375071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c0160770 0x7ff4c00b3cc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 --2- 172.21.15.99:0/712375071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c00d0d60 0x7ff4c00d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 -- 172.21.15.99:0/712375071 >> 172.21.15.99:0/712375071 conn(0x7ff4c008afd0 msgr2=0x7ff4c009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:20.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 -- 172.21.15.99:0/712375071 shutdown_connections 2024-03-29T09:11:20.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.274+0000 7ff4c7400700 1 -- 172.21.15.99:0/712375071 wait complete. 2024-03-29T09:11:20.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.276+0000 7ff4c7400700 1 Processor -- start 2024-03-29T09:11:20.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.276+0000 7ff4c7400700 1 -- start start 2024-03-29T09:11:20.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c00d0d60 0x7ff4c00ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c7400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4c00d3380 0x7ff4c01f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c7400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c0160770 0x7ff4c00b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c7400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00ca260 con 0x7ff4c00d0d60 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c7400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c01f2e20 con 0x7ff4c00d3380 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c7400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00caac0 con 0x7ff4c0160770 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c00d0d60 0x7ff4c00ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c00d0d60 0x7ff4c00ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43806/0 (socket says 172.21.15.99:43806) 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 -- 172.21.15.99:0/2628080802 learned_addr learned my addr 172.21.15.99:0/2628080802 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4bfe00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4c00d3380 0x7ff4c01f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c5400700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c0160770 0x7ff4c00b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 -- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c0160770 msgr2=0x7ff4c00b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c0160770 0x7ff4c00b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 -- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4c00d3380 msgr2=0x7ff4c01f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4c00d3380 0x7ff4c01f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4c4a00700 1 -- 172.21.15.99:0/2628080802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4009550 con 0x7ff4c00d0d60 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.277+0000 7ff4bfe00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4c00d3380 0x7ff4c01f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.278+0000 7ff4c4a00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c00d0d60 0x7ff4c00ccfd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009130 tx=0x7ff4b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.278+0000 7ff4bcc00700 1 -- 172.21.15.99:0/2628080802 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4003e10 con 0x7ff4c00d0d60 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.278+0000 7ff4bcc00700 1 -- 172.21.15.99:0/2628080802 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b400b660 con 0x7ff4c00d0d60 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.278+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c00b6c40 con 0x7ff4c00d0d60 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.278+0000 7ff4bcc00700 1 -- 172.21.15.99:0/2628080802 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400ba10 con 0x7ff4c00d0d60 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.278+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c00ca470 con 0x7ff4c00d0d60 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.279+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c00678b0 con 0x7ff4c00d0d60 2024-03-29T09:11:20.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.280+0000 7ff4bcc00700 1 -- 172.21.15.99:0/2628080802 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff4b400bb70 con 0x7ff4c00d0d60 2024-03-29T09:11:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.280+0000 7ff4bcc00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff4a006ac90 0x7ff4a006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.281+0000 7ff4bcc00700 1 -- 172.21.15.99:0/2628080802 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7ff4b408aef0 con 0x7ff4c00d0d60 2024-03-29T09:11:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.281+0000 7ff4bfe00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff4a006ac90 0x7ff4a006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.281+0000 7ff4bfe00700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff4a006ac90 0x7ff4a006d150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff4c01f6f60 tx=0x7ff4ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:20.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.285+0000 7ff4bcc00700 1 -- 172.21.15.99:0/2628080802 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff4b4055df0 con 0x7ff4c00d0d60 2024-03-29T09:11:20.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.487+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4c00ce1b0 con 0x7ff4c00d0d60 2024-03-29T09:11:20.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.488+0000 7ff4bcc00700 1 -- 172.21.15.99:0/2628080802 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4b401f020 con 0x7ff4c00d0d60 2024-03-29T09:11:20.488 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:11:20.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff4a006ac90 msgr2=0x7ff4a006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff4a006ac90 0x7ff4a006d150 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff4c01f6f60 tx=0x7ff4ac008040 comp rx=0 tx=0).stop 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c00d0d60 msgr2=0x7ff4c00ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c00d0d60 0x7ff4c00ccfd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4009130 tx=0x7ff4b4004750 comp rx=0 tx=0).stop 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 shutdown_connections 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff4a006ac90 0x7ff4a006d150 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4c0160770 0x7ff4c00b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff4c00d0d60 0x7ff4c00ccfd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 --2- 172.21.15.99:0/2628080802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4c00d3380 0x7ff4c01f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 >> 172.21.15.99:0/2628080802 conn(0x7ff4c008afd0 msgr2=0x7ff4c00b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 shutdown_connections 2024-03-29T09:11:20.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.490+0000 7ff4c7400700 1 -- 172.21.15.99:0/2628080802 wait complete. 2024-03-29T09:11:20.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-03-29T09:11:20.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.700+0000 7fb2c4600700 1 Processor -- start 2024-03-29T09:11:20.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.700+0000 7fb2c4600700 1 -- start start 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c4600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2bc0b3cc0 0x7fb2bc0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c4600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0d3380 0x7fb2bc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c4600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0d3810 con 0x7fb2bc160770 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c4600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b4140 con 0x7fb2bc0b3cc0 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c4600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0d1520 con 0x7fb2bc0d3380 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c1200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb2bc0b3cc0 0x7fb2bc0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c1c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb2bc160770 0x7fb2bc0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47392/0 (socket says 172.21.15.99:47392) 2024-03-29T09:11:20.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c1c00700 1 -- 172.21.15.99:0/1712852396 learned_addr learned my addr 172.21.15.99:0/1712852396 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c2600700 1 --2- 172.21.15.99:0/1712852396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0d3380 0x7fb2bc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2bbe00700 1 -- 172.21.15.99:0/1712852396 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200652475 0 0) 0x7fb2bc0d3810 con 0x7fb2bc160770 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c1200700 1 -- 172.21.15.99:0/1712852396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0d3380 msgr2=0x7fb2bc0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c1200700 1 --2- 172.21.15.99:0/1712852396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0d3380 0x7fb2bc0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c1200700 1 -- 172.21.15.99:0/1712852396 >> v1:172.21.15.99:6789/0 conn(0x7fb2bc160770 legacy=0x7fb2bc0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c1200700 1 -- 172.21.15.99:0/1712852396 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0d1800 con 0x7fb2bc0b3cc0 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.702+0000 7fb2c2600700 1 --2- 172.21.15.99:0/1712852396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0d3380 0x7fb2bc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2c1200700 1 --2- 172.21.15.99:0/1712852396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0d2e40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009840 tx=0x7fb2a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135b8db071f89c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2bbe00700 1 -- 172.21.15.99:0/1712852396 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8011830 con 0x7fb2bc0b3cc0 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2bbe00700 1 -- 172.21.15.99:0/1712852396 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a8011990 con 0x7fb2bc0b3cc0 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2bbe00700 1 -- 172.21.15.99:0/1712852396 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8011d20 con 0x7fb2bc0b3cc0 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2c4600700 1 -- 172.21.15.99:0/1712852396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3cc0 msgr2=0x7fb2bc0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2c4600700 1 --2- 172.21.15.99:0/1712852396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0d2e40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009840 tx=0x7fb2a8009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:20.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2c4600700 1 -- 172.21.15.99:0/1712852396 shutdown_connections 2024-03-29T09:11:20.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2c4600700 1 --2- 172.21.15.99:0/1712852396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc0d3380 0x7fb2bc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2c4600700 1 --2- 172.21.15.99:0/1712852396 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0d2e40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.703+0000 7fb2c4600700 1 -- 172.21.15.99:0/1712852396 >> 172.21.15.99:0/1712852396 conn(0x7fb2bc08afd0 msgr2=0x7fb2bc09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:20.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.704+0000 7fb2c4600700 1 -- 172.21.15.99:0/1712852396 shutdown_connections 2024-03-29T09:11:20.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.704+0000 7fb2c4600700 1 -- 172.21.15.99:0/1712852396 wait complete. 2024-03-29T09:11:20.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.705+0000 7fb2c4600700 1 Processor -- start 2024-03-29T09:11:20.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.705+0000 7fb2c4600700 1 -- start start 2024-03-29T09:11:20.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.706+0000 7fb2c4600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.706+0000 7fb2c4600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0d3380 0x7fb2bc0b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.706+0000 7fb2c4600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc160770 0x7fb2bc15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.706+0000 7fb2c4600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc15cc00 con 0x7fb2bc0d3380 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.706+0000 7fb2c4600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b6450 con 0x7fb2bc0b3cc0 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.706+0000 7fb2c4600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b6730 con 0x7fb2bc160770 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0d3380 0x7fb2bc0b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0d3380 0x7fb2bc0b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43820/0 (socket says 172.21.15.99:43820) 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 -- 172.21.15.99:0/2235324321 learned_addr learned my addr 172.21.15.99:0/2235324321 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c2600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc160770 0x7fb2bc15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1c00700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 -- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc160770 msgr2=0x7fb2bc15c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc160770 0x7fb2bc15c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 -- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc0b3cc0 msgr2=0x7fb2bc0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 -- 172.21.15.99:0/2235324321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80094a0 con 0x7fb2bc0d3380 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1c00700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2c1200700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0d3380 0x7fb2bc0b5d50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80090b0 tx=0x7fb2a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2b9600700 1 -- 172.21.15.99:0/2235324321 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a80036a0 con 0x7fb2bc0d3380 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.707+0000 7fb2b9600700 1 -- 172.21.15.99:0/2235324321 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a801f070 con 0x7fb2bc0d3380 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.708+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc0b6a10 con 0x7fb2bc0d3380 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.708+0000 7fb2b9600700 1 -- 172.21.15.99:0/2235324321 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8003a00 con 0x7fb2bc0d3380 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.708+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc0b6fd0 con 0x7fb2bc0d3380 2024-03-29T09:11:20.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.709+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc09e1c0 con 0x7fb2bc0d3380 2024-03-29T09:11:20.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.709+0000 7fb2b9600700 1 -- 172.21.15.99:0/2235324321 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb2a8003b60 con 0x7fb2bc0d3380 2024-03-29T09:11:20.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.710+0000 7fb2b9600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb29c06ac90 0x7fb29c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:20.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.710+0000 7fb2b9600700 1 -- 172.21.15.99:0/2235324321 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb2a8021070 con 0x7fb2bc0d3380 2024-03-29T09:11:20.710 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.710+0000 7fb2c1c00700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb29c06ac90 0x7fb29c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:20.711 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.711+0000 7fb2c1c00700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb29c06ac90 0x7fb29c06d150 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0000f60 tx=0x7fb2b0007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:20.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.714+0000 7fb2b9600700 1 -- 172.21.15.99:0/2235324321 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb2a8055e90 con 0x7fb2bc0d3380 2024-03-29T09:11:20.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.915+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7fb2bc0678b0 con 0x7fb2bc0d3380 2024-03-29T09:11:20.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.916+0000 7fb2b9600700 1 -- 172.21.15.99:0/2235324321 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7fb2a8015020 con 0x7fb2bc0d3380 2024-03-29T09:11:20.916 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb29c06ac90 msgr2=0x7fb29c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb29c06ac90 0x7fb29c06d150 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0000f60 tx=0x7fb2b0007d30 comp rx=0 tx=0).stop 2024-03-29T09:11:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0d3380 msgr2=0x7fb2bc0b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0d3380 0x7fb2bc0b5d50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80090b0 tx=0x7fb2a8004750 comp rx=0 tx=0).stop 2024-03-29T09:11:20.918 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 shutdown_connections 2024-03-29T09:11:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb29c06ac90 0x7fb29c06d150 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb2bc160770 0x7fb2bc15c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2bc0b3cc0 0x7fb2bc0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 --2- 172.21.15.99:0/2235324321 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb2bc0d3380 0x7fb2bc0b5d50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 >> 172.21.15.99:0/2235324321 conn(0x7fb2bc08afd0 msgr2=0x7fb2bc0d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 shutdown_connections 2024-03-29T09:11:20.919 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:20.918+0000 7fb2c4600700 1 -- 172.21.15.99:0/2235324321 wait complete. 2024-03-29T09:11:20.936 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:11:21.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.127+0000 7f2bf3e00700 1 Processor -- start 2024-03-29T09:11:21.127 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.127+0000 7f2bf3e00700 1 -- start start 2024-03-29T09:11:21.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 0x7f2bec0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bf3e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec0cddc0 con 0x7f2bec1608e0 2024-03-29T09:11:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bf3e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec0ce0a0 con 0x7f2bec0d1b20 2024-03-29T09:11:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bf3e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0ce380 con 0x7f2bec160cc0 2024-03-29T09:11:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bf1400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2bec1608e0 0x7f2bec0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47404/0 (socket says 172.21.15.99:47404) 2024-03-29T09:11:21.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bf1400700 1 -- 172.21.15.99:0/2742442965 learned_addr learned my addr 172.21.15.99:0/2742442965 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bf1e00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 0x7f2bec0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332937717 0 0) 0x7f2bec0cddc0 con 0x7f2bec1608e0 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.129+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd40035b0 con 0x7f2bec1608e0 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3587039874 0 0) 0x7f2bd40035b0 con 0x7f2bec1608e0 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 msgr2=0x7f2bec0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 0x7f2bec0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 >> v1:172.21.15.133:6789/0 conn(0x7f2bec0d1b20 legacy=0x7f2bec0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0ce660 con 0x7f2bec1608e0 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f2be0001670 con 0x7f2bec1608e0 2024-03-29T09:11:21.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 >> v1:172.21.15.99:6789/0 conn(0x7f2bec1608e0 legacy=0x7f2bec0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd4003a30 0x7f2bd4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd4004850 0x7f2bd4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 0x7f2bd4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0ce380 con 0x7f2bd4004850 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd400bac0 con 0x7f2bd4003a30 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd400bc30 con 0x7f2bec160cc0 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bf0a00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd4004850 0x7f2bd4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bf1400700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd4003a30 0x7f2bd4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bf1e00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 0x7f2bd4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bf0a00700 1 -- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 msgr2=0x7f2bd4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.130+0000 7f2bf0a00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bec160cc0 0x7f2bd4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bf0a00700 1 -- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd4003a30 msgr2=0x7f2bd4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bf0a00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd4003a30 0x7f2bd4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bf0a00700 1 -- 172.21.15.99:0/2742442965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40035b0 con 0x7f2bd4004850 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bf0a00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd4004850 0x7f2bd4004cd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001880 tx=0x7f2bd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f154f7de3d8e325 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8001370 con 0x7f2bd4004850 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd8012640 con 0x7f2bd4004850 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bebe00700 1 -- 172.21.15.99:0/2742442965 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80129f0 con 0x7f2bd4004850 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bf3e00700 1 -- 172.21.15.99:0/2742442965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd4004850 msgr2=0x7f2bd4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bf3e00700 1 --2- 172.21.15.99:0/2742442965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bd4004850 0x7f2bd4004cd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001880 tx=0x7f2bd4005880 comp rx=0 tx=0).stop 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.131+0000 7f2bf1e00700 1 -- 172.21.15.99:0/2742442965 reap_dead start 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.132+0000 7f2bf3e00700 1 -- 172.21.15.99:0/2742442965 shutdown_connections 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.132+0000 7f2bf3e00700 1 -- 172.21.15.99:0/2742442965 >> 172.21.15.99:0/2742442965 conn(0x7f2bec09c490 msgr2=0x7f2bec0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.132+0000 7f2bf3e00700 1 -- 172.21.15.99:0/2742442965 shutdown_connections 2024-03-29T09:11:21.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.132+0000 7f2bf3e00700 1 -- 172.21.15.99:0/2742442965 wait complete. 2024-03-29T09:11:21.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.133+0000 7f2bf3e00700 1 Processor -- start 2024-03-29T09:11:21.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.133+0000 7f2bf3e00700 1 -- start start 2024-03-29T09:11:21.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.134+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd4004850 0x7f2bec153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.134+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bec1f3e80 0x7f2bec156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.134+0000 7f2bf3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bec1f4300 0x7f2bec154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.134+0000 7f2bf3e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0b3470 con 0x7f2bec1f4300 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.134+0000 7f2bf3e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0b3750 con 0x7f2bec1f3e80 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.134+0000 7f2bf3e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec0b3a30 con 0x7f2bd4004850 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bec1f4300 0x7f2bec154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bec1f4300 0x7f2bec154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43844/0 (socket says 172.21.15.99:43844) 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 -- 172.21.15.99:0/1270994041 learned_addr learned my addr 172.21.15.99:0/1270994041 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf0a00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bec1f3e80 0x7f2bec156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1400700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd4004850 0x7f2bec153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 -- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd4004850 msgr2=0x7f2bec153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd4004850 0x7f2bec153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 -- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bec1f3e80 msgr2=0x7f2bec156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.137 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bec1f3e80 0x7f2bec156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 -- 172.21.15.99:0/1270994041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0b3d10 con 0x7f2bec1f4300 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2bf1e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bec1f4300 0x7f2bec154720 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bec09c930 tx=0x7f2be4011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2be9600700 1 -- 172.21.15.99:0/1270994041 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400e040 con 0x7f2bec1f4300 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.135+0000 7f2be9600700 1 -- 172.21.15.99:0/1270994041 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be400f040 con 0x7f2bec1f4300 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.136+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec0b3ff0 con 0x7f2bec1f4300 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.136+0000 7f2be9600700 1 -- 172.21.15.99:0/1270994041 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400f950 con 0x7f2bec1f4300 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.136+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec156b60 con 0x7f2bec1f4300 2024-03-29T09:11:21.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.136+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec0b49f0 con 0x7f2bec1f4300 2024-03-29T09:11:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.137+0000 7f2be9600700 1 -- 172.21.15.99:0/1270994041 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2be400fab0 con 0x7f2bec1f4300 2024-03-29T09:11:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.138+0000 7f2be9600700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2bcc06ac90 0x7f2bcc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.138+0000 7f2be9600700 1 -- 172.21.15.99:0/1270994041 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f2be4089bc0 con 0x7f2bec1f4300 2024-03-29T09:11:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.139+0000 7f2bf1400700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2bcc06ac90 0x7f2bcc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.139+0000 7f2bf1400700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2bcc06ac90 0x7f2bcc06d150 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2be00017a0 tx=0x7f2be0001590 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:21.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.143+0000 7f2be9600700 1 -- 172.21.15.99:0/1270994041 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2be4054b60 con 0x7f2bec1f4300 2024-03-29T09:11:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.358+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2bec0677a0 con 0x7f2bec1f4300 2024-03-29T09:11:21.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.359+0000 7f2be9600700 1 -- 172.21.15.99:0/1270994041 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2be4058180 con 0x7f2bec1f4300 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2bcc06ac90 msgr2=0x7f2bcc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2bcc06ac90 0x7f2bcc06d150 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2be00017a0 tx=0x7f2be0001590 comp rx=0 tx=0).stop 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bec1f4300 msgr2=0x7f2bec154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bec1f4300 0x7f2bec154720 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2bec09c930 tx=0x7f2be4011f60 comp rx=0 tx=0).stop 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 shutdown_connections 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2bcc06ac90 0x7f2bcc06d150 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2bec1f4300 0x7f2bec154720 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd4004850 0x7f2bec153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 --2- 172.21.15.99:0/1270994041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bec1f3e80 0x7f2bec156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 >> 172.21.15.99:0/1270994041 conn(0x7f2bec09c490 msgr2=0x7f2bec06ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.361+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 shutdown_connections 2024-03-29T09:11:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.362+0000 7f2bf3e00700 1 -- 172.21.15.99:0/1270994041 wait complete. 2024-03-29T09:11:21.363 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:11:21.380 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:21.380 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:21.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:21.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.651+0000 7fbdbdc00700 1 Processor -- start 2024-03-29T09:11:21.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.651+0000 7fbdbdc00700 1 -- start start 2024-03-29T09:11:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.652+0000 7fbdbdc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbdb815ae70 0x7fbdb815b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.652+0000 7fbdbdc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb815c220 con 0x7fbdb80b3e00 2024-03-29T09:11:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.652+0000 7fbdbdc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdb815b8f0 con 0x7fbdb80b0b50 2024-03-29T09:11:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.652+0000 7fbdbdc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdb815bbd0 con 0x7fbdb815ae70 2024-03-29T09:11:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.652+0000 7fbdb7400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbdb80b3e00 0x7fbdb80b1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47410/0 (socket says 172.21.15.99:47410) 2024-03-29T09:11:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.652+0000 7fbdb6a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbdb815ae70 0x7fbdb815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.652+0000 7fbdb7400700 1 -- 172.21.15.99:0/4202726212 learned_addr learned my addr 172.21.15.99:0/4202726212 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb6a00700 1 -- 172.21.15.99:0/4202726212 >> v1:172.21.15.133:6789/0 conn(0x7fbdb80b0b50 legacy=0x7fbdb80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb6a00700 1 -- 172.21.15.99:0/4202726212 >> v1:172.21.15.99:6789/0 conn(0x7fbdb80b3e00 legacy=0x7fbdb80b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb6a00700 1 -- 172.21.15.99:0/4202726212 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb815beb0 con 0x7fbdb815ae70 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb5600700 1 -- 172.21.15.99:0/4202726212 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542982103 0 0) 0x7fbdb815c220 con 0x7fbdb80b3e00 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb6a00700 1 --2- 172.21.15.99:0/4202726212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb815ae70 0x7fbdb815b250 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009840 tx=0x7fbda8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1771138123ab41a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb5600700 1 -- 172.21.15.99:0/4202726212 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8011830 con 0x7fbdb815ae70 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb5600700 1 -- 172.21.15.99:0/4202726212 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda8011990 con 0x7fbdb815ae70 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.653+0000 7fbdb5600700 1 -- 172.21.15.99:0/4202726212 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8011d20 con 0x7fbdb815ae70 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.654+0000 7fbdbdc00700 1 -- 172.21.15.99:0/4202726212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb815ae70 msgr2=0x7fbdb815b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.654 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.654+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/4202726212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb815ae70 0x7fbdb815b250 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009840 tx=0x7fbda8009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.654+0000 7fbdbdc00700 1 -- 172.21.15.99:0/4202726212 shutdown_connections 2024-03-29T09:11:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.654+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/4202726212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb815ae70 0x7fbdb815b250 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.654+0000 7fbdbdc00700 1 -- 172.21.15.99:0/4202726212 >> 172.21.15.99:0/4202726212 conn(0x7fbdb8066540 msgr2=0x7fbdb8066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.654+0000 7fbdbdc00700 1 -- 172.21.15.99:0/4202726212 shutdown_connections 2024-03-29T09:11:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.654+0000 7fbdbdc00700 1 -- 172.21.15.99:0/4202726212 wait complete. 2024-03-29T09:11:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.656+0000 7fbdbdc00700 1 Processor -- start 2024-03-29T09:11:21.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.656+0000 7fbdbdc00700 1 -- start start 2024-03-29T09:11:21.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdbdc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb80b0b50 0x7fbdb8150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdbdc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdb80b3e00 0x7fbdb8152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdbdc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdb815ae70 0x7fbdb814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdbdc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb8153e90 con 0x7fbdb80b0b50 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdbdc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb8153020 con 0x7fbdb80b3e00 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdbdc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb8153210 con 0x7fbdb815ae70 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb80b0b50 0x7fbdb8150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb80b0b50 0x7fbdb8150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43854/0 (socket says 172.21.15.99:43854) 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 -- 172.21.15.99:0/2537663584 learned_addr learned my addr 172.21.15.99:0/2537663584 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 -- 172.21.15.99:0/2537663584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdb815ae70 msgr2=0x7fbdb814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdb815ae70 0x7fbdb814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 -- 172.21.15.99:0/2537663584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdb80b3e00 msgr2=0x7fbdb8152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdb80b3e00 0x7fbdb8152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 -- 172.21.15.99:0/2537663584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80094a0 con 0x7fbdb80b0b50 2024-03-29T09:11:21.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.657+0000 7fbdb6a00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb80b0b50 0x7fbdb8150cb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbda80090b0 tx=0x7fbda8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:21.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.658+0000 7fbdaea00700 1 -- 172.21.15.99:0/2537663584 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda80036a0 con 0x7fbdb80b0b50 2024-03-29T09:11:21.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.658+0000 7fbdaea00700 1 -- 172.21.15.99:0/2537663584 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda801e070 con 0x7fbdb80b0b50 2024-03-29T09:11:21.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.658+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdb81534f0 con 0x7fbdb80b0b50 2024-03-29T09:11:21.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.658+0000 7fbdaea00700 1 -- 172.21.15.99:0/2537663584 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda8003a20 con 0x7fbdb80b0b50 2024-03-29T09:11:21.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.658+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb8153ab0 con 0x7fbdb80b0b50 2024-03-29T09:11:21.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.659+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdb80677a0 con 0x7fbdb80b0b50 2024-03-29T09:11:21.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.660+0000 7fbdaea00700 1 -- 172.21.15.99:0/2537663584 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbda8003b80 con 0x7fbdb80b0b50 2024-03-29T09:11:21.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.660+0000 7fbdaea00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbd9806ac90 0x7fbd9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:21.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.661+0000 7fbdaea00700 1 -- 172.21.15.99:0/2537663584 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fbda808a820 con 0x7fbdb80b0b50 2024-03-29T09:11:21.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.661+0000 7fbdb6000700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbd9806ac90 0x7fbd9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:21.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.661+0000 7fbdb6000700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbd9806ac90 0x7fbd9806d150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000f60 tx=0x7fbda4007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:21.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.665+0000 7fbdaea00700 1 -- 172.21.15.99:0/2537663584 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbda8055720 con 0x7fbdb80b0b50 2024-03-29T09:11:21.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.866+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdb80b1a90 con 0x7fbdb80b0b50 2024-03-29T09:11:21.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.867+0000 7fbdaea00700 1 -- 172.21.15.99:0/2537663584 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7fbda8058d40 con 0x7fbdb80b0b50 2024-03-29T09:11:21.867 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:21.868 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:21.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbd9806ac90 msgr2=0x7fbd9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbd9806ac90 0x7fbd9806d150 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000f60 tx=0x7fbda4007a40 comp rx=0 tx=0).stop 2024-03-29T09:11:21.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb80b0b50 msgr2=0x7fbdb8150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:21.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb80b0b50 0x7fbdb8150cb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbda80090b0 tx=0x7fbda8004750 comp rx=0 tx=0).stop 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 shutdown_connections 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbd9806ac90 0x7fbd9806d150 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdb815ae70 0x7fbdb814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdb80b0b50 0x7fbdb8150cb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 --2- 172.21.15.99:0/2537663584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdb80b3e00 0x7fbdb8152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 >> 172.21.15.99:0/2537663584 conn(0x7fbdb8066540 msgr2=0x7fbdb80b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 shutdown_connections 2024-03-29T09:11:21.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:21.870+0000 7fbdbdc00700 1 -- 172.21.15.99:0/2537663584 wait complete. 2024-03-29T09:11:21.889 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:10:42.037884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:10:42.900862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-03-29T09:10:53.257132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:11:21.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:11:22.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.082+0000 7f798d000700 1 Processor -- start 2024-03-29T09:11:22.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.082+0000 7f798d000700 1 -- start start 2024-03-29T09:11:22.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f798d000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f79880cf090 0x7f7988167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.083 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f798d000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880d1520 con 0x7f79880b3cc0 2024-03-29T09:11:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f798d000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880d1800 con 0x7f7988160770 2024-03-29T09:11:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f798d000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880b4140 con 0x7f79880cf090 2024-03-29T09:11:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f7985400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f79880b3cc0 0x7f79880d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47412/0 (socket says 172.21.15.99:47412) 2024-03-29T09:11:22.084 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f7985400700 1 -- 172.21.15.99:0/2834204467 learned_addr learned my addr 172.21.15.99:0/2834204467 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f7986800700 1 --2- 172.21.15.99:0/2834204467 >> v2:172.21.15.133:3300/0 conn(0x7f79880cf090 0x7f7988167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f7984a00700 1 -- 172.21.15.99:0/2834204467 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780037689 0 0) 0x7f79880d1520 con 0x7f79880b3cc0 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f7984a00700 1 -- 172.21.15.99:0/2834204467 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f796c0035b0 con 0x7f79880b3cc0 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f7984a00700 1 -- 172.21.15.99:0/2834204467 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901882604 0 0) 0x7f79880d1800 con 0x7f7988160770 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.084+0000 7f7984a00700 1 -- 172.21.15.99:0/2834204467 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79880d1520 con 0x7f7988160770 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.085+0000 7f7986800700 1 -- 172.21.15.99:0/2834204467 >> v1:172.21.15.133:6789/0 conn(0x7f7988160770 legacy=0x7f79880d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.085+0000 7f7986800700 1 -- 172.21.15.99:0/2834204467 >> v1:172.21.15.99:6789/0 conn(0x7f79880b3cc0 legacy=0x7f79880d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.085 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.085+0000 7f7986800700 1 -- 172.21.15.99:0/2834204467 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880d1ae0 con 0x7f79880cf090 2024-03-29T09:11:22.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.085+0000 7f7986800700 1 --2- 172.21.15.99:0/2834204467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79880cf090 0x7f7988167f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7980009d90 tx=0x7f798000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d390543d82d73bc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.085+0000 7f7984a00700 1 -- 172.21.15.99:0/2834204467 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798000e070 con 0x7f79880cf090 2024-03-29T09:11:22.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.085+0000 7f7984a00700 1 -- 172.21.15.99:0/2834204467 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798000bb20 con 0x7f79880cf090 2024-03-29T09:11:22.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.085+0000 7f7984a00700 1 -- 172.21.15.99:0/2834204467 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980012430 con 0x7f79880cf090 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.086+0000 7f798d000700 1 -- 172.21.15.99:0/2834204467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79880cf090 msgr2=0x7f7988167f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.086+0000 7f798d000700 1 --2- 172.21.15.99:0/2834204467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79880cf090 0x7f7988167f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7980009d90 tx=0x7f798000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.086+0000 7f798d000700 1 -- 172.21.15.99:0/2834204467 shutdown_connections 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.086+0000 7f798d000700 1 --2- 172.21.15.99:0/2834204467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79880cf090 0x7f7988167f40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.086+0000 7f798d000700 1 -- 172.21.15.99:0/2834204467 >> 172.21.15.99:0/2834204467 conn(0x7f798808afd0 msgr2=0x7f798809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.086+0000 7f798d000700 1 -- 172.21.15.99:0/2834204467 shutdown_connections 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.086+0000 7f798d000700 1 -- 172.21.15.99:0/2834204467 wait complete. 2024-03-29T09:11:22.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.088+0000 7f798d000700 1 Processor -- start 2024-03-29T09:11:22.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.088+0000 7f798d000700 1 -- start start 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f798d000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79880b3cc0 0x7f798809cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f798d000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79880cf090 0x7f7988154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f798d000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7988160770 0x7f79880b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f798d000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798809d480 con 0x7f79880b3cc0 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f798d000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880a0e70 con 0x7f7988160770 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f798d000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880a08b0 con 0x7f79880cf090 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79880b3cc0 0x7f798809cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79880b3cc0 0x7f798809cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43866/0 (socket says 172.21.15.99:43866) 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 -- 172.21.15.99:0/402269762 learned_addr learned my addr 172.21.15.99:0/402269762 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985400700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79880cf090 0x7f7988154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.089 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7986800700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7988160770 0x7f79880b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.090 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 -- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79880cf090 msgr2=0x7f7988154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79880cf090 0x7f7988154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 -- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7988160770 msgr2=0x7f79880b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7988160770 0x7f79880b7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.089+0000 7f7985e00700 1 -- 172.21.15.99:0/402269762 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980009a70 con 0x7f79880b3cc0 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f7986800700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7988160770 0x7f79880b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f7985400700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79880cf090 0x7f7988154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f7985e00700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79880b3cc0 0x7f798809cf40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f797800c500 tx=0x7f797800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f797e000700 1 -- 172.21.15.99:0/402269762 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780052d0 con 0x7f79880b3cc0 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f797e000700 1 -- 172.21.15.99:0/402269762 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978005450 con 0x7f79880b3cc0 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f797e000700 1 -- 172.21.15.99:0/402269762 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780038c0 con 0x7f79880b3cc0 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79880a0b90 con 0x7f79880b3cc0 2024-03-29T09:11:22.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.090+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79880cb9e0 con 0x7f79880b3cc0 2024-03-29T09:11:22.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.091+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79880678b0 con 0x7f79880b3cc0 2024-03-29T09:11:22.092 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.092+0000 7f797e000700 1 -- 172.21.15.99:0/402269762 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7978003a20 con 0x7f79880b3cc0 2024-03-29T09:11:22.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.093+0000 7f797e000700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f796806ac90 0x7f796806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.093+0000 7f797e000700 1 -- 172.21.15.99:0/402269762 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f7978088b50 con 0x7f79880b3cc0 2024-03-29T09:11:22.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.093+0000 7f7985400700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f796806ac90 0x7f796806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.093 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.094+0000 7f7985400700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f796806ac90 0x7f796806d150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f79880cb5f0 tx=0x7f7974009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.097+0000 7f797e000700 1 -- 172.21.15.99:0/402269762 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f79780539c0 con 0x7f79880b3cc0 2024-03-29T09:11:22.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.300+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f798806b540 con 0x7f79880b3cc0 2024-03-29T09:11:22.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.301+0000 7f797e000700 1 -- 172.21.15.99:0/402269762 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79780170a0 con 0x7f79880b3cc0 2024-03-29T09:11:22.301 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:11:22.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f796806ac90 msgr2=0x7f796806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.303 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f796806ac90 0x7f796806d150 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f79880cb5f0 tx=0x7f7974009040 comp rx=0 tx=0).stop 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79880b3cc0 msgr2=0x7f798809cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79880b3cc0 0x7f798809cf40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f797800c500 tx=0x7f797800c8c0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 shutdown_connections 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f796806ac90 0x7f796806d150 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7988160770 0x7f79880b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79880b3cc0 0x7f798809cf40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 --2- 172.21.15.99:0/402269762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79880cf090 0x7f7988154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 >> 172.21.15.99:0/402269762 conn(0x7f798808afd0 msgr2=0x7f798806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 shutdown_connections 2024-03-29T09:11:22.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.303+0000 7f798d000700 1 -- 172.21.15.99:0/402269762 wait complete. 2024-03-29T09:11:22.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:11:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.514+0000 7f5b88800700 1 Processor -- start 2024-03-29T09:11:22.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.514+0000 7f5b88800700 1 -- start start 2024-03-29T09:11:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.515+0000 7f5b88800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5b800b3cc0 0x7f5b800d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.515+0000 7f5b88800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 0x7f5b800ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.515+0000 7f5b88800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800d3810 con 0x7f5b80160770 2024-03-29T09:11:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.515+0000 7f5b88800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b800b4140 con 0x7f5b800b3cc0 2024-03-29T09:11:22.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.515+0000 7f5b88800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800d1520 con 0x7f5b800d3380 2024-03-29T09:11:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.515+0000 7f5b85e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f5b80160770 0x7f5b800d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47420/0 (socket says 172.21.15.99:47420) 2024-03-29T09:11:22.516 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.515+0000 7f5b85e00700 1 -- 172.21.15.99:0/1833750857 learned_addr learned my addr 172.21.15.99:0/1833750857 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:22.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024235100 0 0) 0x7f5b800d3810 con 0x7f5b80160770 2024-03-29T09:11:22.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b86800700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 0x7f5b800ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b680035b0 con 0x7f5b80160770 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b85400700 1 --2- 172.21.15.99:0/1833750857 >> v2:172.21.15.133:3300/0 conn(0x7f5b800b3cc0 0x7f5b800d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3549588479 0 0) 0x7f5b680035b0 con 0x7f5b80160770 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 msgr2=0x7f5b800ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 0x7f5b800ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 >> v2:172.21.15.133:3300/0 conn(0x7f5b800b3cc0 msgr2=0x7f5b800d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 --2- 172.21.15.99:0/1833750857 >> v2:172.21.15.133:3300/0 conn(0x7f5b800b3cc0 0x7f5b800d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800d1800 con 0x7f5b80160770 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f5b74001670 con 0x7f5b80160770 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 >> v1:172.21.15.99:6789/0 conn(0x7f5b80160770 legacy=0x7f5b800d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 0x7f5b68003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b68003f80 0x7f5b68004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b68004d90 0x7f5b68005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800d1520 con 0x7f5b68003f80 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6800ba10 con 0x7f5b68004d90 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.516+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6800bb50 con 0x7f5b800d3380 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85400700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b68003f80 0x7f5b68004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85e00700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 0x7f5b68003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b86800700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b68004d90 0x7f5b68005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85400700 1 -- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 msgr2=0x7f5b68003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85400700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b800d3380 0x7f5b68003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85400700 1 -- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b68004d90 msgr2=0x7f5b68005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85400700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b68004d90 0x7f5b68005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85400700 1 -- 172.21.15.99:0/1833750857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680035b0 con 0x7f5b68003f80 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b85400700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b68003f80 0x7f5b68004850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009b60 tx=0x7f5b68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11ff8810f6a9b2a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000e660 con 0x7f5b68003f80 2024-03-29T09:11:22.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7000e7c0 con 0x7f5b68003f80 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.517+0000 7f5b84a00700 1 -- 172.21.15.99:0/1833750857 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000eb30 con 0x7f5b68003f80 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.518+0000 7f5b88800700 1 -- 172.21.15.99:0/1833750857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b68003f80 msgr2=0x7f5b68004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.518+0000 7f5b88800700 1 --2- 172.21.15.99:0/1833750857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b68003f80 0x7f5b68004850 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009b60 tx=0x7f5b68005880 comp rx=0 tx=0).stop 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.518+0000 7f5b86800700 1 -- 172.21.15.99:0/1833750857 reap_dead start 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.518+0000 7f5b88800700 1 -- 172.21.15.99:0/1833750857 shutdown_connections 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.518+0000 7f5b88800700 1 -- 172.21.15.99:0/1833750857 >> 172.21.15.99:0/1833750857 conn(0x7f5b8008afd0 msgr2=0x7f5b8009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.518+0000 7f5b88800700 1 -- 172.21.15.99:0/1833750857 shutdown_connections 2024-03-29T09:11:22.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.518+0000 7f5b88800700 1 -- 172.21.15.99:0/1833750857 wait complete. 2024-03-29T09:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.520+0000 7f5b88800700 1 Processor -- start 2024-03-29T09:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.520+0000 7f5b88800700 1 -- start start 2024-03-29T09:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b88800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b68003f80 0x7f5b801f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b88800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b800cc270 0x7f5b8015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b88800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b800b7c30 0x7f5b800b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b88800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800b85f0 con 0x7f5b800b7c30 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b88800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cc6f0 con 0x7f5b800cc270 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b88800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cc9c0 con 0x7f5b68003f80 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b86800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b800b7c30 0x7f5b800b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b86800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b800b7c30 0x7f5b800b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43874/0 (socket says 172.21.15.99:43874) 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b86800700 1 -- 172.21.15.99:0/3425819178 learned_addr learned my addr 172.21.15.99:0/3425819178 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b85400700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b800cc270 0x7f5b8015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.521+0000 7f5b85e00700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b68003f80 0x7f5b801f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b86800700 1 -- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b68003f80 msgr2=0x7f5b801f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b86800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b68003f80 0x7f5b801f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b86800700 1 -- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b800cc270 msgr2=0x7f5b8015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b86800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b800cc270 0x7f5b8015c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b86800700 1 -- 172.21.15.99:0/3425819178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800ccca0 con 0x7f5b800b7c30 2024-03-29T09:11:22.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b86800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b800b7c30 0x7f5b800b80b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b8006b4d0 tx=0x7f5b7c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b7a000700 1 -- 172.21.15.99:0/3425819178 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00e040 con 0x7f5b800b7c30 2024-03-29T09:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b7a000700 1 -- 172.21.15.99:0/3425819178 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c00f040 con 0x7f5b800b7c30 2024-03-29T09:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b800ccf80 con 0x7f5b800b7c30 2024-03-29T09:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b7a000700 1 -- 172.21.15.99:0/3425819178 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c00fc40 con 0x7f5b800b7c30 2024-03-29T09:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.522+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b800cd540 con 0x7f5b800b7c30 2024-03-29T09:11:22.525 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.523+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b800678b0 con 0x7f5b800b7c30 2024-03-29T09:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.524+0000 7f5b7a000700 1 -- 172.21.15.99:0/3425819178 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5b7c015430 con 0x7f5b800b7c30 2024-03-29T09:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.525+0000 7f5b7a000700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5b6406ac90 0x7f5b6406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.525+0000 7f5b7a000700 1 -- 172.21.15.99:0/3425819178 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f5b7c089cc0 con 0x7f5b800b7c30 2024-03-29T09:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.525+0000 7f5b85e00700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5b6406ac90 0x7f5b6406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.526+0000 7f5b85e00700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5b6406ac90 0x7f5b6406d150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b74000c00 tx=0x7f5b74007980 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.529+0000 7f5b7a000700 1 -- 172.21.15.99:0/3425819178 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5b7c090080 con 0x7f5b800b7c30 2024-03-29T09:11:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.729+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b80073130 con 0x7f5b800b7c30 2024-03-29T09:11:22.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.729+0000 7f5b7a000700 1 -- 172.21.15.99:0/3425819178 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b7c054bc0 con 0x7f5b800b7c30 2024-03-29T09:11:22.729 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.731+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5b6406ac90 msgr2=0x7f5b6406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5b6406ac90 0x7f5b6406d150 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b74000c00 tx=0x7f5b74007980 comp rx=0 tx=0).stop 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b800b7c30 msgr2=0x7f5b800b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b800b7c30 0x7f5b800b80b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b8006b4d0 tx=0x7f5b7c011ed0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 shutdown_connections 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5b6406ac90 0x7f5b6406d150 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5b800b7c30 0x7f5b800b80b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b68003f80 0x7f5b801f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 --2- 172.21.15.99:0/3425819178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b800cc270 0x7f5b8015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 >> 172.21.15.99:0/3425819178 conn(0x7f5b8008afd0 msgr2=0x7f5b8006ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 shutdown_connections 2024-03-29T09:11:22.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.732+0000 7f5b88800700 1 -- 172.21.15.99:0/3425819178 wait complete. 2024-03-29T09:11:22.749 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:11:22.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.943+0000 7fa841600700 1 Processor -- start 2024-03-29T09:11:22.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.943+0000 7fa841600700 1 -- start start 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa841600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa83c160770 0x7fa83c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa841600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa83c0d3550 0x7fa83c0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa841600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c0d1520 con 0x7fa83c0d0d60 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa841600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c0d1800 con 0x7fa83c160770 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa841600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c0d1ae0 con 0x7fa83c0d3550 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa83aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa83c160770 0x7fa83c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa83aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa83c160770 0x7fa83c0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43878/0 (socket says 172.21.15.99:43878) 2024-03-29T09:11:22.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa83aa00700 1 -- 172.21.15.99:0/3743667366 learned_addr learned my addr 172.21.15.99:0/3743667366 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:22.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.944+0000 7fa83b400700 1 --2- 172.21.15.99:0/3743667366 >> v2:172.21.15.133:3300/0 conn(0x7fa83c0d3550 0x7fa83c0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa83aa00700 1 -- 172.21.15.99:0/3743667366 >> v2:172.21.15.133:3300/0 conn(0x7fa83c0d3550 msgr2=0x7fa83c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:22.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa83aa00700 1 --2- 172.21.15.99:0/3743667366 >> v2:172.21.15.133:3300/0 conn(0x7fa83c0d3550 0x7fa83c0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa83aa00700 1 -- 172.21.15.99:0/3743667366 >> v1:172.21.15.133:6789/0 conn(0x7fa83c0d0d60 legacy=0x7fa83c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa83aa00700 1 -- 172.21.15.99:0/3743667366 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c0d1dc0 con 0x7fa83c160770 2024-03-29T09:11:22.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa83aa00700 1 --2- 172.21.15.99:0/3743667366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c160770 0x7fa83c0b3cc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0b3230 tx=0x7fa82c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca040a80768e2a5a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.946 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa839600700 1 -- 172.21.15.99:0/3743667366 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c011830 con 0x7fa83c160770 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa839600700 1 -- 172.21.15.99:0/3743667366 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c011990 con 0x7fa83c160770 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.945+0000 7fa839600700 1 -- 172.21.15.99:0/3743667366 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c011d20 con 0x7fa83c160770 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 -- 172.21.15.99:0/3743667366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c160770 msgr2=0x7fa83c0b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 --2- 172.21.15.99:0/3743667366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c160770 0x7fa83c0b3cc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0b3230 tx=0x7fa82c009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 -- 172.21.15.99:0/3743667366 shutdown_connections 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 --2- 172.21.15.99:0/3743667366 >> v2:172.21.15.133:3300/0 conn(0x7fa83c0d3550 0x7fa83c0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 --2- 172.21.15.99:0/3743667366 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c160770 0x7fa83c0b3cc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 -- 172.21.15.99:0/3743667366 >> 172.21.15.99:0/3743667366 conn(0x7fa83c08afd0 msgr2=0x7fa83c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 -- 172.21.15.99:0/3743667366 shutdown_connections 2024-03-29T09:11:22.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.946+0000 7fa841600700 1 -- 172.21.15.99:0/3743667366 wait complete. 2024-03-29T09:11:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.948+0000 7fa841600700 1 Processor -- start 2024-03-29T09:11:22.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.948+0000 7fa841600700 1 -- start start 2024-03-29T09:11:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa841600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83c0d0d60 0x7fa83c0c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa841600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c0d3550 0x7fa83c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa841600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c160770 0x7fa83c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa841600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0b7b00 con 0x7fa83c0d3550 2024-03-29T09:11:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa841600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0cbfc0 con 0x7fa83c160770 2024-03-29T09:11:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa841600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0cc2a0 con 0x7fa83c0d0d60 2024-03-29T09:11:22.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa83a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c0d3550 0x7fa83c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa83a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c0d3550 0x7fa83c0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43884/0 (socket says 172.21.15.99:43884) 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa83a000700 1 -- 172.21.15.99:0/4173380104 learned_addr learned my addr 172.21.15.99:0/4173380104 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa83aa00700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83c0d0d60 0x7fa83c0c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.949+0000 7fa83b400700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c160770 0x7fa83c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa83a000700 1 -- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83c0d0d60 msgr2=0x7fa83c0c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa83a000700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83c0d0d60 0x7fa83c0c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa83a000700 1 -- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c160770 msgr2=0x7fa83c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa83a000700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c160770 0x7fa83c15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa83a000700 1 -- 172.21.15.99:0/4173380104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c009550 con 0x7fa83c0d3550 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa83a000700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c0d3550 0x7fa83c0cac70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c4e0 tx=0x7fa82800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa832a00700 1 -- 172.21.15.99:0/4173380104 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280052d0 con 0x7fa83c0d3550 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa832a00700 1 -- 172.21.15.99:0/4173380104 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828005430 con 0x7fa83c0d3550 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c0cc580 con 0x7fa83c0d3550 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa832a00700 1 -- 172.21.15.99:0/4173380104 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280038c0 con 0x7fa83c0d3550 2024-03-29T09:11:22.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.950+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c0ccb40 con 0x7fa83c0d3550 2024-03-29T09:11:22.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.951+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c0d0b30 con 0x7fa83c0d3550 2024-03-29T09:11:22.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.952+0000 7fa832a00700 1 -- 172.21.15.99:0/4173380104 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa828003a20 con 0x7fa83c0d3550 2024-03-29T09:11:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.953+0000 7fa832a00700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81c06ac90 0x7fa81c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.953+0000 7fa832a00700 1 -- 172.21.15.99:0/4173380104 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa828088940 con 0x7fa83c0d3550 2024-03-29T09:11:22.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.953+0000 7fa83aa00700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81c06ac90 0x7fa81c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:22.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.954+0000 7fa83aa00700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81c06ac90 0x7fa81c06d150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa82c014040 tx=0x7fa82c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:22.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:22.958+0000 7fa832a00700 1 -- 172.21.15.99:0/4173380104 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa828053850 con 0x7fa83c0d3550 2024-03-29T09:11:23.016 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:11:23.020 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:11:23.168 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.168+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa83c0678b0 con 0x7fa83c0d3550 2024-03-29T09:11:23.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.169+0000 7fa832a00700 1 -- 172.21.15.99:0/4173380104 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa828017020 con 0x7fa83c0d3550 2024-03-29T09:11:23.169 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:11:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81c06ac90 msgr2=0x7fa81c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81c06ac90 0x7fa81c06d150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa82c014040 tx=0x7fa82c008030 comp rx=0 tx=0).stop 2024-03-29T09:11:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c0d3550 msgr2=0x7fa83c0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c0d3550 0x7fa83c0cac70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa82800c4e0 tx=0x7fa82800c7f0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 shutdown_connections 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa81c06ac90 0x7fa81c06d150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa83c160770 0x7fa83c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa83c0d0d60 0x7fa83c0c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 --2- 172.21.15.99:0/4173380104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa83c0d3550 0x7fa83c0cac70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 >> 172.21.15.99:0/4173380104 conn(0x7fa83c08afd0 msgr2=0x7fa83c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.171+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 shutdown_connections 2024-03-29T09:11:23.172 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.172+0000 7fa841600700 1 -- 172.21.15.99:0/4173380104 wait complete. 2024-03-29T09:11:23.190 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-03-29T09:11:23.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.382+0000 7f0e7b800700 1 Processor -- start 2024-03-29T09:11:23.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.382+0000 7f0e7b800700 1 -- start start 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e7b800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e74160770 0x7f0e740b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e7b800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e740d0d60 0x7f0e740d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e7b800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e740d3810 con 0x7f0e740d3380 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e7b800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e740d1520 con 0x7f0e74160770 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e7b800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e740d1800 con 0x7f0e740d0d60 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e78e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e74160770 0x7f0e740b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e78e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f0e74160770 0x7f0e740b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43892/0 (socket says 172.21.15.99:43892) 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e78e00700 1 -- 172.21.15.99:0/873715275 learned_addr learned my addr 172.21.15.99:0/873715275 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.383+0000 7f0e73e00700 1 --2- 172.21.15.99:0/873715275 >> v2:172.21.15.133:3300/0 conn(0x7f0e740d0d60 0x7f0e740d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e78e00700 1 -- 172.21.15.99:0/873715275 >> v2:172.21.15.133:3300/0 conn(0x7f0e740d0d60 msgr2=0x7f0e740d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e78e00700 1 --2- 172.21.15.99:0/873715275 >> v2:172.21.15.133:3300/0 conn(0x7f0e740d0d60 0x7f0e740d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e78e00700 1 -- 172.21.15.99:0/873715275 >> v1:172.21.15.133:6789/0 conn(0x7f0e740d3380 legacy=0x7f0e740cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:23.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e78e00700 1 -- 172.21.15.99:0/873715275 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740d1ae0 con 0x7f0e74160770 2024-03-29T09:11:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e73e00700 1 --2- 172.21.15.99:0/873715275 >> v2:172.21.15.133:3300/0 conn(0x7f0e740d0d60 0x7f0e740d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e78e00700 1 --2- 172.21.15.99:0/873715275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74160770 0x7f0e740b3cc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0e740b5cf0 tx=0x7f0e68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99435343c14ae59 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e73400700 1 -- 172.21.15.99:0/873715275 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004230 con 0x7f0e74160770 2024-03-29T09:11:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e73400700 1 -- 172.21.15.99:0/873715275 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68004390 con 0x7f0e74160770 2024-03-29T09:11:23.385 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.384+0000 7f0e73400700 1 -- 172.21.15.99:0/873715275 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e680036a0 con 0x7f0e74160770 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 -- 172.21.15.99:0/873715275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74160770 msgr2=0x7f0e740b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 --2- 172.21.15.99:0/873715275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74160770 0x7f0e740b3cc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0e740b5cf0 tx=0x7f0e68009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 -- 172.21.15.99:0/873715275 shutdown_connections 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 --2- 172.21.15.99:0/873715275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e74160770 0x7f0e740b3cc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 --2- 172.21.15.99:0/873715275 >> v2:172.21.15.133:3300/0 conn(0x7f0e740d0d60 0x7f0e740d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 -- 172.21.15.99:0/873715275 >> 172.21.15.99:0/873715275 conn(0x7f0e7408afd0 msgr2=0x7f0e7409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 -- 172.21.15.99:0/873715275 shutdown_connections 2024-03-29T09:11:23.386 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.385+0000 7f0e7b800700 1 -- 172.21.15.99:0/873715275 wait complete. 2024-03-29T09:11:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.387+0000 7f0e7b800700 1 Processor -- start 2024-03-29T09:11:23.387 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.387+0000 7f0e7b800700 1 -- start start 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e7b800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e740d0d60 0x7f0e7415b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e7b800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e740d3380 0x7f0e7414ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e7b800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e74160770 0x7f0e7415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e7b800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7414d370 con 0x7f0e740d3380 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e7b800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7415c980 con 0x7f0e74160770 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e7b800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7415d350 con 0x7f0e740d0d60 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e740d3380 0x7f0e7414ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e740d3380 0x7f0e7414ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43894/0 (socket says 172.21.15.99:43894) 2024-03-29T09:11:23.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 -- 172.21.15.99:0/632697112 learned_addr learned my addr 172.21.15.99:0/632697112 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e78e00700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e740d0d60 0x7f0e7415b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e79800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e74160770 0x7f0e7415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 -- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e740d0d60 msgr2=0x7f0e7415b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e740d0d60 0x7f0e7415b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 -- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e74160770 msgr2=0x7f0e7415ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e74160770 0x7f0e7415ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e73e00700 1 -- 172.21.15.99:0/632697112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e68009550 con 0x7f0e740d3380 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.388+0000 7f0e79800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e74160770 0x7f0e7415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.389+0000 7f0e73e00700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e740d3380 0x7f0e7414ce30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c750 tx=0x7f0e6000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.389+0000 7f0e70c00700 1 -- 172.21.15.99:0/632697112 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60011800 con 0x7f0e740d3380 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.389+0000 7f0e70c00700 1 -- 172.21.15.99:0/632697112 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e60011960 con 0x7f0e740d3380 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.389+0000 7f0e70c00700 1 -- 172.21.15.99:0/632697112 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60011cd0 con 0x7f0e740d3380 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.389+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7415bd70 con 0x7f0e740d3380 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.389+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7415c330 con 0x7f0e740d3380 2024-03-29T09:11:23.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.390+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7415cb10 con 0x7f0e740d3380 2024-03-29T09:11:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.391+0000 7f0e70c00700 1 -- 172.21.15.99:0/632697112 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0e60021350 con 0x7f0e740d3380 2024-03-29T09:11:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.392+0000 7f0e70c00700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e5406acc0 0x7f0e5406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.392+0000 7f0e70c00700 1 -- 172.21.15.99:0/632697112 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f0e600894a0 con 0x7f0e740d3380 2024-03-29T09:11:23.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.392+0000 7f0e78e00700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e5406acc0 0x7f0e5406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.393+0000 7f0e78e00700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e5406acc0 0x7f0e5406d180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009130 tx=0x7f0e68008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:23.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.397+0000 7f0e70c00700 1 -- 172.21.15.99:0/632697112 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0e60054440 con 0x7f0e740d3380 2024-03-29T09:11:23.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.602+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f0e740678b0 con 0x7f0e740d3380 2024-03-29T09:11:23.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.602+0000 7f0e70c00700 1 -- 172.21.15.99:0/632697112 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7f0e60057a60 con 0x7f0e740d3380 2024-03-29T09:11:23.603 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:23.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e5406acc0 msgr2=0x7f0e5406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e5406acc0 0x7f0e5406d180 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009130 tx=0x7f0e68008030 comp rx=0 tx=0).stop 2024-03-29T09:11:23.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e740d3380 msgr2=0x7f0e7414ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e740d3380 0x7f0e7414ce30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000c750 tx=0x7f0e6000ca60 comp rx=0 tx=0).stop 2024-03-29T09:11:23.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 shutdown_connections 2024-03-29T09:11:23.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e5406acc0 0x7f0e5406d180 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e74160770 0x7f0e7415ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e740d0d60 0x7f0e7415b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 --2- 172.21.15.99:0/632697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e740d3380 0x7f0e7414ce30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 >> 172.21.15.99:0/632697112 conn(0x7f0e7408afd0 msgr2=0x7f0e740b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:23.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 shutdown_connections 2024-03-29T09:11:23.606 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.605+0000 7f0e7b800700 1 -- 172.21.15.99:0/632697112 wait complete. 2024-03-29T09:11:23.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:11:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.813+0000 7fb4f6800700 1 Processor -- start 2024-03-29T09:11:23.813 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.813+0000 7fb4f6800700 1 -- start start 2024-03-29T09:11:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.814+0000 7fb4f6800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f01608e0 0x7fb4f00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.814+0000 7fb4f6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4f0160cc0 0x7fb4f00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.814+0000 7fb4f6800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb4f00d3a70 0x7fb4f00d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.815 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.814+0000 7fb4f6800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f00cddc0 con 0x7fb4f0160cc0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.814+0000 7fb4f6800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f00ce0a0 con 0x7fb4f00d3a70 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.814+0000 7fb4f6800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f00ce380 con 0x7fb4f01608e0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4f0160cc0 0x7fb4f00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb4f0160cc0 0x7fb4f00ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43904/0 (socket says 172.21.15.99:43904) 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 -- 172.21.15.99:0/3554412158 learned_addr learned my addr 172.21.15.99:0/3554412158 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4ef400700 1 --2- 172.21.15.99:0/3554412158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f01608e0 0x7fb4f00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4efe00700 1 --2- 172.21.15.99:0/3554412158 >> v2:172.21.15.133:3300/0 conn(0x7fb4f00d3a70 0x7fb4f00d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 -- 172.21.15.99:0/3554412158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f01608e0 msgr2=0x7fb4f00d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 --2- 172.21.15.99:0/3554412158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f01608e0 0x7fb4f00d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 -- 172.21.15.99:0/3554412158 >> v2:172.21.15.133:3300/0 conn(0x7fb4f00d3a70 msgr2=0x7fb4f00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 --2- 172.21.15.99:0/3554412158 >> v2:172.21.15.133:3300/0 conn(0x7fb4f00d3a70 0x7fb4f00d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 -- 172.21.15.99:0/3554412158 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f00ce660 con 0x7fb4f0160cc0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.815+0000 7fb4eea00700 1 --2- 172.21.15.99:0/3554412158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ceb80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb4f008d890 tx=0x7fb4dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67fe2deba0fa7b7d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4ee000700 1 -- 172.21.15.99:0/3554412158 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00b870 con 0x7fb4f0160cc0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4ee000700 1 -- 172.21.15.99:0/3554412158 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00b9d0 con 0x7fb4f0160cc0 2024-03-29T09:11:23.816 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4ee000700 1 -- 172.21.15.99:0/3554412158 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00bd60 con 0x7fb4f0160cc0 2024-03-29T09:11:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 -- 172.21.15.99:0/3554412158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 msgr2=0x7fb4f00ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3554412158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ceb80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb4f008d890 tx=0x7fb4dc009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 -- 172.21.15.99:0/3554412158 shutdown_connections 2024-03-29T09:11:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3554412158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f01608e0 0x7fb4f00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.817 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3554412158 >> v2:172.21.15.133:3300/0 conn(0x7fb4f00d3a70 0x7fb4f00d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3554412158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ceb80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 -- 172.21.15.99:0/3554412158 >> 172.21.15.99:0/3554412158 conn(0x7fb4f009c490 msgr2=0x7fb4f00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 -- 172.21.15.99:0/3554412158 shutdown_connections 2024-03-29T09:11:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.816+0000 7fb4f6800700 1 -- 172.21.15.99:0/3554412158 wait complete. 2024-03-29T09:11:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.818+0000 7fb4f6800700 1 Processor -- start 2024-03-29T09:11:23.818 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.818+0000 7fb4f6800700 1 -- start start 2024-03-29T09:11:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.819+0000 7fb4f6800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f00d3a70 0x7fb4f01580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.819+0000 7fb4f6800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4f01608e0 0x7fb4f00ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.819+0000 7fb4f6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.819 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.819+0000 7fb4f6800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0157cb0 con 0x7fb4f0160cc0 2024-03-29T09:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.819+0000 7fb4f6800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0156d50 con 0x7fb4f01608e0 2024-03-29T09:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.819+0000 7fb4f6800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0157030 con 0x7fb4f00d3a70 2024-03-29T09:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43910/0 (socket says 172.21.15.99:43910) 2024-03-29T09:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 -- 172.21.15.99:0/3372185187 learned_addr learned my addr 172.21.15.99:0/3372185187 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4eea00700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4f01608e0 0x7fb4f00ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.820 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4ef400700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f00d3a70 0x7fb4f01580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 -- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f00d3a70 msgr2=0x7fb4f01580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f00d3a70 0x7fb4f01580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 -- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4f01608e0 msgr2=0x7fb4f00ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4f01608e0 0x7fb4f00ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 -- 172.21.15.99:0/3372185187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc009550 con 0x7fb4f0160cc0 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4efe00700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ca490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8007e40 tx=0x7fb4e800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4e7400700 1 -- 172.21.15.99:0/3372185187 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800ed30 con 0x7fb4f0160cc0 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.820+0000 7fb4e7400700 1 -- 172.21.15.99:0/3372185187 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8014420 con 0x7fb4f0160cc0 2024-03-29T09:11:23.821 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.821+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f0157310 con 0x7fb4f0160cc0 2024-03-29T09:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.821+0000 7fb4e7400700 1 -- 172.21.15.99:0/3372185187 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80147d0 con 0x7fb4f0160cc0 2024-03-29T09:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.821+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f01578d0 con 0x7fb4f0160cc0 2024-03-29T09:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.822+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f00677a0 con 0x7fb4f0160cc0 2024-03-29T09:11:23.822 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.823+0000 7fb4e7400700 1 -- 172.21.15.99:0/3372185187 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb4e8014930 con 0x7fb4f0160cc0 2024-03-29T09:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.823+0000 7fb4e7400700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4d006ac90 0x7fb4d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.823+0000 7fb4e7400700 1 -- 172.21.15.99:0/3372185187 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fb4e8089c20 con 0x7fb4f0160cc0 2024-03-29T09:11:23.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.824+0000 7fb4ef400700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4d006ac90 0x7fb4d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:23.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.824+0000 7fb4ef400700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4d006ac90 0x7fb4d006d150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006b20 tx=0x7fb4e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:23.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:23.828+0000 7fb4e7400700 1 -- 172.21.15.99:0/3372185187 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb4e8054bc0 con 0x7fb4f0160cc0 2024-03-29T09:11:24.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.052+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4f00d0700 con 0x7fb4f0160cc0 2024-03-29T09:11:24.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.054+0000 7fb4e7400700 1 -- 172.21.15.99:0/3372185187 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb4e80581e0 con 0x7fb4f0160cc0 2024-03-29T09:11:24.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4d006ac90 msgr2=0x7fb4d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4d006ac90 0x7fb4d006d150 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006b20 tx=0x7fb4e0008040 comp rx=0 tx=0).stop 2024-03-29T09:11:24.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 msgr2=0x7fb4f00ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ca490 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8007e40 tx=0x7fb4e800e3c0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 shutdown_connections 2024-03-29T09:11:24.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb4d006ac90 0x7fb4d006d150 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb4f0160cc0 0x7fb4f00ca490 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4f00d3a70 0x7fb4f01580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 --2- 172.21.15.99:0/3372185187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4f01608e0 0x7fb4f00ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 >> 172.21.15.99:0/3372185187 conn(0x7fb4f009c490 msgr2=0x7fb4f006c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:24.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 shutdown_connections 2024-03-29T09:11:24.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.057+0000 7fb4f6800700 1 -- 172.21.15.99:0/3372185187 wait complete. 2024-03-29T09:11:24.058 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:11:24.075 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:11:24.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:24.075 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:24.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[62772]: fuse finished with error 0 and tester_r 0 2024-03-29T09:11:24.140 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:11:24.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:11:24.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:24.197 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:11:24.239 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-03-29T09:11:24.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.476+0000 7fe221000700 1 Processor -- start 2024-03-29T09:11:24.477 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.476+0000 7fe221000700 1 -- start start 2024-03-29T09:11:24.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.478+0000 7fe221000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe21c0b3cc0 0x7fe21c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.478+0000 7fe221000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21c0d3810 con 0x7fe21c0d3380 2024-03-29T09:11:24.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.478+0000 7fe221000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21c0d1520 con 0x7fe21c160770 2024-03-29T09:11:24.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.478+0000 7fe221000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21c0b4140 con 0x7fe21c0b3cc0 2024-03-29T09:11:24.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.478+0000 7fe21a800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe21c0d3380 0x7fe21c0ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47426/0 (socket says 172.21.15.99:47426) 2024-03-29T09:11:24.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.478+0000 7fe21a800700 1 -- 172.21.15.99:0/3417636237 learned_addr learned my addr 172.21.15.99:0/3417636237 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.478+0000 7fe219400700 1 --2- 172.21.15.99:0/3417636237 >> v2:172.21.15.133:3300/0 conn(0x7fe21c0b3cc0 0x7fe21c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695170581 0 0) 0x7fe21c0d3810 con 0x7fe21c0d3380 2024-03-29T09:11:24.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2000035b0 con 0x7fe21c0d3380 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040999793 0 0) 0x7fe21c0d1520 con 0x7fe21c160770 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe21c0d3810 con 0x7fe21c160770 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246044054 0 0) 0x7fe2000035b0 con 0x7fe21c0d3380 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 >> v2:172.21.15.133:3300/0 conn(0x7fe21c0b3cc0 msgr2=0x7fe21c0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 --2- 172.21.15.99:0/3417636237 >> v2:172.21.15.133:3300/0 conn(0x7fe21c0b3cc0 0x7fe21c0d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 >> v1:172.21.15.133:6789/0 conn(0x7fe21c160770 legacy=0x7fe21c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c0d1800 con 0x7fe21c0d3380 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fe2140019f0 con 0x7fe21c0d3380 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 >> v1:172.21.15.99:6789/0 conn(0x7fe21c0d3380 legacy=0x7fe21c0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe200003b90 0x7fe200003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200004850 0x7fe200004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe200005210 0x7fe20000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20000c060 con 0x7fe200003b90 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20000c1d0 con 0x7fe200004850 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20000c390 con 0x7fe200005210 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.479+0000 7fe219e00700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe200003b90 0x7fe200003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe219400700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200004850 0x7fe200004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe21a800700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe200005210 0x7fe20000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe219e00700 1 -- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe200005210 msgr2=0x7fe20000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe219e00700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe200005210 0x7fe20000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe219e00700 1 -- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200004850 msgr2=0x7fe200004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe219e00700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200004850 0x7fe200004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe219e00700 1 -- 172.21.15.99:0/3417636237 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000035b0 con 0x7fe200003b90 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe21a800700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe200005210 0x7fe20000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe21a800700 1 -- 172.21.15.99:0/3417636237 reap_dead start 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe219e00700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe200003b90 0x7fe200003f90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe20c001940 tx=0x7fe200005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbe3576825acf65 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:24.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c001430 con 0x7fe200003b90 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c001d70 con 0x7fe200003b90 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.480+0000 7fe218a00700 1 -- 172.21.15.99:0/3417636237 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c005530 con 0x7fe200003b90 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.481+0000 7fe221000700 1 -- 172.21.15.99:0/3417636237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe200003b90 msgr2=0x7fe200003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.481+0000 7fe221000700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe200003b90 0x7fe200003f90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe20c001940 tx=0x7fe200005880 comp rx=0 tx=0).stop 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.481+0000 7fe221000700 1 -- 172.21.15.99:0/3417636237 shutdown_connections 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.481+0000 7fe221000700 1 --2- 172.21.15.99:0/3417636237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe200003b90 0x7fe200003f90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.481+0000 7fe221000700 1 -- 172.21.15.99:0/3417636237 >> 172.21.15.99:0/3417636237 conn(0x7fe21c08afd0 msgr2=0x7fe21c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.481+0000 7fe221000700 1 -- 172.21.15.99:0/3417636237 shutdown_connections 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.481+0000 7fe221000700 1 -- 172.21.15.99:0/3417636237 wait complete. 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.483+0000 7fe221000700 1 Processor -- start 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.483+0000 7fe221000700 1 -- start start 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe221000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200003b90 0x7fe21c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe221000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe21c1f5e30 0x7fe21c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe221000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c1f3020 0x7fe21c06bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe221000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c15dea0 con 0x7fe21c1f3020 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe221000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1f65d0 con 0x7fe200003b90 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe221000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1f68b0 con 0x7fe21c1f5e30 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c1f3020 0x7fe21c06bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c1f3020 0x7fe21c06bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43924/0 (socket says 172.21.15.99:43924) 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 -- 172.21.15.99:0/913392291 learned_addr learned my addr 172.21.15.99:0/913392291 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe219400700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe21c1f5e30 0x7fe21c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe219e00700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200003b90 0x7fe21c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 -- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe21c1f5e30 msgr2=0x7fe21c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe21c1f5e30 0x7fe21c15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 -- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200003b90 msgr2=0x7fe21c0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200003b90 0x7fe21c0cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe21a800700 1 -- 172.21.15.99:0/913392291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c1f6b90 con 0x7fe21c1f3020 2024-03-29T09:11:24.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.484+0000 7fe219400700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe21c1f5e30 0x7fe21c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:24.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.485+0000 7fe21a800700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c1f3020 0x7fe21c06bfe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe214001330 tx=0x7fe21400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:24.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.485+0000 7fe212000700 1 -- 172.21.15.99:0/913392291 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21400ed90 con 0x7fe21c1f3020 2024-03-29T09:11:24.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.485+0000 7fe212000700 1 -- 172.21.15.99:0/913392291 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214013420 con 0x7fe21c1f3020 2024-03-29T09:11:24.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.485+0000 7fe212000700 1 -- 172.21.15.99:0/913392291 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140137d0 con 0x7fe21c1f3020 2024-03-29T09:11:24.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.485+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c1f2b00 con 0x7fe21c1f3020 2024-03-29T09:11:24.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.485+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c1f62b0 con 0x7fe21c1f3020 2024-03-29T09:11:24.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.486+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21c15f3a0 con 0x7fe21c1f3020 2024-03-29T09:11:24.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.487+0000 7fe212000700 1 -- 172.21.15.99:0/913392291 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe214013930 con 0x7fe21c1f3020 2024-03-29T09:11:24.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.488+0000 7fe212000700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe1fc06ac90 0x7fe1fc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:24.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.488+0000 7fe212000700 1 -- 172.21.15.99:0/913392291 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fe214089bd0 con 0x7fe21c1f3020 2024-03-29T09:11:24.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.488+0000 7fe219e00700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe1fc06ac90 0x7fe1fc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:24.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.489+0000 7fe219e00700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe1fc06ac90 0x7fe1fc06d150 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe20c005040 tx=0x7fe20c00a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:24.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.492+0000 7fe212000700 1 -- 172.21.15.99:0/913392291 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe214054ad0 con 0x7fe21c1f3020 2024-03-29T09:11:24.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.780+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe21c1f2cf0 con 0x7fe21c1f3020 2024-03-29T09:11:24.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.784+0000 7fe212000700 1 -- 172.21.15.99:0/913392291 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe2140580f0 con 0x7fe21c1f3020 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe1fc06ac90 msgr2=0x7fe1fc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe1fc06ac90 0x7fe1fc06d150 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe20c005040 tx=0x7fe20c00a000 comp rx=0 tx=0).stop 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c1f3020 msgr2=0x7fe21c06bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c1f3020 0x7fe21c06bfe0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe214001330 tx=0x7fe21400e440 comp rx=0 tx=0).stop 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 shutdown_connections 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe1fc06ac90 0x7fe1fc06d150 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe21c1f3020 0x7fe21c06bfe0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe200003b90 0x7fe21c0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 --2- 172.21.15.99:0/913392291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe21c1f5e30 0x7fe21c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:24.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 >> 172.21.15.99:0/913392291 conn(0x7fe21c08afd0 msgr2=0x7fe21c15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:24.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 shutdown_connections 2024-03-29T09:11:24.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:24.787+0000 7fe221000700 1 -- 172.21.15.99:0/913392291 wait complete. 2024-03-29T09:11:24.788 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:11:24.805 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T09:11:24.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:11:24.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:11:24.975 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T09:11:24.976 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:24.976 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:25.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345608k,nr_inodes=4086402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-03-29T09:11:25.018 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:11:25.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:25.018 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:11:25.098 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-29T09:11:25.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:11:25.100 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:11:25.100 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:25.144 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:11:25.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:11:25.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:25.144 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T09:11:25.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:11:25.276 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T09:11:25.276 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:25.276 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:25.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345608k,nr_inodes=4086402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-03-29T09:11:25.320 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:11:25.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:25.320 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T09:11:25.401 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T09:11:25.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:11:25.402 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:11:25.402 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:25.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:11:25.445 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:11:25.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:25.445 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T09:11:25.525 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T09:11:25.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:11:25.527 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:11:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.717+0000 7f35cde00700 1 Processor -- start 2024-03-29T09:11:25.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.717+0000 7f35cde00700 1 -- start start 2024-03-29T09:11:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.718+0000 7f35cde00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c80d1e50 0x7f35c80cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.718+0000 7f35cde00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c80afde0 con 0x7f35c815d690 2024-03-29T09:11:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.718+0000 7f35cde00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c80d01d0 con 0x7f35c815da70 2024-03-29T09:11:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.718+0000 7f35cde00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80d04b0 con 0x7f35c80d1e50 2024-03-29T09:11:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.718+0000 7f35c6a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f35c815d690 0x7f35c80cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47430/0 (socket says 172.21.15.99:47430) 2024-03-29T09:11:25.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.718+0000 7f35c6a00700 1 -- 172.21.15.99:0/1452588879 learned_addr learned my addr 172.21.15.99:0/1452588879 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:25.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c7400700 1 --2- 172.21.15.99:0/1452588879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c80d1e50 0x7f35c80cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:25.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c5600700 1 -- 172.21.15.99:0/1452588879 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125742195 0 0) 0x7f35c80afde0 con 0x7f35c815d690 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c5600700 1 -- 172.21.15.99:0/1452588879 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ac0035b0 con 0x7f35c815d690 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c5600700 1 -- 172.21.15.99:0/1452588879 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001170485 0 0) 0x7f35c80d01d0 con 0x7f35c815da70 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c5600700 1 -- 172.21.15.99:0/1452588879 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c80afde0 con 0x7f35c815da70 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c7400700 1 -- 172.21.15.99:0/1452588879 >> v1:172.21.15.133:6789/0 conn(0x7f35c815da70 legacy=0x7f35c80d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c7400700 1 -- 172.21.15.99:0/1452588879 >> v1:172.21.15.99:6789/0 conn(0x7f35c815d690 legacy=0x7f35c80cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c7400700 1 -- 172.21.15.99:0/1452588879 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80b0030 con 0x7f35c80d1e50 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.719+0000 7f35c7400700 1 --2- 172.21.15.99:0/1452588879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c80d1e50 0x7f35c80cfb60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009dc0 tx=0x7f35c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff903a482d87cabd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35c5600700 1 -- 172.21.15.99:0/1452588879 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000e070 con 0x7f35c80d1e50 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35c5600700 1 -- 172.21.15.99:0/1452588879 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c000bb20 con 0x7f35c80d1e50 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35c5600700 1 -- 172.21.15.99:0/1452588879 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0012430 con 0x7f35c80d1e50 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35cde00700 1 -- 172.21.15.99:0/1452588879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c80d1e50 msgr2=0x7f35c80cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35cde00700 1 --2- 172.21.15.99:0/1452588879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c80d1e50 0x7f35c80cfb60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009dc0 tx=0x7f35c000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35cde00700 1 -- 172.21.15.99:0/1452588879 shutdown_connections 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35cde00700 1 --2- 172.21.15.99:0/1452588879 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c80d1e50 0x7f35c80cfb60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:25.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35cde00700 1 -- 172.21.15.99:0/1452588879 >> 172.21.15.99:0/1452588879 conn(0x7f35c809c480 msgr2=0x7f35c80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35cde00700 1 -- 172.21.15.99:0/1452588879 shutdown_connections 2024-03-29T09:11:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.720+0000 7f35cde00700 1 -- 172.21.15.99:0/1452588879 wait complete. 2024-03-29T09:11:25.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.723+0000 7f35cde00700 1 Processor -- start 2024-03-29T09:11:25.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.723+0000 7f35cde00700 1 -- start start 2024-03-29T09:11:25.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35cde00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80d1e50 0x7f35c80cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35cde00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c815d690 0x7f35c80cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35cde00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c815da70 0x7f35c815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35cde00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80cb300 con 0x7f35c815da70 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35cde00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c815f740 con 0x7f35c80d1e50 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35cde00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80cccd0 con 0x7f35c815d690 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35c6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80d1e50 0x7f35c80cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35c7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c815da70 0x7f35c815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35c6a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80d1e50 0x7f35c80cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.99:46966/0 (socket says 172.21.15.99:46966) 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35c6a00700 1 -- 172.21.15.99:0/1340709224 learned_addr learned my addr 172.21.15.99:0/1340709224 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.724+0000 7f35c6000700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c815d690 0x7f35c80cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35c6a00700 1 -- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c815d690 msgr2=0x7f35c80cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:25.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35c6a00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c815d690 0x7f35c80cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35c6a00700 1 -- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c815da70 msgr2=0x7f35c815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35c6a00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c815da70 0x7f35c815a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35c6a00700 1 -- 172.21.15.99:0/1340709224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c0009a70 con 0x7f35c80d1e50 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35c7400700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c815da70 0x7f35c815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35c6a00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80d1e50 0x7f35c80cadc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c700 tx=0x7f35b800ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35bf400700 1 -- 172.21.15.99:0/1340709224 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8005d70 con 0x7f35c80d1e50 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35bf400700 1 -- 172.21.15.99:0/1340709224 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b80070d0 con 0x7f35c80d1e50 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c815ad10 con 0x7f35c80d1e50 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35bf400700 1 -- 172.21.15.99:0/1340709224 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8007440 con 0x7f35c80d1e50 2024-03-29T09:11:25.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.725+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c815f0f0 con 0x7f35c80d1e50 2024-03-29T09:11:25.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.726+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c815ef30 con 0x7f35c80d1e50 2024-03-29T09:11:25.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.727+0000 7f35bf400700 1 -- 172.21.15.99:0/1340709224 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f35b80075a0 con 0x7f35c80d1e50 2024-03-29T09:11:25.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.728+0000 7f35bf400700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f35a806ac90 0x7f35a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:25.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.728+0000 7f35bf400700 1 -- 172.21.15.99:0/1340709224 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f35b80889c0 con 0x7f35c80d1e50 2024-03-29T09:11:25.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.728+0000 7f35c6000700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f35a806ac90 0x7f35a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:25.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.729+0000 7f35c6000700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f35a806ac90 0x7f35a806d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f35c80b3520 tx=0x7f35b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:25.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.733+0000 7f35bf400700 1 -- 172.21.15.99:0/1340709224 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f35b80538e0 con 0x7f35c80d1e50 2024-03-29T09:11:25.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.985+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35c80677a0 con 0x7f35c80d1e50 2024-03-29T09:11:25.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.986+0000 7f35bf400700 1 -- 172.21.15.99:0/1340709224 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f35b8017090 con 0x7f35c80d1e50 2024-03-29T09:11:25.986 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:25.986 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:10:43.923687+0000","modified":"2024-03-29T09:10:54.493701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/1841077010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1841077010},{"type":"v1","addr":"172.21.15.99:6835","nonce":1841077010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T09:11:25.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f35a806ac90 msgr2=0x7f35a806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:25.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f35a806ac90 0x7f35a806d150 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f35c80b3520 tx=0x7f35b4009040 comp rx=0 tx=0).stop 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80d1e50 msgr2=0x7f35c80cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80d1e50 0x7f35c80cadc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c700 tx=0x7f35b800ca10 comp rx=0 tx=0).stop 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 shutdown_connections 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f35a806ac90 0x7f35a806d150 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35c815da70 0x7f35c815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35c80d1e50 0x7f35c80cadc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 --2- 172.21.15.99:0/1340709224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35c815d690 0x7f35c80cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 >> 172.21.15.99:0/1340709224 conn(0x7f35c809c480 msgr2=0x7f35c80b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 shutdown_connections 2024-03-29T09:11:25.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:25.988+0000 7f35cde00700 1 -- 172.21.15.99:0/1340709224 wait complete. 2024-03-29T09:11:25.990 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2024-03-29T09:11:26.006 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:26.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.196+0000 7efe01c00700 1 Processor -- start 2024-03-29T09:11:26.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.197+0000 7efe01c00700 1 -- start start 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efe01c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efdfc1608e0 0x7efdfc0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efe01c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efdfc160cc0 0x7efdfc0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efe01c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc0d3a70 0x7efdfc0d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efe01c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0cddc0 con 0x7efdfc1608e0 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efe01c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0ce0a0 con 0x7efdfc160cc0 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efe01c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0ce380 con 0x7efdfc0d3a70 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efdfc1608e0 0x7efdfc0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efdfc1608e0 0x7efdfc0d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43936/0 (socket says 172.21.15.99:43936) 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 -- 172.21.15.99:0/789491780 learned_addr learned my addr 172.21.15.99:0/789491780 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:26.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfa000700 1 --2- 172.21.15.99:0/789491780 >> v2:172.21.15.133:3300/0 conn(0x7efdfc160cc0 0x7efdfc0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfb400700 1 --2- 172.21.15.99:0/789491780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc0d3a70 0x7efdfc0d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 -- 172.21.15.99:0/789491780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc0d3a70 msgr2=0x7efdfc0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 --2- 172.21.15.99:0/789491780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc0d3a70 0x7efdfc0d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 -- 172.21.15.99:0/789491780 >> v2:172.21.15.133:3300/0 conn(0x7efdfc160cc0 msgr2=0x7efdfc0ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 --2- 172.21.15.99:0/789491780 >> v2:172.21.15.133:3300/0 conn(0x7efdfc160cc0 0x7efdfc0ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfaa00700 1 -- 172.21.15.99:0/789491780 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0ce660 con 0x7efdfc1608e0 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfa000700 1 --2- 172.21.15.99:0/789491780 >> v2:172.21.15.133:3300/0 conn(0x7efdfc160cc0 0x7efdfc0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.198+0000 7efdfb400700 1 --2- 172.21.15.99:0/789491780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc0d3a70 0x7efdfc0d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efdfaa00700 1 --2- 172.21.15.99:0/789491780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc0d0150 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efdfc06ac50 tx=0x7efdec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede9ce566a2faeae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efdf9600700 1 -- 172.21.15.99:0/789491780 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00b870 con 0x7efdfc1608e0 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efdf9600700 1 -- 172.21.15.99:0/789491780 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec00b9d0 con 0x7efdfc1608e0 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efdf9600700 1 -- 172.21.15.99:0/789491780 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00bd60 con 0x7efdfc1608e0 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efe01c00700 1 -- 172.21.15.99:0/789491780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 msgr2=0x7efdfc0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efe01c00700 1 --2- 172.21.15.99:0/789491780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc0d0150 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7efdfc06ac50 tx=0x7efdec009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efe01c00700 1 -- 172.21.15.99:0/789491780 shutdown_connections 2024-03-29T09:11:26.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efe01c00700 1 --2- 172.21.15.99:0/789491780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc0d3a70 0x7efdfc0d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efe01c00700 1 --2- 172.21.15.99:0/789491780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc0d0150 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efe01c00700 1 --2- 172.21.15.99:0/789491780 >> v2:172.21.15.133:3300/0 conn(0x7efdfc160cc0 0x7efdfc0ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.199+0000 7efe01c00700 1 -- 172.21.15.99:0/789491780 >> 172.21.15.99:0/789491780 conn(0x7efdfc09c490 msgr2=0x7efdfc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:26.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.200+0000 7efe01c00700 1 -- 172.21.15.99:0/789491780 shutdown_connections 2024-03-29T09:11:26.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.200+0000 7efe01c00700 1 -- 172.21.15.99:0/789491780 wait complete. 2024-03-29T09:11:26.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.201+0000 7efe01c00700 1 Processor -- start 2024-03-29T09:11:26.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.201+0000 7efe01c00700 1 -- start start 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.202+0000 7efe01c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc0d3a70 0x7efdfc0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.202+0000 7efe01c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc1f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.202+0000 7efe01c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc160cc0 0x7efdfc1f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.202+0000 7efe01c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0c9c00 con 0x7efdfc1608e0 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.202+0000 7efe01c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc0ca8b0 con 0x7efdfc0d3a70 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.202+0000 7efe01c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc157fa0 con 0x7efdfc160cc0 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.202+0000 7efdfa000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc1f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc1f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43944/0 (socket says 172.21.15.99:43944) 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 -- 172.21.15.99:0/1126745151 learned_addr learned my addr 172.21.15.99:0/1126745151 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:26.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfaa00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc0d3a70 0x7efdfc0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfb400700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc160cc0 0x7efdfc1f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 -- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc160cc0 msgr2=0x7efdfc1f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc160cc0 0x7efdfc1f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 -- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc0d3a70 msgr2=0x7efdfc0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc0d3a70 0x7efdfc0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 -- 172.21.15.99:0/1126745151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009550 con 0x7efdfc1608e0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.203+0000 7efdfa000700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc1f21a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efde800c7f0 tx=0x7efde800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.204+0000 7efdf2a00700 1 -- 172.21.15.99:0/1126745151 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8011800 con 0x7efdfc1608e0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.204+0000 7efdf2a00700 1 -- 172.21.15.99:0/1126745151 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde8011960 con 0x7efdfc1608e0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.204+0000 7efdf2a00700 1 -- 172.21.15.99:0/1126745151 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde8011d10 con 0x7efdfc1608e0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.204+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc158240 con 0x7efdfc1608e0 2024-03-29T09:11:26.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.204+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0cd520 con 0x7efdfc1608e0 2024-03-29T09:11:26.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.205+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc0677a0 con 0x7efdfc1608e0 2024-03-29T09:11:26.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.206+0000 7efdf2a00700 1 -- 172.21.15.99:0/1126745151 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efde8018360 con 0x7efdfc1608e0 2024-03-29T09:11:26.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.206+0000 7efdf2a00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efddc06ac90 0x7efddc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.206+0000 7efdf2a00700 1 -- 172.21.15.99:0/1126745151 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7efde80893f0 con 0x7efdfc1608e0 2024-03-29T09:11:26.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.207+0000 7efdfaa00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efddc06ac90 0x7efddc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.207+0000 7efdfaa00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efddc06ac90 0x7efddc06d150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdec000c00 tx=0x7efdec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:26.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.211+0000 7efdf2a00700 1 -- 172.21.15.99:0/1126745151 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efde80542f0 con 0x7efdfc1608e0 2024-03-29T09:11:26.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.415+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdfc0d0700 con 0x7efdfc1608e0 2024-03-29T09:11:26.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.417+0000 7efdf2a00700 1 -- 172.21.15.99:0/1126745151 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7efde8057910 con 0x7efdfc1608e0 2024-03-29T09:11:26.417 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:26.417 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:26.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efddc06ac90 msgr2=0x7efddc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efddc06ac90 0x7efddc06d150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efdec000c00 tx=0x7efdec008030 comp rx=0 tx=0).stop 2024-03-29T09:11:26.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 msgr2=0x7efdfc1f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc1f21a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efde800c7f0 tx=0x7efde800cb00 comp rx=0 tx=0).stop 2024-03-29T09:11:26.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 shutdown_connections 2024-03-29T09:11:26.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efddc06ac90 0x7efddc06d150 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdfc160cc0 0x7efdfc1f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc0d3a70 0x7efdfc0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 --2- 172.21.15.99:0/1126745151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdfc1608e0 0x7efdfc1f21a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 >> 172.21.15.99:0/1126745151 conn(0x7efdfc09c490 msgr2=0x7efdfc06c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:26.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 shutdown_connections 2024-03-29T09:11:26.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.419+0000 7efe01c00700 1 -- 172.21.15.99:0/1126745151 wait complete. 2024-03-29T09:11:26.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:10:42.037884+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:10:42.900862+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2024-03-29T09:10:53.257132+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:11:26.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.634+0000 7fbdcc200700 1 Processor -- start 2024-03-29T09:11:26.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.634+0000 7fbdcc200700 1 -- start start 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdcc200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbdc415d420 0x7fbdc40d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdcc200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdcc200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc40afdd0 con 0x7fbdc415d800 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdcc200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc40d12b0 con 0x7fbdc415d420 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdcc200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40d3b00 con 0x7fbdc40d3560 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdc9800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbdc415d420 0x7fbdc40d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdc8e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fbdc415d800 0x7fbdc40d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47444/0 (socket says 172.21.15.99:47444) 2024-03-29T09:11:26.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.635+0000 7fbdc8e00700 1 -- 172.21.15.99:0/3926189527 learned_addr learned my addr 172.21.15.99:0/3926189527 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdca200700 1 --2- 172.21.15.99:0/3926189527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc9800700 1 -- 172.21.15.99:0/3926189527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 msgr2=0x7fbdc40d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc9800700 1 --2- 172.21.15.99:0/3926189527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc9800700 1 -- 172.21.15.99:0/3926189527 >> v1:172.21.15.99:6789/0 conn(0x7fbdc415d800 legacy=0x7fbdc40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc9800700 1 -- 172.21.15.99:0/3926189527 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc40b0070 con 0x7fbdc415d420 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc9800700 1 --2- 172.21.15.99:0/3926189527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40d1780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009840 tx=0x7fbdb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e957951d46a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc3e00700 1 -- 172.21.15.99:0/3926189527 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8011830 con 0x7fbdc415d420 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc3e00700 1 -- 172.21.15.99:0/3926189527 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8011990 con 0x7fbdc415d420 2024-03-29T09:11:26.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.636+0000 7fbdc3e00700 1 -- 172.21.15.99:0/3926189527 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8011d20 con 0x7fbdc415d420 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 -- 172.21.15.99:0/3926189527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 msgr2=0x7fbdc40d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 --2- 172.21.15.99:0/3926189527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40d1780 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009840 tx=0x7fbdb8009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 -- 172.21.15.99:0/3926189527 shutdown_connections 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 --2- 172.21.15.99:0/3926189527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 --2- 172.21.15.99:0/3926189527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40d1780 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 -- 172.21.15.99:0/3926189527 >> 172.21.15.99:0/3926189527 conn(0x7fbdc408afd0 msgr2=0x7fbdc409c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 -- 172.21.15.99:0/3926189527 shutdown_connections 2024-03-29T09:11:26.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.637+0000 7fbdcc200700 1 -- 172.21.15.99:0/3926189527 wait complete. 2024-03-29T09:11:26.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.639+0000 7fbdcc200700 1 Processor -- start 2024-03-29T09:11:26.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.639+0000 7fbdcc200700 1 -- start start 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdcc200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdcc200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdcc200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdc4154650 0x7fbdc40ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdcc200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40cb330 con 0x7fbdc415d420 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdcc200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40b6300 con 0x7fbdc4154650 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdcc200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40b6580 con 0x7fbdc40d3560 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40b5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43952/0 (socket says 172.21.15.99:43952) 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 -- 172.21.15.99:0/1079663577 learned_addr learned my addr 172.21.15.99:0/1079663577 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdca200700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdc4154650 0x7fbdc40ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc9800700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 -- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 msgr2=0x7fbdc40cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 -- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdc4154650 msgr2=0x7fbdc40ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdc4154650 0x7fbdc40ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc8e00700 1 -- 172.21.15.99:0/1079663577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80094a0 con 0x7fbdc415d420 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.640+0000 7fbdc9800700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.641+0000 7fbdc8e00700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40b5dc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c480 tx=0x7fbdb000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.641+0000 7fbdc1600700 1 -- 172.21.15.99:0/1079663577 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0005d70 con 0x7fbdc415d420 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.641+0000 7fbdc1600700 1 -- 172.21.15.99:0/1079663577 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb00070d0 con 0x7fbdc415d420 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.641+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40b6860 con 0x7fbdc415d420 2024-03-29T09:11:26.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.641+0000 7fbdc1600700 1 -- 172.21.15.99:0/1079663577 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb0007440 con 0x7fbdc415d420 2024-03-29T09:11:26.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.641+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc40b6e20 con 0x7fbdc415d420 2024-03-29T09:11:26.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.642+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc40678b0 con 0x7fbdc415d420 2024-03-29T09:11:26.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.643+0000 7fbdc1600700 1 -- 172.21.15.99:0/1079663577 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbdb00075a0 con 0x7fbdc415d420 2024-03-29T09:11:26.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.644+0000 7fbdc1600700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbda406ac90 0x7fbda406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:26.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.644+0000 7fbdc1600700 1 -- 172.21.15.99:0/1079663577 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fbdb0088ad0 con 0x7fbdc415d420 2024-03-29T09:11:26.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.644+0000 7fbdc9800700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbda406ac90 0x7fbda406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:26.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.645+0000 7fbdc9800700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbda406ac90 0x7fbda406d150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80090b0 tx=0x7fbdb8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:26.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.648+0000 7fbdc1600700 1 -- 172.21.15.99:0/1079663577 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbdb00539f0 con 0x7fbdc415d420 2024-03-29T09:11:26.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.848+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbdc40b23d0 con 0x7fbdc415d420 2024-03-29T09:11:26.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.849+0000 7fbdc1600700 1 -- 172.21.15.99:0/1079663577 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbdb00170a0 con 0x7fbdc415d420 2024-03-29T09:11:26.849 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:11:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbda406ac90 msgr2=0x7fbda406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbda406ac90 0x7fbda406d150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80090b0 tx=0x7fbdb8008030 comp rx=0 tx=0).stop 2024-03-29T09:11:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 msgr2=0x7fbdc40b5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40b5dc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbdb000c480 tx=0x7fbdb000c840 comp rx=0 tx=0).stop 2024-03-29T09:11:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 shutdown_connections 2024-03-29T09:11:26.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbda406ac90 0x7fbda406d150 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdc4154650 0x7fbdc40ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbdc40d3560 0x7fbdc40cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 --2- 172.21.15.99:0/1079663577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbdc415d420 0x7fbdc40b5dc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 >> 172.21.15.99:0/1079663577 conn(0x7fbdc408afd0 msgr2=0x7fbdc406c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 shutdown_connections 2024-03-29T09:11:26.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:26.851+0000 7fbdcc200700 1 -- 172.21.15.99:0/1079663577 wait complete. 2024-03-29T09:11:26.870 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:11:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.063+0000 7fac0ba00700 1 Processor -- start 2024-03-29T09:11:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.063+0000 7fac0ba00700 1 -- start start 2024-03-29T09:11:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac0ba00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac04160770 0x7fac040b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac0ba00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040d1520 con 0x7fac040d3550 2024-03-29T09:11:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac0ba00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac040d1800 con 0x7fac040d0d60 2024-03-29T09:11:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac0ba00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac040d1ae0 con 0x7fac04160770 2024-03-29T09:11:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac09000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac04160770 0x7fac040b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac09000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fac04160770 0x7fac040b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43968/0 (socket says 172.21.15.99:43968) 2024-03-29T09:11:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac09000700 1 -- 172.21.15.99:0/133606991 learned_addr learned my addr 172.21.15.99:0/133606991 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:27.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.064+0000 7fac09a00700 1 --1- 172.21.15.99:0/133606991 >> v1:172.21.15.99:6789/0 conn(0x7fac040d3550 0x7fac040ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47458/0 (socket says 172.21.15.99:47458) 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.065+0000 7fac09000700 1 -- 172.21.15.99:0/133606991 >> v1:172.21.15.133:6789/0 conn(0x7fac040d0d60 legacy=0x7fac040d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.065+0000 7fac09000700 1 -- 172.21.15.99:0/133606991 >> v1:172.21.15.99:6789/0 conn(0x7fac040d3550 legacy=0x7fac040ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.065+0000 7fac09000700 1 -- 172.21.15.99:0/133606991 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040d1dc0 con 0x7fac04160770 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.065+0000 7fac09000700 1 --2- 172.21.15.99:0/133606991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040b3cc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fabf80098f0 tx=0x7fabf8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe5de4d1936bcef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.065+0000 7fac03400700 1 -- 172.21.15.99:0/133606991 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800b8a0 con 0x7fac04160770 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.065+0000 7fac03400700 1 -- 172.21.15.99:0/133606991 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf800ba00 con 0x7fac04160770 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.065+0000 7fac03400700 1 -- 172.21.15.99:0/133606991 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf800bd90 con 0x7fac04160770 2024-03-29T09:11:27.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.066+0000 7fac0ba00700 1 -- 172.21.15.99:0/133606991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 msgr2=0x7fac040b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.066+0000 7fac0ba00700 1 --2- 172.21.15.99:0/133606991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040b3cc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fabf80098f0 tx=0x7fabf8009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.066+0000 7fac0ba00700 1 -- 172.21.15.99:0/133606991 shutdown_connections 2024-03-29T09:11:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.066+0000 7fac0ba00700 1 --2- 172.21.15.99:0/133606991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040b3cc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.066+0000 7fac0ba00700 1 -- 172.21.15.99:0/133606991 >> 172.21.15.99:0/133606991 conn(0x7fac0408afd0 msgr2=0x7fac0409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.066+0000 7fac0ba00700 1 -- 172.21.15.99:0/133606991 shutdown_connections 2024-03-29T09:11:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.066+0000 7fac0ba00700 1 -- 172.21.15.99:0/133606991 wait complete. 2024-03-29T09:11:27.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.068+0000 7fac0ba00700 1 Processor -- start 2024-03-29T09:11:27.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.068+0000 7fac0ba00700 1 -- start start 2024-03-29T09:11:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac0ba00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac040d0d60 0x7fac040ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac0ba00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac040d3550 0x7fac041f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac0ba00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac0ba00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040cb200 con 0x7fac04160770 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac0ba00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac040b9030 con 0x7fac040d0d60 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac0ba00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac041f2a20 con 0x7fac040d3550 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac09a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac09a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43984/0 (socket says 172.21.15.99:43984) 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac09a00700 1 -- 172.21.15.99:0/1560057932 learned_addr learned my addr 172.21.15.99:0/1560057932 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac03e00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac040d3550 0x7fac041f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.069+0000 7fac09000700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac040d0d60 0x7fac040ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac09a00700 1 -- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac040d3550 msgr2=0x7fac041f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac09a00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac040d3550 0x7fac041f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac09a00700 1 -- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac040d0d60 msgr2=0x7fac040ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac09a00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac040d0d60 0x7fac040ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac09a00700 1 -- 172.21.15.99:0/1560057932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf8009550 con 0x7fac04160770 2024-03-29T09:11:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac09a00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040cf7f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00cba0 tx=0x7fabfc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac00c00700 1 -- 172.21.15.99:0/1560057932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc00aaa0 con 0x7fac04160770 2024-03-29T09:11:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac00c00700 1 -- 172.21.15.99:0/1560057932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabfc00ac00 con 0x7fac04160770 2024-03-29T09:11:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac00c00700 1 -- 172.21.15.99:0/1560057932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabfc007070 con 0x7fac04160770 2024-03-29T09:11:27.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac040ca960 con 0x7fac04160770 2024-03-29T09:11:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.070+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac040caf20 con 0x7fac04160770 2024-03-29T09:11:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.071+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac040678b0 con 0x7fac04160770 2024-03-29T09:11:27.072 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.072+0000 7fac00c00700 1 -- 172.21.15.99:0/1560057932 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fabfc0071d0 con 0x7fac04160770 2024-03-29T09:11:27.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.073+0000 7fac00c00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ac90 0x7fabe406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.073+0000 7fac00c00700 1 -- 172.21.15.99:0/1560057932 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fabfc0891b0 con 0x7fac04160770 2024-03-29T09:11:27.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.073+0000 7fac09000700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ac90 0x7fabe406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.073 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.074+0000 7fac09000700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ac90 0x7fabe406d150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fabf8014040 tx=0x7fabf8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.077+0000 7fac00c00700 1 -- 172.21.15.99:0/1560057932 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fabfc054160 con 0x7fac04160770 2024-03-29T09:11:27.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.278+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fac040b43c0 con 0x7fac04160770 2024-03-29T09:11:27.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.278+0000 7fac00c00700 1 -- 172.21.15.99:0/1560057932 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabfc017020 con 0x7fac04160770 2024-03-29T09:11:27.278 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.280+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ac90 msgr2=0x7fabe406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.280+0000 7fac0ba00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ac90 0x7fabe406d150 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fabf8014040 tx=0x7fabf8008030 comp rx=0 tx=0).stop 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.280+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 msgr2=0x7fac040cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.280+0000 7fac0ba00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040cf7f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fabfc00cba0 tx=0x7fabfc00ceb0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 shutdown_connections 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fabe406ac90 0x7fabe406d150 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fac04160770 0x7fac040cf7f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac040d0d60 0x7fac040ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 --2- 172.21.15.99:0/1560057932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac040d3550 0x7fac041f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 >> 172.21.15.99:0/1560057932 conn(0x7fac0408afd0 msgr2=0x7fac0415fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:27.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 shutdown_connections 2024-03-29T09:11:27.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.281+0000 7fac0ba00700 1 -- 172.21.15.99:0/1560057932 wait complete. 2024-03-29T09:11:27.299 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:11:27.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.491+0000 7fa032e00700 1 Processor -- start 2024-03-29T09:11:27.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.491+0000 7fa032e00700 1 -- start start 2024-03-29T09:11:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa032e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa032e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa02c0ac500 0x7fa02c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa032e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02c06c720 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa032e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02c0717a0 con 0x7fa02c0ac500 2024-03-29T09:11:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa032e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c159cc0 con 0x7fa02c0b3d40 2024-03-29T09:11:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa02be00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa02c0b1fd0 0x7fa02c070ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:47460/0 (socket says 172.21.15.99:47460) 2024-03-29T09:11:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa02be00700 1 -- 172.21.15.99:0/9123674 learned_addr learned my addr 172.21.15.99:0/9123674 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:27.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa030e00700 1 --2- 172.21.15.99:0/9123674 >> v2:172.21.15.133:3300/0 conn(0x7fa02c0ac500 0x7fa02c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa023e00700 1 --2- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa02b400700 1 -- 172.21.15.99:0/9123674 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164708044 0 0) 0x7fa02c06c720 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.494 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.493+0000 7fa02b400700 1 -- 172.21.15.99:0/9123674 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0100035b0 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa030e00700 1 -- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 msgr2=0x7fa02c0abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:27.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa030e00700 1 --2- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c0abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa02b400700 1 -- 172.21.15.99:0/9123674 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2368868874 0 0) 0x7fa0100035b0 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa030e00700 1 -- 172.21.15.99:0/9123674 >> v1:172.21.15.99:6789/0 conn(0x7fa02c0b1fd0 legacy=0x7fa02c070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa030e00700 1 -- 172.21.15.99:0/9123674 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c06ca00 con 0x7fa02c0ac500 2024-03-29T09:11:27.495 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa030e00700 1 --2- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 0x7fa02c06d4e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa024009d90 tx=0x7fa02400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18311e7220c7bf11 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa02b400700 1 -- 172.21.15.99:0/9123674 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02400e070 con 0x7fa02c0ac500 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa02b400700 1 -- 172.21.15.99:0/9123674 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa024003a20 con 0x7fa02c0ac500 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.494+0000 7fa02b400700 1 -- 172.21.15.99:0/9123674 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c001670 con 0x7fa02c0ac500 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 -- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 msgr2=0x7fa02c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 --2- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 0x7fa02c06d4e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa024009d90 tx=0x7fa02400b330 comp rx=0 tx=0).stop 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 -- 172.21.15.99:0/9123674 shutdown_connections 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 --2- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 0x7fa02c06d4e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 --2- 172.21.15.99:0/9123674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c0abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 -- 172.21.15.99:0/9123674 >> 172.21.15.99:0/9123674 conn(0x7fa02c066540 msgr2=0x7fa02c066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 -- 172.21.15.99:0/9123674 shutdown_connections 2024-03-29T09:11:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.495+0000 7fa032e00700 1 -- 172.21.15.99:0/9123674 wait complete. 2024-03-29T09:11:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.497+0000 7fa032e00700 1 Processor -- start 2024-03-29T09:11:27.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.498+0000 7fa032e00700 1 -- start start 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa032e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 0x7fa02c1f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa032e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa02c0b1fd0 0x7fa02c1f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa032e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa032e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c1f5840 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa032e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c153a70 con 0x7fa02c0ac500 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa032e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c153d50 con 0x7fa02c0b3d40 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa02c0b1fd0 0x7fa02c1f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa02c0b1fd0 0x7fa02c1f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:43988/0 (socket says 172.21.15.99:43988) 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 -- 172.21.15.99:0/115744951 learned_addr learned my addr 172.21.15.99:0/115744951 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa02be00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 0x7fa02c1f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa030e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 -- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 msgr2=0x7fa02c153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 -- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 msgr2=0x7fa02c1f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 0x7fa02c1f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa023e00700 1 -- 172.21.15.99:0/115744951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa024009a70 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.499+0000 7fa030e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.500+0000 7fa023e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa02c0b1fd0 0x7fa02c1f5300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa01400e870 tx=0x7fa01400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.500+0000 7fa028c00700 1 -- 172.21.15.99:0/115744951 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014009590 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.500+0000 7fa028c00700 1 -- 172.21.15.99:0/115744951 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa014013680 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.500+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02c154030 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.500+0000 7fa028c00700 1 -- 172.21.15.99:0/115744951 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140139f0 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.500+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02c099dc0 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.501+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02c09a130 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.502+0000 7fa028c00700 1 -- 172.21.15.99:0/115744951 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa014013b50 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.503+0000 7fa028c00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa00806ac90 0x7fa00806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.503+0000 7fa028c00700 1 -- 172.21.15.99:0/115744951 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fa014089590 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.503+0000 7fa02be00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa00806ac90 0x7fa00806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.503+0000 7fa02be00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa00806ac90 0x7fa00806d150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa01c000f60 tx=0x7fa01c001230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.507+0000 7fa028c00700 1 -- 172.21.15.99:0/115744951 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa014054400 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.709+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa02c0678b0 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.709 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.710+0000 7fa028c00700 1 -- 172.21.15.99:0/115744951 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa014057a20 con 0x7fa02c0b1fd0 2024-03-29T09:11:27.710 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:11:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa00806ac90 msgr2=0x7fa00806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa00806ac90 0x7fa00806d150 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa01c000f60 tx=0x7fa01c001230 comp rx=0 tx=0).stop 2024-03-29T09:11:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa02c0b1fd0 msgr2=0x7fa02c1f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa02c0b1fd0 0x7fa02c1f5300 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa01400e870 tx=0x7fa01400eb80 comp rx=0 tx=0).stop 2024-03-29T09:11:27.712 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 shutdown_connections 2024-03-29T09:11:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa00806ac90 0x7fa00806d150 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa02c0b3d40 0x7fa02c153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa02c0ac500 0x7fa02c1f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 --2- 172.21.15.99:0/115744951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa02c0b1fd0 0x7fa02c1f5300 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 >> 172.21.15.99:0/115744951 conn(0x7fa02c066540 msgr2=0x7fa02c066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 shutdown_connections 2024-03-29T09:11:27.713 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.712+0000 7fa032e00700 1 -- 172.21.15.99:0/115744951 wait complete. 2024-03-29T09:11:27.730 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2024-03-29T09:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.922+0000 7f965a000700 1 Processor -- start 2024-03-29T09:11:27.922 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.922+0000 7f965a000700 1 -- start start 2024-03-29T09:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.923+0000 7f965a000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96540d1220 0x7f96540d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.923 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.923+0000 7f965a000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96540d1680 con 0x7f96540d3130 2024-03-29T09:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.923+0000 7f965a000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965415d800 con 0x7f965415d420 2024-03-29T09:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.923+0000 7f965a000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96540d1900 con 0x7f96540d1220 2024-03-29T09:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9653e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f96540d3130 0x7f96540b3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45364/0 (socket says 172.21.15.99:45364) 2024-03-29T09:11:27.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9653e00700 1 -- 172.21.15.99:0/433933512 learned_addr learned my addr 172.21.15.99:0/433933512 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652a00700 1 --2- 172.21.15.99:0/433933512 >> v2:172.21.15.133:3300/0 conn(0x7f96540d1220 0x7f96540d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006896378 0 0) 0x7f96540d1680 con 0x7f96540d3130 2024-03-29T09:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96380035b0 con 0x7f96540d3130 2024-03-29T09:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089433806 0 0) 0x7f96380035b0 con 0x7f96540d3130 2024-03-29T09:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 >> v2:172.21.15.133:3300/0 conn(0x7f96540d1220 msgr2=0x7f96540d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:27.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 --2- 172.21.15.99:0/433933512 >> v2:172.21.15.133:3300/0 conn(0x7f96540d1220 0x7f96540d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 >> v1:172.21.15.133:6789/0 conn(0x7f965415d420 legacy=0x7f96540ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540d1be0 con 0x7f96540d3130 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.924+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f964c0019f0 con 0x7f96540d3130 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 >> v1:172.21.15.99:6789/0 conn(0x7f96540d3130 legacy=0x7f96540b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652000700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f9638003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652000700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9638004850 0x7f9638004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652000700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9638005210 0x7f963800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540d1900 con 0x7f9638003a50 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963800c110 con 0x7f9638005210 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963800c2d0 con 0x7f9638004850 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653400700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f9638003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9652a00700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9638004850 0x7f9638004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653e00700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9638005210 0x7f963800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653400700 1 -- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9638004850 msgr2=0x7f9638004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653400700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9638004850 0x7f9638004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653400700 1 -- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9638005210 msgr2=0x7f963800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653400700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9638005210 0x7f963800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653400700 1 -- 172.21.15.99:0/433933512 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96380035b0 con 0x7f9638003a50 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653e00700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9638005210 0x7f963800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.925+0000 7f9653e00700 1 -- 172.21.15.99:0/433933512 reap_dead start 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f9653400700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f9638003e70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f96440017c0 tx=0x7f9638005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c5527dd043cc8c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96440052d0 con 0x7f9638003a50 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9644005430 con 0x7f9638003a50 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f9652000700 1 -- 172.21.15.99:0/433933512 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96440038c0 con 0x7f9638003a50 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f965a000700 1 -- 172.21.15.99:0/433933512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 msgr2=0x7f9638003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f965a000700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f9638003e70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f96440017c0 tx=0x7f9638005880 comp rx=0 tx=0).stop 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f965a000700 1 -- 172.21.15.99:0/433933512 shutdown_connections 2024-03-29T09:11:27.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f965a000700 1 --2- 172.21.15.99:0/433933512 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f9638003e70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f965a000700 1 -- 172.21.15.99:0/433933512 >> 172.21.15.99:0/433933512 conn(0x7f965408afd0 msgr2=0x7f965409c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:27.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f965a000700 1 -- 172.21.15.99:0/433933512 shutdown_connections 2024-03-29T09:11:27.930 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.926+0000 7f965a000700 1 -- 172.21.15.99:0/433933512 wait complete. 2024-03-29T09:11:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.928+0000 7f965a000700 1 Processor -- start 2024-03-29T09:11:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.928+0000 7f965a000700 1 -- start start 2024-03-29T09:11:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f965a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f96540cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f965a000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9654152a70 0x7f96540cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f965a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96540c6f30 0x7f96540c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f965a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965415dc30 con 0x7f9638003a50 2024-03-29T09:11:27.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f965a000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540cc440 con 0x7f96540c6f30 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f965a000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540c7fe0 con 0x7f9654152a70 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f9653400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f96540cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f9653400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f96540cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40554/0 (socket says 172.21.15.99:40554) 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.929+0000 7f9653400700 1 -- 172.21.15.99:0/2889513173 learned_addr learned my addr 172.21.15.99:0/2889513173 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9652a00700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9654152a70 0x7f96540cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9653e00700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96540c6f30 0x7f96540c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9653400700 1 -- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9654152a70 msgr2=0x7f96540cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9653400700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9654152a70 0x7f96540cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9653400700 1 -- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96540c6f30 msgr2=0x7f96540c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9653400700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96540c6f30 0x7f96540c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9653400700 1 -- 172.21.15.99:0/2889513173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540c8280 con 0x7f9638003a50 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9652a00700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9654152a70 0x7f96540cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f9653400700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f96540cb9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9644015040 tx=0x7f9644001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f964b400700 1 -- 172.21.15.99:0/2889513173 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964401d680 con 0x7f9638003a50 2024-03-29T09:11:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f964b400700 1 -- 172.21.15.99:0/2889513173 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964401d800 con 0x7f9638003a50 2024-03-29T09:11:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f964b400700 1 -- 172.21.15.99:0/2889513173 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964401dbb0 con 0x7f9638003a50 2024-03-29T09:11:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96540c8560 con 0x7f9638003a50 2024-03-29T09:11:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.930+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96540c7900 con 0x7f9638003a50 2024-03-29T09:11:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.931+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96540cdbd0 con 0x7f9638003a50 2024-03-29T09:11:27.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.932+0000 7f964b400700 1 -- 172.21.15.99:0/2889513173 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9644019040 con 0x7f9638003a50 2024-03-29T09:11:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.933+0000 7f964b400700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f963406ac90 0x7f963406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.933+0000 7f964b400700 1 -- 172.21.15.99:0/2889513173 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f964408a1f0 con 0x7f9638003a50 2024-03-29T09:11:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.933+0000 7f9652a00700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f963406ac90 0x7f963406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:27.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.934+0000 7f9652a00700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f963406ac90 0x7f963406d150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9654152eb0 tx=0x7f9640008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:27.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:27.938+0000 7f964b400700 1 -- 172.21.15.99:0/2889513173 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f96440550f0 con 0x7f9638003a50 2024-03-29T09:11:28.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.144+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f96540678b0 con 0x7f9638003a50 2024-03-29T09:11:28.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.144+0000 7f964b400700 1 -- 172.21.15.99:0/2889513173 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v42) v1 ==== 97+0+11 (secure 0 0 0) 0x7f9644058710 con 0x7f9638003a50 2024-03-29T09:11:28.144 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:28.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f963406ac90 msgr2=0x7f963406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f963406ac90 0x7f963406d150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9654152eb0 tx=0x7f9640008040 comp rx=0 tx=0).stop 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 msgr2=0x7f96540cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f96540cb9c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9644015040 tx=0x7f9644001120 comp rx=0 tx=0).stop 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 shutdown_connections 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f963406ac90 0x7f963406d150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96540c6f30 0x7f96540c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9638003a50 0x7f96540cb9c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 --2- 172.21.15.99:0/2889513173 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9654152a70 0x7f96540cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.146+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 >> 172.21.15.99:0/2889513173 conn(0x7f965408afd0 msgr2=0x7f9654073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.147+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 shutdown_connections 2024-03-29T09:11:28.147 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.147+0000 7f965a000700 1 -- 172.21.15.99:0/2889513173 wait complete. 2024-03-29T09:11:28.164 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:11:28.352 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.352+0000 7fd615000700 1 Processor -- start 2024-03-29T09:11:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.353+0000 7fd615000700 1 -- start start 2024-03-29T09:11:28.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd615000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6101608e0 0x7fd6100d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd615000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd610160cc0 0x7fd6100ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd615000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6100cddc0 con 0x7fd6100d3a70 2024-03-29T09:11:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd615000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6100ce0a0 con 0x7fd610160cc0 2024-03-29T09:11:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd615000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6100ce380 con 0x7fd6101608e0 2024-03-29T09:11:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd60e800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd6100d3a70 0x7fd6100d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45370/0 (socket says 172.21.15.99:45370) 2024-03-29T09:11:28.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd60e800700 1 -- 172.21.15.99:0/421104777 learned_addr learned my addr 172.21.15.99:0/421104777 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd60d400700 1 --2- 172.21.15.99:0/421104777 >> v2:172.21.15.133:3300/0 conn(0x7fd610160cc0 0x7fd6100ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd60de00700 1 --2- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6101608e0 0x7fd6100d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd60ca00700 1 -- 172.21.15.99:0/421104777 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251781439 0 0) 0x7fd6100cddc0 con 0x7fd6100d3a70 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.354+0000 7fd60ca00700 1 -- 172.21.15.99:0/421104777 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5f40035b0 con 0x7fd6100d3a70 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60ca00700 1 -- 172.21.15.99:0/421104777 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2358537517 0 0) 0x7fd5f40035b0 con 0x7fd6100d3a70 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60d400700 1 -- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6101608e0 msgr2=0x7fd6100d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60d400700 1 --2- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6101608e0 0x7fd6100d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60d400700 1 -- 172.21.15.99:0/421104777 >> v1:172.21.15.99:6789/0 conn(0x7fd6100d3a70 legacy=0x7fd6100d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60d400700 1 -- 172.21.15.99:0/421104777 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100ce660 con 0x7fd610160cc0 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60de00700 1 --2- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6101608e0 0x7fd6100d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60d400700 1 --2- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd610160cc0 0x7fd6100ceb80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009840 tx=0x7fd5fc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca88bd72047b7931 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60ca00700 1 -- 172.21.15.99:0/421104777 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080019f0 con 0x7fd610160cc0 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60ca00700 1 -- 172.21.15.99:0/421104777 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc004030 con 0x7fd610160cc0 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.355+0000 7fd60ca00700 1 -- 172.21.15.99:0/421104777 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0043c0 con 0x7fd610160cc0 2024-03-29T09:11:28.356 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 -- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd610160cc0 msgr2=0x7fd6100ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 --2- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd610160cc0 0x7fd6100ceb80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc009840 tx=0x7fd5fc009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 -- 172.21.15.99:0/421104777 shutdown_connections 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 --2- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6101608e0 0x7fd6100d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 --2- 172.21.15.99:0/421104777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd610160cc0 0x7fd6100ceb80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 -- 172.21.15.99:0/421104777 >> 172.21.15.99:0/421104777 conn(0x7fd61009c490 msgr2=0x7fd6100d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 -- 172.21.15.99:0/421104777 shutdown_connections 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.356+0000 7fd615000700 1 -- 172.21.15.99:0/421104777 wait complete. 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.358+0000 7fd615000700 1 Processor -- start 2024-03-29T09:11:28.357 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.358+0000 7fd615000700 1 -- start start 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd615000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6100d3a70 0x7fd61015a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd615000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6101608e0 0x7fd6100cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd615000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd610160cc0 0x7fd6100b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd615000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6101f5e00 con 0x7fd610160cc0 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd615000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6100cc7a0 con 0x7fd6101608e0 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd615000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6101f6050 con 0x7fd6100d3a70 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd610160cc0 0x7fd6100b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd610160cc0 0x7fd6100b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40566/0 (socket says 172.21.15.99:40566) 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 -- 172.21.15.99:0/2034412832 learned_addr learned my addr 172.21.15.99:0/2034412832 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:28.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60d400700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6101608e0 0x7fd6100cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60de00700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6100d3a70 0x7fd61015a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 -- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6100d3a70 msgr2=0x7fd61015a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6100d3a70 0x7fd61015a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 -- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6101608e0 msgr2=0x7fd6100cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6101608e0 0x7fd6100cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60e800700 1 -- 172.21.15.99:0/2034412832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6000094a0 con 0x7fd610160cc0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.359+0000 7fd60d400700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6101608e0 0x7fd6100cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.360+0000 7fd60e800700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd610160cc0 0x7fd6100b3470 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd60800cc00 tx=0x7fd60800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.360+0000 7fd606000700 1 -- 172.21.15.99:0/2034412832 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800ab80 con 0x7fd610160cc0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.360+0000 7fd606000700 1 -- 172.21.15.99:0/2034412832 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60800ad00 con 0x7fd610160cc0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.360+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc0094a0 con 0x7fd610160cc0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.360+0000 7fd606000700 1 -- 172.21.15.99:0/2034412832 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608007310 con 0x7fd610160cc0 2024-03-29T09:11:28.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.360+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6101f6610 con 0x7fd610160cc0 2024-03-29T09:11:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.361+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6100677a0 con 0x7fd610160cc0 2024-03-29T09:11:28.362 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.362+0000 7fd606000700 1 -- 172.21.15.99:0/2034412832 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd608007470 con 0x7fd610160cc0 2024-03-29T09:11:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.363+0000 7fd606000700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd5f006ac90 0x7fd5f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.363+0000 7fd606000700 1 -- 172.21.15.99:0/2034412832 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fd608089410 con 0x7fd610160cc0 2024-03-29T09:11:28.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.363+0000 7fd60de00700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd5f006ac90 0x7fd5f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.364 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.364+0000 7fd60de00700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd5f006ac90 0x7fd5f006d150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd600009d00 tx=0x7fd600008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:28.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.367+0000 7fd606000700 1 -- 172.21.15.99:0/2034412832 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd6080543b0 con 0x7fd610160cc0 2024-03-29T09:11:28.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.612+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6100d0700 con 0x7fd610160cc0 2024-03-29T09:11:28.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.613+0000 7fd606000700 1 -- 172.21.15.99:0/2034412832 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4742 (secure 0 0 0) 0x7fd6080579d0 con 0x7fd610160cc0 2024-03-29T09:11:28.613 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:28.613 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:10:43.923687+0000","modified":"2024-03-29T09:10:54.493701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/1841077010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1841077010},{"type":"v1","addr":"172.21.15.99:6835","nonce":1841077010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T09:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd5f006ac90 msgr2=0x7fd5f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd5f006ac90 0x7fd5f006d150 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd600009d00 tx=0x7fd600008040 comp rx=0 tx=0).stop 2024-03-29T09:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd610160cc0 msgr2=0x7fd6100b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd610160cc0 0x7fd6100b3470 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd60800cc00 tx=0x7fd60800a060 comp rx=0 tx=0).stop 2024-03-29T09:11:28.615 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 shutdown_connections 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd5f006ac90 0x7fd5f006d150 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd610160cc0 0x7fd6100b3470 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6100d3a70 0x7fd61015a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 --2- 172.21.15.99:0/2034412832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6101608e0 0x7fd6100cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 >> 172.21.15.99:0/2034412832 conn(0x7fd61009c490 msgr2=0x7fd6100cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 shutdown_connections 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.615+0000 7fd615000700 1 -- 172.21.15.99:0/2034412832 wait complete. 2024-03-29T09:11:28.616 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2024-03-29T09:11:28.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:28.823 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.823+0000 7f7a86400700 1 Processor -- start 2024-03-29T09:11:28.824 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.824+0000 7f7a86400700 1 -- start start 2024-03-29T09:11:28.826 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a86400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7a800d1b20 0x7f7a800d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a86400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a80160cc0 0x7f7a800ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a86400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a800cddc0 con 0x7f7a801608e0 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a86400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a800ce0a0 con 0x7f7a800d1b20 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a86400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800ce380 con 0x7f7a80160cc0 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a7ea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7a800d1b20 0x7f7a800d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a7ea00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7a800d1b20 0x7f7a800d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40578/0 (socket says 172.21.15.99:40578) 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a7ea00700 1 -- 172.21.15.99:0/1029877296 learned_addr learned my addr 172.21.15.99:0/1029877296 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a7fe00700 1 --2- 172.21.15.99:0/1029877296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a80160cc0 0x7f7a800ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a7ea00700 1 -- 172.21.15.99:0/1029877296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a80160cc0 msgr2=0x7f7a800ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a7ea00700 1 --2- 172.21.15.99:0/1029877296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a80160cc0 0x7f7a800ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.825+0000 7f7a7ea00700 1 -- 172.21.15.99:0/1029877296 >> v1:172.21.15.133:6789/0 conn(0x7f7a801608e0 legacy=0x7f7a800d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a7ea00700 1 -- 172.21.15.99:0/1029877296 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800ce660 con 0x7f7a800d1b20 2024-03-29T09:11:28.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a7e000700 1 -- 172.21.15.99:0/1029877296 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886167064 0 0) 0x7f7a800cddc0 con 0x7f7a801608e0 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a7ea00700 1 --2- 172.21.15.99:0/1029877296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a800d1b20 0x7f7a800d0150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0098f0 tx=0x7f7a6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d526f42a8c5df49 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a7e000700 1 -- 172.21.15.99:0/1029877296 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c011830 con 0x7f7a800d1b20 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a7e000700 1 -- 172.21.15.99:0/1029877296 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c011990 con 0x7f7a800d1b20 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a7e000700 1 -- 172.21.15.99:0/1029877296 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c011d20 con 0x7f7a800d1b20 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a86400700 1 -- 172.21.15.99:0/1029877296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a800d1b20 msgr2=0x7f7a800d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.826+0000 7f7a86400700 1 --2- 172.21.15.99:0/1029877296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a800d1b20 0x7f7a800d0150 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0098f0 tx=0x7f7a6c009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.827+0000 7f7a86400700 1 -- 172.21.15.99:0/1029877296 shutdown_connections 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.827+0000 7f7a86400700 1 --2- 172.21.15.99:0/1029877296 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a80160cc0 0x7f7a800ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.827+0000 7f7a86400700 1 --2- 172.21.15.99:0/1029877296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a800d1b20 0x7f7a800d0150 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.827+0000 7f7a86400700 1 -- 172.21.15.99:0/1029877296 >> 172.21.15.99:0/1029877296 conn(0x7f7a8009c490 msgr2=0x7f7a800d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.827+0000 7f7a86400700 1 -- 172.21.15.99:0/1029877296 shutdown_connections 2024-03-29T09:11:28.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.827+0000 7f7a86400700 1 -- 172.21.15.99:0/1029877296 wait complete. 2024-03-29T09:11:28.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.828+0000 7f7a86400700 1 Processor -- start 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.828+0000 7f7a86400700 1 -- start start 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.829+0000 7f7a86400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a800d1b20 0x7f7a801534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.829+0000 7f7a86400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a801608e0 0x7f7a80155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.829+0000 7f7a86400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a80160cc0 0x7f7a800b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.829+0000 7f7a86400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a801539e0 con 0x7f7a801608e0 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.829+0000 7f7a86400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80155f40 con 0x7f7a80160cc0 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.829+0000 7f7a86400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80156220 con 0x7f7a800d1b20 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a801608e0 0x7f7a80155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a801608e0 0x7f7a80155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40590/0 (socket says 172.21.15.99:40590) 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 -- 172.21.15.99:0/3241361114 learned_addr learned my addr 172.21.15.99:0/3241361114 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7f400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a800d1b20 0x7f7a801534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.831 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7fe00700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a80160cc0 0x7f7a800b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 -- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a800d1b20 msgr2=0x7f7a801534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a800d1b20 0x7f7a801534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 -- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a80160cc0 msgr2=0x7f7a800b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a80160cc0 0x7f7a800b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 -- 172.21.15.99:0/3241361114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c009550 con 0x7f7a801608e0 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a7ea00700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a801608e0 0x7f7a80155560 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009c40 tx=0x7f7a6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a77400700 1 -- 172.21.15.99:0/3241361114 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c004480 con 0x7f7a801608e0 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a77400700 1 -- 172.21.15.99:0/3241361114 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c01f070 con 0x7f7a801608e0 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.830+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a80156500 con 0x7f7a801608e0 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.831+0000 7f7a77400700 1 -- 172.21.15.99:0/3241361114 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c0038c0 con 0x7f7a801608e0 2024-03-29T09:11:28.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.831+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a80156ac0 con 0x7f7a801608e0 2024-03-29T09:11:28.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.832+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a800677a0 con 0x7f7a801608e0 2024-03-29T09:11:28.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.833+0000 7f7a77400700 1 -- 172.21.15.99:0/3241361114 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7a6c021030 con 0x7f7a801608e0 2024-03-29T09:11:28.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.833+0000 7f7a77400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a6006ac90 0x7f7a6006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:28.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.833+0000 7f7a77400700 1 -- 172.21.15.99:0/3241361114 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f7a6c08ab80 con 0x7f7a801608e0 2024-03-29T09:11:28.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.834+0000 7f7a7f400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a6006ac90 0x7f7a6006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:28.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.834+0000 7f7a7f400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a6006ac90 0x7f7a6006d150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7a70000f60 tx=0x7f7a70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:28.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:28.838+0000 7f7a77400700 1 -- 172.21.15.99:0/3241361114 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7a6c055b10 con 0x7f7a801608e0 2024-03-29T09:11:29.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.044+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a800c9f90 con 0x7f7a801608e0 2024-03-29T09:11:29.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.045+0000 7f7a77400700 1 -- 172.21.15.99:0/3241361114 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7f7a6c015020 con 0x7f7a801608e0 2024-03-29T09:11:29.045 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:29.046 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a6006ac90 msgr2=0x7f7a6006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a6006ac90 0x7f7a6006d150 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7a70000f60 tx=0x7f7a70009040 comp rx=0 tx=0).stop 2024-03-29T09:11:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a801608e0 msgr2=0x7f7a80155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a801608e0 0x7f7a80155560 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009c40 tx=0x7f7a6c004750 comp rx=0 tx=0).stop 2024-03-29T09:11:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 shutdown_connections 2024-03-29T09:11:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7a6006ac90 0x7f7a6006d150 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.048 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a80160cc0 0x7f7a800b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a800d1b20 0x7f7a801534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 --2- 172.21.15.99:0/3241361114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7a801608e0 0x7f7a80155560 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 >> 172.21.15.99:0/3241361114 conn(0x7f7a8009c490 msgr2=0x7f7a8006c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 shutdown_connections 2024-03-29T09:11:29.049 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.048+0000 7f7a86400700 1 -- 172.21.15.99:0/3241361114 wait complete. 2024-03-29T09:11:29.066 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T09:11:29.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:11:29.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.257+0000 7f8342600700 1 Processor -- start 2024-03-29T09:11:29.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.257+0000 7f8342600700 1 -- start start 2024-03-29T09:11:29.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.258+0000 7f8342600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f833c0d1b20 0x7f833c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.258+0000 7f8342600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0cddc0 con 0x7f833c1608e0 2024-03-29T09:11:29.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f8342600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0ce0a0 con 0x7f833c160cc0 2024-03-29T09:11:29.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f8342600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c0ce380 con 0x7f833c0d1b20 2024-03-29T09:11:29.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f833b400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f833c1608e0 0x7f833c0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45374/0 (socket says 172.21.15.99:45374) 2024-03-29T09:11:29.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f833b400700 1 -- 172.21.15.99:0/2191070833 learned_addr learned my addr 172.21.15.99:0/2191070833 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:29.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f833aa00700 1 --2- 172.21.15.99:0/2191070833 >> v2:172.21.15.99:3300/0 conn(0x7f833c0d1b20 0x7f833c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f833aa00700 1 -- 172.21.15.99:0/2191070833 >> v1:172.21.15.133:6789/0 conn(0x7f833c160cc0 legacy=0x7f833c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:29.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f833aa00700 1 -- 172.21.15.99:0/2191070833 >> v1:172.21.15.99:6789/0 conn(0x7f833c1608e0 legacy=0x7f833c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.260 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f833aa00700 1 -- 172.21.15.99:0/2191070833 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0ce660 con 0x7f833c0d1b20 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.259+0000 7f833a000700 1 -- 172.21.15.99:0/2191070833 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 115291436 0 0) 0x7f833c0cddc0 con 0x7f833c1608e0 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f833aa00700 1 --2- 172.21.15.99:0/2191070833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c0d1b20 0x7f833c0d0150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8328009840 tx=0x7f8328009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2012f7ee7a5b18fe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f833a000700 1 -- 172.21.15.99:0/2191070833 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8328004230 con 0x7f833c0d1b20 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f833a000700 1 -- 172.21.15.99:0/2191070833 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8328004390 con 0x7f833c0d1b20 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f833a000700 1 -- 172.21.15.99:0/2191070833 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280036a0 con 0x7f833c0d1b20 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f8342600700 1 -- 172.21.15.99:0/2191070833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c0d1b20 msgr2=0x7f833c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f8342600700 1 --2- 172.21.15.99:0/2191070833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c0d1b20 0x7f833c0d0150 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8328009840 tx=0x7f8328009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f8342600700 1 -- 172.21.15.99:0/2191070833 shutdown_connections 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f8342600700 1 --2- 172.21.15.99:0/2191070833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c0d1b20 0x7f833c0d0150 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f8342600700 1 -- 172.21.15.99:0/2191070833 >> 172.21.15.99:0/2191070833 conn(0x7f833c09c490 msgr2=0x7f833c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f8342600700 1 -- 172.21.15.99:0/2191070833 shutdown_connections 2024-03-29T09:11:29.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.260+0000 7f8342600700 1 -- 172.21.15.99:0/2191070833 wait complete. 2024-03-29T09:11:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.262+0000 7f8342600700 1 Processor -- start 2024-03-29T09:11:29.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.262+0000 7f8342600700 1 -- start start 2024-03-29T09:11:29.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f8342600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c0d1b20 0x7f833c1574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f8342600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f833c1608e0 0x7f833c153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f8342600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c160cc0 0x7f833c159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f8342600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0b3ff0 con 0x7f833c160cc0 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f8342600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c155780 con 0x7f833c0d1b20 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f8342600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0ca220 con 0x7f833c1608e0 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f833be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c160cc0 0x7f833c159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f833be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c160cc0 0x7f833c159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40606/0 (socket says 172.21.15.99:40606) 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f833be00700 1 -- 172.21.15.99:0/2978613527 learned_addr learned my addr 172.21.15.99:0/2978613527 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f833b400700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c0d1b20 0x7f833c1574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.263+0000 7f833aa00700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f833c1608e0 0x7f833c153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f833be00700 1 -- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f833c1608e0 msgr2=0x7f833c153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f833be00700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f833c1608e0 0x7f833c153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f833be00700 1 -- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c0d1b20 msgr2=0x7f833c1574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f833be00700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c0d1b20 0x7f833c1574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f833be00700 1 -- 172.21.15.99:0/2978613527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83280094a0 con 0x7f833c160cc0 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f833b400700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c0d1b20 0x7f833c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f833be00700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c160cc0 0x7f833c159d10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f833400b970 tx=0x7f833400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f8333400700 1 -- 172.21.15.99:0/2978613527 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400c930 con 0x7f833c160cc0 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f8333400700 1 -- 172.21.15.99:0/2978613527 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833400cad0 con 0x7f833c160cc0 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c1579f0 con 0x7f833c160cc0 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f8333400700 1 -- 172.21.15.99:0/2978613527 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400ce80 con 0x7f833c160cc0 2024-03-29T09:11:29.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.264+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c0b3d10 con 0x7f833c160cc0 2024-03-29T09:11:29.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.265+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c1558c0 con 0x7f833c160cc0 2024-03-29T09:11:29.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.266+0000 7f8333400700 1 -- 172.21.15.99:0/2978613527 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8334014440 con 0x7f833c160cc0 2024-03-29T09:11:29.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.267+0000 7f8333400700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f831c06ac90 0x7f831c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.267+0000 7f8333400700 1 -- 172.21.15.99:0/2978613527 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f83340891b0 con 0x7f833c160cc0 2024-03-29T09:11:29.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.267+0000 7f833b400700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f831c06ac90 0x7f831c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.268+0000 7f833b400700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f831c06ac90 0x7f831c06d150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f832c000f60 tx=0x7f832c007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:29.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.271+0000 7f8333400700 1 -- 172.21.15.99:0/2978613527 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f833c1558c0 con 0x7f833c160cc0 2024-03-29T09:11:29.517 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.518+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f833c1558c0 con 0x7f833c160cc0 2024-03-29T09:11:29.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.518+0000 7f8333400700 1 -- 172.21.15.99:0/2978613527 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f833c1558c0 con 0x7f833c160cc0 2024-03-29T09:11:29.518 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:29.518 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:10:43.923687+0000","modified":"2024-03-29T09:10:54.493701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/1841077010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1841077010},{"type":"v1","addr":"172.21.15.99:6835","nonce":1841077010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f831c06ac90 msgr2=0x7f831c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f831c06ac90 0x7f831c06d150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f832c000f60 tx=0x7f832c007ca0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c160cc0 msgr2=0x7f833c159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c160cc0 0x7f833c159d10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f833400b970 tx=0x7f833400bc80 comp rx=0 tx=0).stop 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 shutdown_connections 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f831c06ac90 0x7f831c06d150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f833c160cc0 0x7f833c159d10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833c0d1b20 0x7f833c1574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 --2- 172.21.15.99:0/2978613527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f833c1608e0 0x7f833c153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 >> 172.21.15.99:0/2978613527 conn(0x7f833c09c490 msgr2=0x7f833c0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 shutdown_connections 2024-03-29T09:11:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.521+0000 7f8342600700 1 -- 172.21.15.99:0/2978613527 wait complete. 2024-03-29T09:11:29.522 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2024-03-29T09:11:29.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:11:29.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.727+0000 7f1c57400700 1 Processor -- start 2024-03-29T09:11:29.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.727+0000 7f1c57400700 1 -- start start 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c57400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1c501608e0 0x7f1c500d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c57400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1c50160cc0 0x7f1c500ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c57400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c500d3a70 0x7f1c500d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c57400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c500cddc0 con 0x7f1c501608e0 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c57400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c500ce0a0 con 0x7f1c50160cc0 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c57400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500ce380 con 0x7f1c500d3a70 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c54a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1c501608e0 0x7f1c500d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c54a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1c501608e0 0x7f1c500d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40622/0 (socket says 172.21.15.99:40622) 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.728+0000 7f1c54a00700 1 -- 172.21.15.99:0/3155866981 learned_addr learned my addr 172.21.15.99:0/3155866981 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c54a00700 1 -- 172.21.15.99:0/3155866981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c500d3a70 msgr2=0x7f1c500d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:29.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c54a00700 1 --2- 172.21.15.99:0/3155866981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c500d3a70 0x7f1c500d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c54a00700 1 -- 172.21.15.99:0/3155866981 >> v2:172.21.15.133:3300/0 conn(0x7f1c50160cc0 msgr2=0x7f1c500ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c54a00700 1 --2- 172.21.15.99:0/3155866981 >> v2:172.21.15.133:3300/0 conn(0x7f1c50160cc0 0x7f1c500ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c54a00700 1 -- 172.21.15.99:0/3155866981 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500ce660 con 0x7f1c501608e0 2024-03-29T09:11:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c54a00700 1 --2- 172.21.15.99:0/3155866981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c500d0150 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1c5006ac50 tx=0x7f1c44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a204027ce4332e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c4f400700 1 -- 172.21.15.99:0/3155866981 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400b870 con 0x7f1c501608e0 2024-03-29T09:11:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c4f400700 1 -- 172.21.15.99:0/3155866981 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4400b9d0 con 0x7f1c501608e0 2024-03-29T09:11:29.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.729+0000 7f1c4f400700 1 -- 172.21.15.99:0/3155866981 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400bd60 con 0x7f1c501608e0 2024-03-29T09:11:29.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 -- 172.21.15.99:0/3155866981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 msgr2=0x7f1c500d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 --2- 172.21.15.99:0/3155866981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c500d0150 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1c5006ac50 tx=0x7f1c44009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 -- 172.21.15.99:0/3155866981 shutdown_connections 2024-03-29T09:11:29.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 --2- 172.21.15.99:0/3155866981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c500d3a70 0x7f1c500d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 --2- 172.21.15.99:0/3155866981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c500d0150 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 --2- 172.21.15.99:0/3155866981 >> v2:172.21.15.133:3300/0 conn(0x7f1c50160cc0 0x7f1c500ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 -- 172.21.15.99:0/3155866981 >> 172.21.15.99:0/3155866981 conn(0x7f1c5009c490 msgr2=0x7f1c500d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:29.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 -- 172.21.15.99:0/3155866981 shutdown_connections 2024-03-29T09:11:29.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.730+0000 7f1c57400700 1 -- 172.21.15.99:0/3155866981 wait complete. 2024-03-29T09:11:29.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.731+0000 7f1c57400700 1 Processor -- start 2024-03-29T09:11:29.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.732+0000 7f1c57400700 1 -- start start 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.732+0000 7f1c57400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c500d3a70 0x7f1c500ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c57400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c501f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c57400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c50160cc0 0x7f1c50155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c57400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500cd4a0 con 0x7f1c501608e0 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c57400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5015cb30 con 0x7f1c500d3a70 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c57400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500cce50 con 0x7f1c50160cc0 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c501f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c501f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40628/0 (socket says 172.21.15.99:40628) 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 -- 172.21.15.99:0/2237931921 learned_addr learned my addr 172.21.15.99:0/2237931921 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c54a00700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c500d3a70 0x7f1c500ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c55400700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c50160cc0 0x7f1c50155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 -- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c50160cc0 msgr2=0x7f1c50155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c50160cc0 0x7f1c50155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 -- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c500d3a70 msgr2=0x7f1c500ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c500d3a70 0x7f1c500ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.733+0000 7f1c4fe00700 1 -- 172.21.15.99:0/2237931921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c44009550 con 0x7f1c501608e0 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.734+0000 7f1c4fe00700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c501f6790 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b450 tx=0x7f1c3c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.734+0000 7f1c4cc00700 1 -- 172.21.15.99:0/2237931921 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c011830 con 0x7f1c501608e0 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.734+0000 7f1c4cc00700 1 -- 172.21.15.99:0/2237931921 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c011990 con 0x7f1c501608e0 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.734+0000 7f1c4cc00700 1 -- 172.21.15.99:0/2237931921 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c011d40 con 0x7f1c501608e0 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.734+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c500cd130 con 0x7f1c501608e0 2024-03-29T09:11:29.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.734+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5015ce10 con 0x7f1c501608e0 2024-03-29T09:11:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.735+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c500677a0 con 0x7f1c501608e0 2024-03-29T09:11:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.736+0000 7f1c4cc00700 1 -- 172.21.15.99:0/2237931921 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1c3c01a400 con 0x7f1c501608e0 2024-03-29T09:11:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.737+0000 7f1c4cc00700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c3006ac90 0x7f1c3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.737+0000 7f1c4cc00700 1 -- 172.21.15.99:0/2237931921 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f1c3c0895c0 con 0x7f1c501608e0 2024-03-29T09:11:29.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.737+0000 7f1c54a00700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c3006ac90 0x7f1c3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:29.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.737+0000 7f1c54a00700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c3006ac90 0x7f1c3006d150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1c44000c00 tx=0x7f1c44008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:29.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.741+0000 7f1c4cc00700 1 -- 172.21.15.99:0/2237931921 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1c3c054430 con 0x7f1c501608e0 2024-03-29T09:11:29.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.986+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c500cd2c0 con 0x7f1c501608e0 2024-03-29T09:11:29.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.987+0000 7f1c4cc00700 1 -- 172.21.15.99:0/2237931921 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4742 (secure 0 0 0) 0x7f1c3c009280 con 0x7f1c501608e0 2024-03-29T09:11:29.987 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:29.987 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:10:43.923687+0000","modified":"2024-03-29T09:10:54.493701+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5433":{"gid":5433,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6835/1841077010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1841077010},{"type":"v1","addr":"172.21.15.99:6835","nonce":1841077010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c3006ac90 msgr2=0x7f1c3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c3006ac90 0x7f1c3006d150 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1c44000c00 tx=0x7f1c44008030 comp rx=0 tx=0).stop 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 msgr2=0x7f1c501f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c501f6790 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b450 tx=0x7f1c3c00b760 comp rx=0 tx=0).stop 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 shutdown_connections 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1c3006ac90 0x7f1c3006d150 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c50160cc0 0x7f1c50155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c500d3a70 0x7f1c500ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 --2- 172.21.15.99:0/2237931921 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c501608e0 0x7f1c501f6790 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:29.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 >> 172.21.15.99:0/2237931921 conn(0x7f1c5009c490 msgr2=0x7f1c500b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:29.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 shutdown_connections 2024-03-29T09:11:29.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:29.990+0000 7f1c57400700 1 -- 172.21.15.99:0/2237931921 wait complete. 2024-03-29T09:11:29.991 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 17 2024-03-29T09:11:30.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:30.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.198+0000 7fcf42200700 1 Processor -- start 2024-03-29T09:11:30.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.199+0000 7fcf42200700 1 -- start start 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.200+0000 7fcf42200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c15da70 0x7fcf3c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.200+0000 7fcf42200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c0afde0 con 0x7fcf3c0d1740 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.200+0000 7fcf42200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c0d01d0 con 0x7fcf3c15d690 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.200+0000 7fcf42200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0d04b0 con 0x7fcf3c15da70 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3be00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcf3c0d1740 0x7fcf3c0b0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45386/0 (socket says 172.21.15.99:45386) 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3be00700 1 -- 172.21.15.99:0/1633386820 learned_addr learned my addr 172.21.15.99:0/1633386820 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3aa00700 1 --2- 172.21.15.99:0/1633386820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c15da70 0x7fcf3c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3a000700 1 -- 172.21.15.99:0/1633386820 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947272033 0 0) 0x7fcf3c0afde0 con 0x7fcf3c0d1740 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3a000700 1 -- 172.21.15.99:0/1633386820 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf200035b0 con 0x7fcf3c0d1740 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3a000700 1 -- 172.21.15.99:0/1633386820 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470981996 0 0) 0x7fcf3c0d01d0 con 0x7fcf3c15d690 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3a000700 1 -- 172.21.15.99:0/1633386820 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf3c0afde0 con 0x7fcf3c15d690 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3aa00700 1 -- 172.21.15.99:0/1633386820 >> v1:172.21.15.133:6789/0 conn(0x7fcf3c15d690 legacy=0x7fcf3c0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3aa00700 1 -- 172.21.15.99:0/1633386820 >> v1:172.21.15.99:6789/0 conn(0x7fcf3c0d1740 legacy=0x7fcf3c0b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.201+0000 7fcf3aa00700 1 -- 172.21.15.99:0/1633386820 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0b0030 con 0x7fcf3c15da70 2024-03-29T09:11:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf3aa00700 1 --2- 172.21.15.99:0/1633386820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c15da70 0x7fcf3c0cfb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009870 tx=0x7fcf28009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb20d44e84597d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf3a000700 1 -- 172.21.15.99:0/1633386820 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28011830 con 0x7fcf3c15da70 2024-03-29T09:11:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf3a000700 1 -- 172.21.15.99:0/1633386820 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf28011990 con 0x7fcf3c15da70 2024-03-29T09:11:30.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf3a000700 1 -- 172.21.15.99:0/1633386820 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28011d20 con 0x7fcf3c15da70 2024-03-29T09:11:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf42200700 1 -- 172.21.15.99:0/1633386820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c15da70 msgr2=0x7fcf3c0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf42200700 1 --2- 172.21.15.99:0/1633386820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c15da70 0x7fcf3c0cfb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009870 tx=0x7fcf28009b80 comp rx=0 tx=0).stop 2024-03-29T09:11:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf42200700 1 -- 172.21.15.99:0/1633386820 shutdown_connections 2024-03-29T09:11:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf42200700 1 --2- 172.21.15.99:0/1633386820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c15da70 0x7fcf3c0cfb60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf42200700 1 -- 172.21.15.99:0/1633386820 >> 172.21.15.99:0/1633386820 conn(0x7fcf3c09c480 msgr2=0x7fcf3c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:30.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.202+0000 7fcf42200700 1 -- 172.21.15.99:0/1633386820 shutdown_connections 2024-03-29T09:11:30.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.203+0000 7fcf42200700 1 -- 172.21.15.99:0/1633386820 wait complete. 2024-03-29T09:11:30.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.204+0000 7fcf42200700 1 Processor -- start 2024-03-29T09:11:30.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.204+0000 7fcf42200700 1 -- start start 2024-03-29T09:11:30.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf42200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c0d1740 0x7fcf3c160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf42200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf3c15d690 0x7fcf3c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf42200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf3c15da70 0x7fcf3c157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf42200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0b71a0 con 0x7fcf3c15d690 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf42200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0b6520 con 0x7fcf3c15da70 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf42200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0b6800 con 0x7fcf3c0d1740 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf3aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf3c15d690 0x7fcf3c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf3aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf3c15d690 0x7fcf3c0b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40642/0 (socket says 172.21.15.99:40642) 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.205+0000 7fcf3aa00700 1 -- 172.21.15.99:0/1802098530 learned_addr learned my addr 172.21.15.99:0/1802098530 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3be00700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf3c15da70 0x7fcf3c157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3b400700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c0d1740 0x7fcf3c160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3aa00700 1 -- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c0d1740 msgr2=0x7fcf3c160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3aa00700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c0d1740 0x7fcf3c160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3aa00700 1 -- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf3c15da70 msgr2=0x7fcf3c157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3aa00700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf3c15da70 0x7fcf3c157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3aa00700 1 -- 172.21.15.99:0/1802098530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280094a0 con 0x7fcf3c15d690 2024-03-29T09:11:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf3aa00700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf3c15d690 0x7fcf3c0b5cd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009bc0 tx=0x7fcf28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf33400700 1 -- 172.21.15.99:0/1802098530 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2801a860 con 0x7fcf3c15d690 2024-03-29T09:11:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf33400700 1 -- 172.21.15.99:0/1802098530 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2801a9c0 con 0x7fcf3c15d690 2024-03-29T09:11:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf33400700 1 -- 172.21.15.99:0/1802098530 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2801ad50 con 0x7fcf3c15d690 2024-03-29T09:11:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.206+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c0b6ae0 con 0x7fcf3c15d690 2024-03-29T09:11:30.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.207+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c0b6210 con 0x7fcf3c15d690 2024-03-29T09:11:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.208+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c0b72e0 con 0x7fcf3c15d690 2024-03-29T09:11:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.209+0000 7fcf33400700 1 -- 172.21.15.99:0/1802098530 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcf28022070 con 0x7fcf3c15d690 2024-03-29T09:11:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.209+0000 7fcf33400700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcf1c06ac90 0x7fcf1c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.209+0000 7fcf33400700 1 -- 172.21.15.99:0/1802098530 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7fcf2808a2e0 con 0x7fcf3c15d690 2024-03-29T09:11:30.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.210+0000 7fcf3b400700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcf1c06ac90 0x7fcf1c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.210+0000 7fcf3b400700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcf1c06ac90 0x7fcf1c06d150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c007fd0 tx=0x7fcf2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:30.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.214+0000 7fcf33400700 1 -- 172.21.15.99:0/1802098530 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcf3c0b72e0 con 0x7fcf3c15d690 2024-03-29T09:11:30.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.418+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf3c0b72e0 con 0x7fcf3c15d690 2024-03-29T09:11:30.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.420+0000 7fcf33400700 1 -- 172.21.15.99:0/1802098530 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18081 (secure 0 0 0) 0x7fcf3c0b72e0 con 0x7fcf3c15d690 2024-03-29T09:11:30.420 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:30.420 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":42,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:10:55.495958+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:10:42.037884+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-29T09:10:42.900862+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2024-03-29T09:10:53.257132+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:30.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcf1c06ac90 msgr2=0x7fcf1c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcf1c06ac90 0x7fcf1c06d150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c007fd0 tx=0x7fcf2c009040 comp rx=0 tx=0).stop 2024-03-29T09:11:30.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf3c15d690 msgr2=0x7fcf3c0b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.422 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf3c15d690 0x7fcf3c0b5cd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009bc0 tx=0x7fcf28004750 comp rx=0 tx=0).stop 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 shutdown_connections 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcf1c06ac90 0x7fcf1c06d150 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf3c15da70 0x7fcf3c157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf3c0d1740 0x7fcf3c160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 --2- 172.21.15.99:0/1802098530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf3c15d690 0x7fcf3c0b5cd0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.422+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 >> 172.21.15.99:0/1802098530 conn(0x7fcf3c09c480 msgr2=0x7fcf3c0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.423+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 shutdown_connections 2024-03-29T09:11:30.423 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.423+0000 7fcf42200700 1 -- 172.21.15.99:0/1802098530 wait complete. 2024-03-29T09:11:30.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T09:11:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.633+0000 7f6b64000700 1 Processor -- start 2024-03-29T09:11:30.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.633+0000 7f6b64000700 1 -- start start 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.634+0000 7f6b64000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6b5c1608e0 0x7f6b5c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b64000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c0cddc0 con 0x7f6b5c0d1b20 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b64000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5c0ce0a0 con 0x7f6b5c160cc0 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b64000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b5c0ce380 con 0x7f6b5c1608e0 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b61600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6b5c1608e0 0x7f6b5c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b61600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f6b5c1608e0 0x7f6b5c0d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40650/0 (socket says 172.21.15.99:40650) 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b61600700 1 -- 172.21.15.99:0/480003669 learned_addr learned my addr 172.21.15.99:0/480003669 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b62000700 1 --1- 172.21.15.99:0/480003669 >> v1:172.21.15.99:6789/0 conn(0x7f6b5c0d1b20 0x7f6b5c0afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45390/0 (socket says 172.21.15.99:45390) 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b61600700 1 -- 172.21.15.99:0/480003669 >> v1:172.21.15.133:6789/0 conn(0x7f6b5c160cc0 legacy=0x7f6b5c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.635 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b61600700 1 -- 172.21.15.99:0/480003669 >> v1:172.21.15.99:6789/0 conn(0x7f6b5c0d1b20 legacy=0x7f6b5c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.636 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.635+0000 7f6b61600700 1 -- 172.21.15.99:0/480003669 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0ce660 con 0x7f6b5c1608e0 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b61600700 1 --2- 172.21.15.99:0/480003669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0d0150 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6b500098f0 tx=0x7f6b50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c8f7fa5afff387 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b5be00700 1 -- 172.21.15.99:0/480003669 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000b8a0 con 0x7f6b5c1608e0 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b5be00700 1 -- 172.21.15.99:0/480003669 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5000ba00 con 0x7f6b5c1608e0 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b5be00700 1 -- 172.21.15.99:0/480003669 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000bd90 con 0x7f6b5c1608e0 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b64000700 1 -- 172.21.15.99:0/480003669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c1608e0 msgr2=0x7f6b5c0d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b64000700 1 --2- 172.21.15.99:0/480003669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0d0150 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6b500098f0 tx=0x7f6b50009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b64000700 1 -- 172.21.15.99:0/480003669 shutdown_connections 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b64000700 1 --2- 172.21.15.99:0/480003669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0d0150 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b64000700 1 -- 172.21.15.99:0/480003669 >> 172.21.15.99:0/480003669 conn(0x7f6b5c09c490 msgr2=0x7f6b5c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.636+0000 7f6b64000700 1 -- 172.21.15.99:0/480003669 shutdown_connections 2024-03-29T09:11:30.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.637+0000 7f6b64000700 1 -- 172.21.15.99:0/480003669 wait complete. 2024-03-29T09:11:30.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.638+0000 7f6b64000700 1 Processor -- start 2024-03-29T09:11:30.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.638+0000 7f6b64000700 1 -- start start 2024-03-29T09:11:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.639+0000 7f6b64000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b5c0d1b20 0x7f6b5c0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.639+0000 7f6b64000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.639+0000 7f6b64000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c160cc0 0x7f6b5c0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.639+0000 7f6b64000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0b5620 con 0x7f6b5c160cc0 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.639+0000 7f6b64000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0b5900 con 0x7f6b5c1608e0 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.639+0000 7f6b64000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5c0b5be0 con 0x7f6b5c0d1b20 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c160cc0 0x7f6b5c0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c160cc0 0x7f6b5c0b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40664/0 (socket says 172.21.15.99:40664) 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 -- 172.21.15.99:0/2606911496 learned_addr learned my addr 172.21.15.99:0/2606911496 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b61600700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b5c0d1b20 0x7f6b5c0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b60c00700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 -- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b5c0d1b20 msgr2=0x7f6b5c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b5c0d1b20 0x7f6b5c0b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 -- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5c1608e0 msgr2=0x7f6b5c0b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 -- 172.21.15.99:0/2606911496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50009550 con 0x7f6b5c160cc0 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b60c00700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:30.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b62000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c160cc0 0x7f6b5c0b66c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400cbd0 tx=0x7f6b5400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b59600700 1 -- 172.21.15.99:0/2606911496 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400ab00 con 0x7f6b5c160cc0 2024-03-29T09:11:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b59600700 1 -- 172.21.15.99:0/2606911496 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5400ac60 con 0x7f6b5c160cc0 2024-03-29T09:11:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5c0b5ec0 con 0x7f6b5c160cc0 2024-03-29T09:11:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.640+0000 7f6b59600700 1 -- 172.21.15.99:0/2606911496 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54007070 con 0x7f6b5c160cc0 2024-03-29T09:11:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.641+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5c0b7c60 con 0x7f6b5c160cc0 2024-03-29T09:11:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.642+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5c0677a0 con 0x7f6b5c160cc0 2024-03-29T09:11:30.642 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.642+0000 7f6b59600700 1 -- 172.21.15.99:0/2606911496 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6b540071d0 con 0x7f6b5c160cc0 2024-03-29T09:11:30.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.643+0000 7f6b59600700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6b3c06ac90 0x7f6b3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:30.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.643+0000 7f6b59600700 1 -- 172.21.15.99:0/2606911496 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6424+0+0 (secure 0 0 0) 0x7f6b54089310 con 0x7f6b5c160cc0 2024-03-29T09:11:30.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.643+0000 7f6b61600700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6b3c06ac90 0x7f6b3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:30.643 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.644+0000 7f6b61600700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6b3c06ac90 0x7f6b3c06d150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009c40 tx=0x7f6b50008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:30.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.647+0000 7f6b59600700 1 -- 172.21.15.99:0/2606911496 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6b54054230 con 0x7f6b5c160cc0 2024-03-29T09:11:30.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.896+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b5c0d0700 con 0x7f6b5c160cc0 2024-03-29T09:11:30.897 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-03-29T09:11:30.897+0000 7fe659e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T09:11:30.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.908+0000 7f6b59600700 1 -- 172.21.15.99:0/2606911496 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6b54057850 con 0x7f6b5c160cc0 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6b3c06ac90 msgr2=0x7f6b3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6b3c06ac90 0x7f6b3c06d150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009c40 tx=0x7f6b50008030 comp rx=0 tx=0).stop 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c160cc0 msgr2=0x7f6b5c0b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c160cc0 0x7f6b5c0b66c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400cbd0 tx=0x7f6b5400cee0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 shutdown_connections 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6b3c06ac90 0x7f6b3c06d150 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6b5c160cc0 0x7f6b5c0b66c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6b5c0d1b20 0x7f6b5c0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 --2- 172.21.15.99:0/2606911496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6b5c1608e0 0x7f6b5c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.910+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 >> 172.21.15.99:0/2606911496 conn(0x7f6b5c09c490 msgr2=0x7f6b5c15fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.911+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 shutdown_connections 2024-03-29T09:11:30.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:30.911+0000 7f6b64000700 1 -- 172.21.15.99:0/2606911496 wait complete. 2024-03-29T09:11:30.912 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T09:11:30.915 INFO:tasks.ceph.mds.a.smithi099.stderr: -3865> 2024-03-29T09:10:46.388+0000 7f1109600700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T09:11:30.930 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T09:11:30.966 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-03-29T09:11:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.126+0000 7f74d7000700 1 Processor -- start 2024-03-29T09:11:31.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.126+0000 7f74d7000700 1 -- start start 2024-03-29T09:11:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74d7000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f74d00b3cc0 0x7f74d00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74d7000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d00d3380 0x7f74d00ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74d7000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d00d3810 con 0x7f74d0160770 2024-03-29T09:11:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74d7000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d00b4140 con 0x7f74d00b3cc0 2024-03-29T09:11:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74d7000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d00d1520 con 0x7f74d00d3380 2024-03-29T09:11:31.128 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74cf400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f74d00b3cc0 0x7f74d00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74cf400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f74d00b3cc0 0x7f74d00d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40690/0 (socket says 172.21.15.99:40690) 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.127+0000 7f74cf400700 1 -- 172.21.15.99:0/3296121022 learned_addr learned my addr 172.21.15.99:0/3296121022 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74d5000700 1 --2- 172.21.15.99:0/3296121022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d00d3380 0x7f74d00ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cf400700 1 -- 172.21.15.99:0/3296121022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d00d3380 msgr2=0x7f74d00ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cf400700 1 --2- 172.21.15.99:0/3296121022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d00d3380 0x7f74d00ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cf400700 1 -- 172.21.15.99:0/3296121022 >> v1:172.21.15.99:6789/0 conn(0x7f74d0160770 legacy=0x7f74d00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cf400700 1 -- 172.21.15.99:0/3296121022 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d00d1800 con 0x7f74d00b3cc0 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cf400700 1 --2- 172.21.15.99:0/3296121022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00d2e40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0098f0 tx=0x7f74bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471f9d01bc7ec56 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cea00700 1 -- 172.21.15.99:0/3296121022 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00b8a0 con 0x7f74d00b3cc0 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cea00700 1 -- 172.21.15.99:0/3296121022 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc00ba00 con 0x7f74d00b3cc0 2024-03-29T09:11:31.129 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.128+0000 7f74cea00700 1 -- 172.21.15.99:0/3296121022 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00bd90 con 0x7f74d00b3cc0 2024-03-29T09:11:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 -- 172.21.15.99:0/3296121022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 msgr2=0x7f74d00d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:31.130 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 --2- 172.21.15.99:0/3296121022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00d2e40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f74bc0098f0 tx=0x7f74bc009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 -- 172.21.15.99:0/3296121022 shutdown_connections 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 --2- 172.21.15.99:0/3296121022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d00d3380 0x7f74d00ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 --2- 172.21.15.99:0/3296121022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00d2e40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 -- 172.21.15.99:0/3296121022 >> 172.21.15.99:0/3296121022 conn(0x7f74d008afd0 msgr2=0x7f74d009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 -- 172.21.15.99:0/3296121022 shutdown_connections 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.129+0000 7f74d7000700 1 -- 172.21.15.99:0/3296121022 wait complete. 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.131+0000 7f74d7000700 1 Processor -- start 2024-03-29T09:11:31.131 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.131+0000 7f74d7000700 1 -- start start 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74d7000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74d7000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74d00d3380 0x7f74d00b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74d7000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d0160770 0x7f74d015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74d7000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d00b8300 con 0x7f74d00b3cc0 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74d7000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d015b950 con 0x7f74d00d3380 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74d7000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d00cb630 con 0x7f74d0160770 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40696/0 (socket says 172.21.15.99:40696) 2024-03-29T09:11:31.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74cfe00700 1 -- 172.21.15.99:0/1192532546 learned_addr learned my addr 172.21.15.99:0/1192532546 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74cf400700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74d00d3380 0x7f74d00b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74d5000700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d0160770 0x7f74d015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74cfe00700 1 -- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d0160770 msgr2=0x7f74d015b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.132+0000 7f74cfe00700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d0160770 0x7f74d015b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74cfe00700 1 -- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74d00d3380 msgr2=0x7f74d00b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74cfe00700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74d00d3380 0x7f74d00b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74cfe00700 1 -- 172.21.15.99:0/1192532546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc009550 con 0x7f74d00b3cc0 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74cf400700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74d00d3380 0x7f74d00b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74cfe00700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00cbac0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f74c000c480 tx=0x7f74c000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74c7e00700 1 -- 172.21.15.99:0/1192532546 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00052d0 con 0x7f74d00b3cc0 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74c7e00700 1 -- 172.21.15.99:0/1192532546 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0005430 con 0x7f74d00b3cc0 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d00caa20 con 0x7f74d00b3cc0 2024-03-29T09:11:31.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74c7e00700 1 -- 172.21.15.99:0/1192532546 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00038c0 con 0x7f74d00b3cc0 2024-03-29T09:11:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.133+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d00cafe0 con 0x7f74d00b3cc0 2024-03-29T09:11:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.134+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d00cb910 con 0x7f74d00b3cc0 2024-03-29T09:11:31.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.135+0000 7f74c7e00700 1 -- 172.21.15.99:0/1192532546 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f74c0003a20 con 0x7f74d00b3cc0 2024-03-29T09:11:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.136+0000 7f74c7e00700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74b006ac90 0x7f74b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.136+0000 7f74c7e00700 1 -- 172.21.15.99:0/1192532546 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6510+0+0 (secure 0 0 0) 0x7f74c0088a90 con 0x7f74d00b3cc0 2024-03-29T09:11:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.136+0000 7f74cf400700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74b006ac90 0x7f74b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:31.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.137+0000 7f74cf400700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74b006ac90 0x7f74b006d150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009c40 tx=0x7f74bc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:31.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.140+0000 7f74c7e00700 1 -- 172.21.15.99:0/1192532546 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f74c0053950 con 0x7f74d00b3cc0 2024-03-29T09:11:31.389 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.389+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f74d00678b0 con 0x7f74d00b3cc0 2024-03-29T09:11:31.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.910+0000 7f74c7e00700 1 -- 172.21.15.99:0/1192532546 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f74c0056f70 con 0x7f74d00b3cc0 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74b006ac90 msgr2=0x7f74b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74b006ac90 0x7f74b006d150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f74bc009c40 tx=0x7f74bc008030 comp rx=0 tx=0).stop 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 msgr2=0x7f74d00cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00cbac0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f74c000c480 tx=0x7f74c000c840 comp rx=0 tx=0).stop 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 shutdown_connections 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f74b006ac90 0x7f74b006d150 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74d0160770 0x7f74d015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f74d00b3cc0 0x7f74d00cbac0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.913 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 --2- 172.21.15.99:0/1192532546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74d00d3380 0x7f74d00b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:31.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 >> 172.21.15.99:0/1192532546 conn(0x7f74d008afd0 msgr2=0x7f74d00b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:31.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 shutdown_connections 2024-03-29T09:11:31.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:31.913+0000 7f74d7000700 1 -- 172.21.15.99:0/1192532546 wait complete. 2024-03-29T09:11:31.931 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T09:11:32.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.166+0000 7f5602400700 1 Processor -- start 2024-03-29T09:11:32.166 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.166+0000 7f5602400700 1 -- start start 2024-03-29T09:11:32.169 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f5602400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f55fc067110 0x7f55fc15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f5602400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f55fc13e800 0x7f55fc13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f5602400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc0cd6f0 con 0x7f55fc15af80 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f5602400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc140560 con 0x7f55fc067110 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f5602400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc13f9e0 con 0x7f55fc13e800 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55faa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f55fc067110 0x7f55fc15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55faa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f55fc067110 0x7f55fc15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40704/0 (socket says 172.21.15.99:40704) 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55faa00700 1 -- 172.21.15.99:0/1267865312 learned_addr learned my addr 172.21.15.99:0/1267865312 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55faa00700 1 -- 172.21.15.99:0/1267865312 >> v2:172.21.15.133:3300/0 conn(0x7f55fc13e800 msgr2=0x7f55fc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55fbe00700 1 --2- 172.21.15.99:0/1267865312 >> v2:172.21.15.133:3300/0 conn(0x7f55fc13e800 0x7f55fc13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55faa00700 1 --2- 172.21.15.99:0/1267865312 >> v2:172.21.15.133:3300/0 conn(0x7f55fc13e800 0x7f55fc13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55faa00700 1 -- 172.21.15.99:0/1267865312 >> v1:172.21.15.133:6789/0 conn(0x7f55fc15af80 legacy=0x7f55fc066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55faa00700 1 -- 172.21.15.99:0/1267865312 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc13fcc0 con 0x7f55fc067110 2024-03-29T09:11:32.170 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.168+0000 7f55fbe00700 1 --2- 172.21.15.99:0/1267865312 >> v2:172.21.15.133:3300/0 conn(0x7f55fc13e800 0x7f55fc13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.169+0000 7f55faa00700 1 --2- 172.21.15.99:0/1267865312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc067110 0x7f55fc15e4e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55fc06e750 tx=0x7f55e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5dd33cd43b1eb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.169+0000 7f55fa000700 1 -- 172.21.15.99:0/1267865312 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8004230 con 0x7f55fc067110 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.169+0000 7f55fa000700 1 -- 172.21.15.99:0/1267865312 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8004390 con 0x7f55fc067110 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.169+0000 7f55fa000700 1 -- 172.21.15.99:0/1267865312 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e80036a0 con 0x7f55fc067110 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.169+0000 7f5602400700 1 -- 172.21.15.99:0/1267865312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc067110 msgr2=0x7f55fc15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.169+0000 7f5602400700 1 --2- 172.21.15.99:0/1267865312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc067110 0x7f55fc15e4e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55fc06e750 tx=0x7f55e8009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.170+0000 7f5602400700 1 -- 172.21.15.99:0/1267865312 shutdown_connections 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.170+0000 7f5602400700 1 --2- 172.21.15.99:0/1267865312 >> v2:172.21.15.133:3300/0 conn(0x7f55fc13e800 0x7f55fc13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.170+0000 7f5602400700 1 --2- 172.21.15.99:0/1267865312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc067110 0x7f55fc15e4e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.170+0000 7f5602400700 1 -- 172.21.15.99:0/1267865312 >> 172.21.15.99:0/1267865312 conn(0x7f55fc06dac0 msgr2=0x7f55fc06ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.170+0000 7f5602400700 1 -- 172.21.15.99:0/1267865312 shutdown_connections 2024-03-29T09:11:32.171 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.170+0000 7f5602400700 1 -- 172.21.15.99:0/1267865312 wait complete. 2024-03-29T09:11:32.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.171+0000 7f5602400700 1 Processor -- start 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.172+0000 7f5602400700 1 -- start start 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.172+0000 7f5602400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55fc067110 0x7f55fc0cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.172+0000 7f5602400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc13e800 0x7f55fc13b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.172+0000 7f5602400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55fc15af80 0x7f55fc13bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.172+0000 7f5602400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc159e40 con 0x7f55fc13e800 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.172+0000 7f5602400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0b29e0 con 0x7f55fc15af80 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.172+0000 7f5602400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0b1c20 con 0x7f55fc067110 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc13e800 0x7f55fc13b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc13e800 0x7f55fc13b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40718/0 (socket says 172.21.15.99:40718) 2024-03-29T09:11:32.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 -- 172.21.15.99:0/360609975 learned_addr learned my addr 172.21.15.99:0/360609975 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55fbe00700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55fc15af80 0x7f55fc13bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 -- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55fc067110 msgr2=0x7f55fc0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55fb400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55fc067110 0x7f55fc0cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55fc067110 0x7f55fc0cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 -- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55fc15af80 msgr2=0x7f55fc13bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55fc15af80 0x7f55fc13bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 -- 172.21.15.99:0/360609975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e8009550 con 0x7f55fc13e800 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55fbe00700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55fc15af80 0x7f55fc13bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55fb400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55fc067110 0x7f55fc0cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.173+0000 7f55faa00700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc13e800 0x7f55fc13b5e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55fc06eac0 tx=0x7f55e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.174+0000 7f55f3400700 1 -- 172.21.15.99:0/360609975 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011630 con 0x7f55fc13e800 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.174+0000 7f55f3400700 1 -- 172.21.15.99:0/360609975 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8011790 con 0x7f55fc13e800 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.174+0000 7f55f3400700 1 -- 172.21.15.99:0/360609975 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011b40 con 0x7f55fc13e800 2024-03-29T09:11:32.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.174+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc0b1f00 con 0x7f55fc13e800 2024-03-29T09:11:32.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.174+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc0b24c0 con 0x7f55fc13e800 2024-03-29T09:11:32.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.175+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc0678b0 con 0x7f55fc13e800 2024-03-29T09:11:32.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.176+0000 7f55f3400700 1 -- 172.21.15.99:0/360609975 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f55e8011ca0 con 0x7f55fc13e800 2024-03-29T09:11:32.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.177+0000 7f55f3400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f55dc06ac90 0x7f55dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:32.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.177+0000 7f55f3400700 1 -- 172.21.15.99:0/360609975 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6510+0+0 (secure 0 0 0) 0x7f55e808ada0 con 0x7f55fc13e800 2024-03-29T09:11:32.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.177+0000 7f55fb400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f55dc06ac90 0x7f55dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:32.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.178+0000 7f55fb400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f55dc06ac90 0x7f55dc06d150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f55ec003f40 tx=0x7f55ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:32.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.181+0000 7f55f3400700 1 -- 172.21.15.99:0/360609975 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f55e8055c50 con 0x7f55fc13e800 2024-03-29T09:11:32.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:32.382+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f55fc0678b0 con 0x7f55fc13e800 2024-03-29T09:11:33.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.101+0000 7f55f3400700 1 -- 172.21.15.99:0/360609975 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f55e8059270 con 0x7f55fc13e800 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f55dc06ac90 msgr2=0x7f55dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f55dc06ac90 0x7f55dc06d150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f55ec003f40 tx=0x7f55ec009040 comp rx=0 tx=0).stop 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc13e800 msgr2=0x7f55fc13b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc13e800 0x7f55fc13b5e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55fc06eac0 tx=0x7f55e8004750 comp rx=0 tx=0).stop 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 shutdown_connections 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f55dc06ac90 0x7f55dc06d150 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55fc15af80 0x7f55fc13bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55fc067110 0x7f55fc0cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 --2- 172.21.15.99:0/360609975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55fc13e800 0x7f55fc13b5e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.104+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 >> 172.21.15.99:0/360609975 conn(0x7f55fc06dac0 msgr2=0x7f55fc06c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:33.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.105+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 shutdown_connections 2024-03-29T09:11:33.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.105+0000 7f5602400700 1 -- 172.21.15.99:0/360609975 wait complete. 2024-03-29T09:11:33.106 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-03-29T09:11:33.125 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T09:11:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.321+0000 7f1a6dc00700 1 Processor -- start 2024-03-29T09:11:33.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.321+0000 7f1a6dc00700 1 -- start start 2024-03-29T09:11:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.322+0000 7f1a6dc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a68160b30 0x7f1a680cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.322+0000 7f1a6dc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a680d1fa0 con 0x7f1a680d0090 2024-03-29T09:11:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.322+0000 7f1a6dc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a68160f10 con 0x7f1a680b1f50 2024-03-29T09:11:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.322+0000 7f1a6dc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680cf440 con 0x7f1a68160b30 2024-03-29T09:11:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.322+0000 7f1a66a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1a68160b30 0x7f1a680cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.322+0000 7f1a67400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1a680d0090 0x7f1a680b32a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45404/0 (socket says 172.21.15.99:45404) 2024-03-29T09:11:33.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.322+0000 7f1a67400700 1 -- 172.21.15.99:0/1585481692 learned_addr learned my addr 172.21.15.99:0/1585481692 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.323+0000 7f1a66a00700 1 -- 172.21.15.99:0/1585481692 >> v1:172.21.15.133:6789/0 conn(0x7f1a680b1f50 legacy=0x7f1a680b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.323+0000 7f1a66a00700 1 -- 172.21.15.99:0/1585481692 >> v1:172.21.15.99:6789/0 conn(0x7f1a680d0090 legacy=0x7f1a680b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.323+0000 7f1a66a00700 1 -- 172.21.15.99:0/1585481692 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680d1670 con 0x7f1a68160b30 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.323+0000 7f1a66a00700 1 --2- 172.21.15.99:0/1585481692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680cfb50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009840 tx=0x7f1a58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c88c0817a14aba server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.323+0000 7f1a65600700 1 -- 172.21.15.99:0/1585481692 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58011830 con 0x7f1a68160b30 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.323+0000 7f1a65600700 1 -- 172.21.15.99:0/1585481692 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58011990 con 0x7f1a68160b30 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.323+0000 7f1a65600700 1 -- 172.21.15.99:0/1585481692 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58011d20 con 0x7f1a68160b30 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.324+0000 7f1a6dc00700 1 -- 172.21.15.99:0/1585481692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 msgr2=0x7f1a680cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.324+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/1585481692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680cfb50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009840 tx=0x7f1a58009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.324+0000 7f1a6dc00700 1 -- 172.21.15.99:0/1585481692 shutdown_connections 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.324+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/1585481692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680cfb50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.324+0000 7f1a6dc00700 1 -- 172.21.15.99:0/1585481692 >> 172.21.15.99:0/1585481692 conn(0x7f1a6809c4d0 msgr2=0x7f1a680d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:33.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.324+0000 7f1a6dc00700 1 -- 172.21.15.99:0/1585481692 shutdown_connections 2024-03-29T09:11:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.324+0000 7f1a6dc00700 1 -- 172.21.15.99:0/1585481692 wait complete. 2024-03-29T09:11:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.326+0000 7f1a6dc00700 1 Processor -- start 2024-03-29T09:11:33.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.326+0000 7f1a6dc00700 1 -- start start 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a6dc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a680b1f50 0x7f1a680cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a6dc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a680d0090 0x7f1a6815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a6dc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a6dc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6809cf40 con 0x7f1a68160b30 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a6dc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6815ae80 con 0x7f1a680d0090 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a6dc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680a0dd0 con 0x7f1a680b1f50 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680a04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40726/0 (socket says 172.21.15.99:40726) 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 -- 172.21.15.99:0/395898287 learned_addr learned my addr 172.21.15.99:0/395898287 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a66a00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a680b1f50 0x7f1a680cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a66000700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a680d0090 0x7f1a6815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 -- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a680b1f50 msgr2=0x7f1a680cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:33.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a680b1f50 0x7f1a680cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 -- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a680d0090 msgr2=0x7f1a6815a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a680d0090 0x7f1a6815a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.327+0000 7f1a67400700 1 -- 172.21.15.99:0/395898287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580094a0 con 0x7f1a68160b30 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.328+0000 7f1a67400700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680a04c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000cbc0 tx=0x7f1a6000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.328+0000 7f1a5ea00700 1 -- 172.21.15.99:0/395898287 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000a9e0 con 0x7f1a68160b30 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.328+0000 7f1a5ea00700 1 -- 172.21.15.99:0/395898287 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6000ab40 con 0x7f1a68160b30 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.328+0000 7f1a5ea00700 1 -- 172.21.15.99:0/395898287 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60012400 con 0x7f1a68160b30 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.328+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6809d120 con 0x7f1a68160b30 2024-03-29T09:11:33.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.328+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6809d6e0 con 0x7f1a68160b30 2024-03-29T09:11:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.329+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6809d900 con 0x7f1a68160b30 2024-03-29T09:11:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.330+0000 7f1a5ea00700 1 -- 172.21.15.99:0/395898287 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1a6001b3b0 con 0x7f1a68160b30 2024-03-29T09:11:33.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.331+0000 7f1a5ea00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a4806acc0 0x7f1a4806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.331+0000 7f1a5ea00700 1 -- 172.21.15.99:0/395898287 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6072+0+0 (secure 0 0 0) 0x7f1a60088fd0 con 0x7f1a68160b30 2024-03-29T09:11:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.331+0000 7f1a66a00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a4806acc0 0x7f1a4806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:33.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.332+0000 7f1a66a00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a4806acc0 0x7f1a4806d180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a580090b0 tx=0x7f1a58008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:33.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.335+0000 7f1a5ea00700 1 -- 172.21.15.99:0/395898287 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1a60054050 con 0x7f1a68160b30 2024-03-29T09:11:33.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:33.540+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1a680678b0 con 0x7f1a68160b30 2024-03-29T09:11:34.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.106+0000 7f1a5ea00700 1 -- 172.21.15.99:0/395898287 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1a600170d0 con 0x7f1a68160b30 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a4806acc0 msgr2=0x7f1a4806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a4806acc0 0x7f1a4806d180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a580090b0 tx=0x7f1a58008030 comp rx=0 tx=0).stop 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 msgr2=0x7f1a680a04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680a04c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1a6000cbc0 tx=0x7f1a6000ced0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 shutdown_connections 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a4806acc0 0x7f1a4806d180 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a68160b30 0x7f1a680a04c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a680b1f50 0x7f1a680cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 --2- 172.21.15.99:0/395898287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a680d0090 0x7f1a6815a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 >> 172.21.15.99:0/395898287 conn(0x7f1a6809c4d0 msgr2=0x7f1a6806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:34.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 shutdown_connections 2024-03-29T09:11:34.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.110+0000 7f1a6dc00700 1 -- 172.21.15.99:0/395898287 wait complete. 2024-03-29T09:11:34.111 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-03-29T09:11:34.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-03-29T09:11:34.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.326+0000 7fbe8c800700 1 Processor -- start 2024-03-29T09:11:34.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.326+0000 7fbe8c800700 1 -- start start 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe8c800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe84160b30 0x7fbe840cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe8c800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbe840d0090 0x7fbe840ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe8c800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe840d1fa0 con 0x7fbe840b1f50 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe8c800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840cf440 con 0x7fbe84160b30 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe8c800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840d0500 con 0x7fbe840d0090 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe89e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe84160b30 0x7fbe840cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe89e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fbe84160b30 0x7fbe840cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40742/0 (socket says 172.21.15.99:40742) 2024-03-29T09:11:34.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.327+0000 7fbe89e00700 1 -- 172.21.15.99:0/3537776013 learned_addr learned my addr 172.21.15.99:0/3537776013 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe89e00700 1 -- 172.21.15.99:0/3537776013 >> v2:172.21.15.133:3300/0 conn(0x7fbe840d0090 msgr2=0x7fbe840ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe8a800700 1 --2- 172.21.15.99:0/3537776013 >> v2:172.21.15.133:3300/0 conn(0x7fbe840d0090 0x7fbe840ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe89e00700 1 --2- 172.21.15.99:0/3537776013 >> v2:172.21.15.133:3300/0 conn(0x7fbe840d0090 0x7fbe840ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe89e00700 1 -- 172.21.15.99:0/3537776013 >> v1:172.21.15.133:6789/0 conn(0x7fbe840b1f50 legacy=0x7fbe840b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe89e00700 1 -- 172.21.15.99:0/3537776013 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84160f10 con 0x7fbe84160b30 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe8a800700 1 --2- 172.21.15.99:0/3537776013 >> v2:172.21.15.133:3300/0 conn(0x7fbe840d0090 0x7fbe840ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe89e00700 1 --2- 172.21.15.99:0/3537776013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe84160b30 0x7fbe840cfb50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbe8408c010 tx=0x7fbe78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6165c7c33ca77bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe88a00700 1 -- 172.21.15.99:0/3537776013 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78004230 con 0x7fbe84160b30 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe88a00700 1 -- 172.21.15.99:0/3537776013 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78004390 con 0x7fbe84160b30 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.328+0000 7fbe88a00700 1 -- 172.21.15.99:0/3537776013 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780036a0 con 0x7fbe84160b30 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 -- 172.21.15.99:0/3537776013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe84160b30 msgr2=0x7fbe840cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 --2- 172.21.15.99:0/3537776013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe84160b30 0x7fbe840cfb50 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbe8408c010 tx=0x7fbe78009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 -- 172.21.15.99:0/3537776013 shutdown_connections 2024-03-29T09:11:34.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 --2- 172.21.15.99:0/3537776013 >> v2:172.21.15.133:3300/0 conn(0x7fbe840d0090 0x7fbe840ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 --2- 172.21.15.99:0/3537776013 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe84160b30 0x7fbe840cfb50 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 -- 172.21.15.99:0/3537776013 >> 172.21.15.99:0/3537776013 conn(0x7fbe8409c4d0 msgr2=0x7fbe840d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 -- 172.21.15.99:0/3537776013 shutdown_connections 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.329+0000 7fbe8c800700 1 -- 172.21.15.99:0/3537776013 wait complete. 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.331+0000 7fbe8c800700 1 Processor -- start 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.331+0000 7fbe8c800700 1 -- start start 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe8c800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe840b1f50 0x7fbe8415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe8c800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840d0090 0x7fbe840b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe8c800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe84160b30 0x7fbe8415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe8c800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840cc190 con 0x7fbe840b1f50 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe8c800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840cc470 con 0x7fbe840d0090 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe8c800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840cc750 con 0x7fbe84160b30 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe840b1f50 0x7fbe8415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe840b1f50 0x7fbe8415a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40758/0 (socket says 172.21.15.99:40758) 2024-03-29T09:11:34.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 -- 172.21.15.99:0/1221261349 learned_addr learned my addr 172.21.15.99:0/1221261349 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89400700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840d0090 0x7fbe840b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe8a800700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe84160b30 0x7fbe8415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 -- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe84160b30 msgr2=0x7fbe8415b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe84160b30 0x7fbe8415b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 -- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840d0090 msgr2=0x7fbe840b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840d0090 0x7fbe840b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.332+0000 7fbe89e00700 1 -- 172.21.15.99:0/1221261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78009550 con 0x7fbe840b1f50 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.333+0000 7fbe89e00700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe840b1f50 0x7fbe8415a4f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009870 tx=0x7fbe78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.333+0000 7fbe7e000700 1 -- 172.21.15.99:0/1221261349 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78011830 con 0x7fbe840b1f50 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.333+0000 7fbe7e000700 1 -- 172.21.15.99:0/1221261349 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe780119b0 con 0x7fbe840b1f50 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.333+0000 7fbe7e000700 1 -- 172.21.15.99:0/1221261349 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78011d60 con 0x7fbe840b1f50 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.333+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe840b4c20 con 0x7fbe840b1f50 2024-03-29T09:11:34.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.333+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8415aa30 con 0x7fbe840b1f50 2024-03-29T09:11:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.334+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe840678b0 con 0x7fbe840b1f50 2024-03-29T09:11:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.335+0000 7fbe7e000700 1 -- 172.21.15.99:0/1221261349 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fbe7801b440 con 0x7fbe840b1f50 2024-03-29T09:11:34.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.336+0000 7fbe7e000700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbe6806ac90 0x7fbe6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.336+0000 7fbe7e000700 1 -- 172.21.15.99:0/1221261349 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5658+0+0 (secure 0 0 0) 0x7fbe7808acf0 con 0x7fbe840b1f50 2024-03-29T09:11:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.336+0000 7fbe89400700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbe6806ac90 0x7fbe6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:34.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.337+0000 7fbe89400700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbe6806ac90 0x7fbe6806d150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbe841f62e0 tx=0x7fbe74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:34.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.340+0000 7fbe7e000700 1 -- 172.21.15.99:0/1221261349 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fbe78055ef0 con 0x7fbe840b1f50 2024-03-29T09:11:34.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:34.549+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe8415abc0 con 0x7fbe840b1f50 2024-03-29T09:11:35.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.112+0000 7fbe7e000700 1 -- 172.21.15.99:0/1221261349 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v46) v1 ==== 160+0+0 (secure 0 0 0) 0x7fbe78059510 con 0x7fbe840b1f50 2024-03-29T09:11:35.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbe6806ac90 msgr2=0x7fbe6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbe6806ac90 0x7fbe6806d150 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbe841f62e0 tx=0x7fbe74009040 comp rx=0 tx=0).stop 2024-03-29T09:11:35.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe840b1f50 msgr2=0x7fbe8415a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe840b1f50 0x7fbe8415a4f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbe78009870 tx=0x7fbe78004750 comp rx=0 tx=0).stop 2024-03-29T09:11:35.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 shutdown_connections 2024-03-29T09:11:35.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fbe6806ac90 0x7fbe6806d150 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe84160b30 0x7fbe8415b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe840b1f50 0x7fbe8415a4f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 --2- 172.21.15.99:0/1221261349 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe840d0090 0x7fbe840b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 >> 172.21.15.99:0/1221261349 conn(0x7fbe8409c4d0 msgr2=0x7fbe8415b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:35.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 shutdown_connections 2024-03-29T09:11:35.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.115+0000 7fbe8c800700 1 -- 172.21.15.99:0/1221261349 wait complete. 2024-03-29T09:11:35.116 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'data_new' removed 2024-03-29T09:11:35.134 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T09:11:35.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.327+0000 7f07faa00700 1 Processor -- start 2024-03-29T09:11:35.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.328+0000 7f07faa00700 1 -- start start 2024-03-29T09:11:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07faa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07f415d740 0x7f07f40d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07faa00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f40cf1b0 con 0x7f07f40d1d40 2024-03-29T09:11:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07faa00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f40d2140 con 0x7f07f415d360 2024-03-29T09:11:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07faa00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f40b60c0 con 0x7f07f415d740 2024-03-29T09:11:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07ebe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f07f415d740 0x7f07f40d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07f8a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f07f40d1d40 0x7f07f40b6db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45418/0 (socket says 172.21.15.99:45418) 2024-03-29T09:11:35.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07f8a00700 1 -- 172.21.15.99:0/2859567824 learned_addr learned my addr 172.21.15.99:0/2859567824 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07ebe00700 1 -- 172.21.15.99:0/2859567824 >> v1:172.21.15.133:6789/0 conn(0x7f07f415d360 legacy=0x7f07f40cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07ebe00700 1 -- 172.21.15.99:0/2859567824 >> v1:172.21.15.99:6789/0 conn(0x7f07f40d1d40 legacy=0x7f07f40b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.329+0000 7f07f3400700 1 -- 172.21.15.99:0/2859567824 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827476912 0 0) 0x7f07f40cf1b0 con 0x7f07f40d1d40 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.330+0000 7f07ebe00700 1 -- 172.21.15.99:0/2859567824 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f40d0690 con 0x7f07f415d740 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.330+0000 7f07ebe00700 1 --2- 172.21.15.99:0/2859567824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f415d740 0x7f07f40d1800 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009840 tx=0x7f07dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b2d71ddfc89aaf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.330+0000 7f07f3400700 1 -- 172.21.15.99:0/2859567824 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc004230 con 0x7f07f415d740 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.330+0000 7f07f3400700 1 -- 172.21.15.99:0/2859567824 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc004390 con 0x7f07f415d740 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.330+0000 7f07f3400700 1 -- 172.21.15.99:0/2859567824 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0036a0 con 0x7f07f415d740 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.330+0000 7f07faa00700 1 -- 172.21.15.99:0/2859567824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f415d740 msgr2=0x7f07f40d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.330+0000 7f07faa00700 1 --2- 172.21.15.99:0/2859567824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f415d740 0x7f07f40d1800 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009840 tx=0x7f07dc009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:35.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.331+0000 7f07faa00700 1 -- 172.21.15.99:0/2859567824 shutdown_connections 2024-03-29T09:11:35.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.331+0000 7f07faa00700 1 --2- 172.21.15.99:0/2859567824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f415d740 0x7f07f40d1800 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.331+0000 7f07faa00700 1 -- 172.21.15.99:0/2859567824 >> 172.21.15.99:0/2859567824 conn(0x7f07f409c530 msgr2=0x7f07f40d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:35.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.331+0000 7f07faa00700 1 -- 172.21.15.99:0/2859567824 shutdown_connections 2024-03-29T09:11:35.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.331+0000 7f07faa00700 1 -- 172.21.15.99:0/2859567824 wait complete. 2024-03-29T09:11:35.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.333+0000 7f07faa00700 1 Processor -- start 2024-03-29T09:11:35.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.333+0000 7f07faa00700 1 -- start start 2024-03-29T09:11:35.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.334+0000 7f07faa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f40d1d40 0x7f07f40c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.334+0000 7f07faa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07f415d360 0x7f07f4160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.334+0000 7f07faa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f415d740 0x7f07f41f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.334+0000 7f07faa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f41f3ad0 con 0x7f07f40d1d40 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.334+0000 7f07faa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f4160bc0 con 0x7f07f415d740 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.334+0000 7f07faa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f41f2fb0 con 0x7f07f415d360 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.334+0000 7f07f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f40d1d40 0x7f07f40c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f40d1d40 0x7f07f40c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40784/0 (socket says 172.21.15.99:40784) 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 -- 172.21.15.99:0/1545487418 learned_addr learned my addr 172.21.15.99:0/1545487418 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07ebe00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07f415d360 0x7f07f4160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f8a00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f415d740 0x7f07f41f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 -- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07f415d360 msgr2=0x7f07f4160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07f415d360 0x7f07f4160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 -- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f415d740 msgr2=0x7f07f41f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f415d740 0x7f07f41f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 -- 172.21.15.99:0/1545487418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0094a0 con 0x7f07f40d1d40 2024-03-29T09:11:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f3e00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f40d1d40 0x7f07f40c9450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c4d0 tx=0x7f07e400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f0c00700 1 -- 172.21.15.99:0/1545487418 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4011790 con 0x7f07f40d1d40 2024-03-29T09:11:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.335+0000 7f07f0c00700 1 -- 172.21.15.99:0/1545487418 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e40118f0 con 0x7f07f40d1d40 2024-03-29T09:11:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.336+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f41f3230 con 0x7f07f40d1d40 2024-03-29T09:11:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.336+0000 7f07f0c00700 1 -- 172.21.15.99:0/1545487418 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4011ca0 con 0x7f07f40d1d40 2024-03-29T09:11:35.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.336+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f41f37f0 con 0x7f07f40d1d40 2024-03-29T09:11:35.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.337+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f40678b0 con 0x7f07f40d1d40 2024-03-29T09:11:35.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.338+0000 7f07f0c00700 1 -- 172.21.15.99:0/1545487418 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f07e401c460 con 0x7f07f40d1d40 2024-03-29T09:11:35.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.338+0000 7f07f0c00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f07d006ac90 0x7f07d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.339+0000 7f07f0c00700 1 -- 172.21.15.99:0/1545487418 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f07e4088ed0 con 0x7f07f40d1d40 2024-03-29T09:11:35.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.339+0000 7f07ebe00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f07d006ac90 0x7f07d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.339+0000 7f07ebe00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f07d006ac90 0x7f07d006d150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0090b0 tx=0x7f07dc01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:35.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.343+0000 7f07f0c00700 1 -- 172.21.15.99:0/1545487418 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f07e4054270 con 0x7f07f40d1d40 2024-03-29T09:11:35.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.551+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f07f4067b00 con 0x7f07f40d1d40 2024-03-29T09:11:35.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.555+0000 7f07f0c00700 1 -- 172.21.15.99:0/1545487418 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f07e4057890 con 0x7f07f40d1d40 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.557+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f07d006ac90 msgr2=0x7f07d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.557+0000 7f07faa00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f07d006ac90 0x7f07d006d150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0090b0 tx=0x7f07dc01c040 comp rx=0 tx=0).stop 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.557+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f40d1d40 msgr2=0x7f07f40c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.557+0000 7f07faa00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f40d1d40 0x7f07f40c9450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c4d0 tx=0x7f07e400c7e0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 shutdown_connections 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f07d006ac90 0x7f07d006d150 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07f415d740 0x7f07f41f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07f40d1d40 0x7f07f40c9450 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 --2- 172.21.15.99:0/1545487418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07f415d360 0x7f07f4160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 >> 172.21.15.99:0/1545487418 conn(0x7f07f409c530 msgr2=0x7f07f40d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 shutdown_connections 2024-03-29T09:11:35.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.558+0000 7f07faa00700 1 -- 172.21.15.99:0/1545487418 wait complete. 2024-03-29T09:11:35.576 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-03-29T09:11:35.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.768+0000 7fb886800700 1 Processor -- start 2024-03-29T09:11:35.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.769+0000 7fb886800700 1 -- start start 2024-03-29T09:11:35.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb886800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 0x7fb8800d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb886800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8800d38d0 con 0x7fb8800cfed0 2024-03-29T09:11:35.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb886800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8800d0b00 con 0x7fb8800ce610 2024-03-29T09:11:35.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb886800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800d0de0 con 0x7fb88015d140 2024-03-29T09:11:35.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb885e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb8800cfed0 0x7fb8800b5d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45428/0 (socket says 172.21.15.99:45428) 2024-03-29T09:11:35.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb885e00700 1 -- 172.21.15.99:0/2506749069 learned_addr learned my addr 172.21.15.99:0/2506749069 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:35.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb885400700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 0x7fb8800d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.770+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792905127 0 0) 0x7fb8800d38d0 con 0x7fb8800cfed0 2024-03-29T09:11:35.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8680035b0 con 0x7fb8800cfed0 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968422631 0 0) 0x7fb8800d0b00 con 0x7fb8800ce610 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8800d38d0 con 0x7fb8800ce610 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3579886292 0 0) 0x7fb8680035b0 con 0x7fb8800cfed0 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 msgr2=0x7fb8800d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 0x7fb8800d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 >> v1:172.21.15.133:6789/0 conn(0x7fb8800ce610 legacy=0x7fb8800cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800d10c0 con 0x7fb8800cfed0 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fb8780019f0 con 0x7fb8800cfed0 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 >> v1:172.21.15.99:6789/0 conn(0x7fb8800cfed0 legacy=0x7fb8800b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb868003bc0 0x7fb868003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb868004850 0x7fb868004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 0x7fb868005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800d0de0 con 0x7fb868004850 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86800bac0 con 0x7fb868003bc0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86800bc50 con 0x7fb88015d140 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb884a00700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb868004850 0x7fb868004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb885400700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb868003bc0 0x7fb868003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.771+0000 7fb885e00700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 0x7fb868005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb885400700 1 -- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 msgr2=0x7fb868005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb885400700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb88015d140 0x7fb868005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb885400700 1 -- 172.21.15.99:0/2506749069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb868004850 msgr2=0x7fb868004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb885400700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb868004850 0x7fb868004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb885400700 1 -- 172.21.15.99:0/2506749069 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680035b0 con 0x7fb868003bc0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb885400700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb868003bc0 0x7fb868003fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb874009b30 tx=0x7fb868005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f02f99eb4989c915 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400e6c0 con 0x7fb868003bc0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87400e820 con 0x7fb868003bc0 2024-03-29T09:11:35.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.772+0000 7fb87fe00700 1 -- 172.21.15.99:0/2506749069 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400eb90 con 0x7fb868003bc0 2024-03-29T09:11:35.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.773+0000 7fb886800700 1 -- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb868003bc0 msgr2=0x7fb868003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.773+0000 7fb886800700 1 --2- 172.21.15.99:0/2506749069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb868003bc0 0x7fb868003fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb874009b30 tx=0x7fb868005880 comp rx=0 tx=0).stop 2024-03-29T09:11:35.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.773+0000 7fb885e00700 1 -- 172.21.15.99:0/2506749069 reap_dead start 2024-03-29T09:11:35.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.773+0000 7fb886800700 1 -- 172.21.15.99:0/2506749069 shutdown_connections 2024-03-29T09:11:35.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.773+0000 7fb886800700 1 -- 172.21.15.99:0/2506749069 >> 172.21.15.99:0/2506749069 conn(0x7fb88009c2e0 msgr2=0x7fb8800d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:35.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.773+0000 7fb886800700 1 -- 172.21.15.99:0/2506749069 shutdown_connections 2024-03-29T09:11:35.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.773+0000 7fb886800700 1 -- 172.21.15.99:0/2506749069 wait complete. 2024-03-29T09:11:35.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.775+0000 7fb886800700 1 Processor -- start 2024-03-29T09:11:35.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.775+0000 7fb886800700 1 -- start start 2024-03-29T09:11:35.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb886800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb868003bc0 0x7fb8801552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb886800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8800c9980 0x7fb880157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb886800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8801f27c0 0x7fb8801f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb886800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880155050 con 0x7fb8801f27c0 2024-03-29T09:11:35.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb886800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb880154a00 con 0x7fb8800c9980 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb886800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb880155fa0 con 0x7fb868003bc0 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8801f27c0 0x7fb8801f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8801f27c0 0x7fb8801f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40800/0 (socket says 172.21.15.99:40800) 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 -- 172.21.15.99:0/1405758533 learned_addr learned my addr 172.21.15.99:0/1405758533 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885400700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb868003bc0 0x7fb8801552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb884a00700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8800c9980 0x7fb880157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 -- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb868003bc0 msgr2=0x7fb8801552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb868003bc0 0x7fb8801552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 -- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8800c9980 msgr2=0x7fb880157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8800c9980 0x7fb880157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb885e00700 1 -- 172.21.15.99:0/1405758533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880154c70 con 0x7fb8801f27c0 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.776+0000 7fb884a00700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8800c9980 0x7fb880157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.777+0000 7fb885e00700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8801f27c0 0x7fb8801f2c50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb878000f80 tx=0x7fb87800a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:35.779 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.777+0000 7fb87d600700 1 -- 172.21.15.99:0/1405758533 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87800ade0 con 0x7fb8801f27c0 2024-03-29T09:11:35.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.777+0000 7fb87d600700 1 -- 172.21.15.99:0/1405758533 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8780133f0 con 0x7fb8801f27c0 2024-03-29T09:11:35.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.777+0000 7fb87d600700 1 -- 172.21.15.99:0/1405758533 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8780137a0 con 0x7fb8801f27c0 2024-03-29T09:11:35.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.777+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8801f3190 con 0x7fb8801f27c0 2024-03-29T09:11:35.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.777+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb880156280 con 0x7fb8801f27c0 2024-03-29T09:11:35.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.778+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb880067730 con 0x7fb8801f27c0 2024-03-29T09:11:35.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.779+0000 7fb87d600700 1 -- 172.21.15.99:0/1405758533 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb878013900 con 0x7fb8801f27c0 2024-03-29T09:11:35.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.780+0000 7fb87d600700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb86006ac90 0x7fb86006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:35.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.780+0000 7fb87d600700 1 -- 172.21.15.99:0/1405758533 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb878089c50 con 0x7fb8801f27c0 2024-03-29T09:11:35.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.780+0000 7fb885400700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb86006ac90 0x7fb86006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:35.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.781+0000 7fb885400700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb86006ac90 0x7fb86006d150 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb87401f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:35.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:35.785+0000 7fb87d600700 1 -- 172.21.15.99:0/1405758533 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb87808f020 con 0x7fb8801f27c0 2024-03-29T09:11:36.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.070+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7fb8800c67a0 con 0x7fb8801f27c0 2024-03-29T09:11:36.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.116+0000 7fb87d600700 1 -- 172.21.15.99:0/1405758533 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v163) v1 ==== 147+0+0 (secure 0 0 0) 0x7fb878017070 con 0x7fb8801f27c0 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.118+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb86006ac90 msgr2=0x7fb86006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb86006ac90 0x7fb86006d150 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb874000c00 tx=0x7fb87401f040 comp rx=0 tx=0).stop 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8801f27c0 msgr2=0x7fb8801f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8801f27c0 0x7fb8801f2c50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb878000f80 tx=0x7fb87800a5b0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 shutdown_connections 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb86006ac90 0x7fb86006d150 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb8801f27c0 0x7fb8801f2c50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb868003bc0 0x7fb8801552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 --2- 172.21.15.99:0/1405758533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8800c9980 0x7fb880157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 >> 172.21.15.99:0/1405758533 conn(0x7fb88009c2e0 msgr2=0x7fb88015e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:36.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 shutdown_connections 2024-03-29T09:11:36.120 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.119+0000 7fb886800700 1 -- 172.21.15.99:0/1405758533 wait complete. 2024-03-29T09:11:36.137 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2024-03-29T09:11:36.137 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-03-29T09:11:36.138 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-03-29T09:11:36.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.327+0000 7fd93bc00700 1 Processor -- start 2024-03-29T09:11:36.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.327+0000 7fd93bc00700 1 -- start start 2024-03-29T09:11:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.328+0000 7fd93bc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd9341608e0 0x7fd9340d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.328+0000 7fd93bc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 0x7fd9340ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.328+0000 7fd93bc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9340cddc0 con 0x7fd934160cc0 2024-03-29T09:11:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.328+0000 7fd93bc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9340ce0a0 con 0x7fd9341608e0 2024-03-29T09:11:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd93bc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340ce380 con 0x7fd9340d1b20 2024-03-29T09:11:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fd934160cc0 0x7fd9340d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45430/0 (socket says 172.21.15.99:45430) 2024-03-29T09:11:36.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933e00700 1 -- 172.21.15.99:0/354297802 learned_addr learned my addr 172.21.15.99:0/354297802 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:36.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd939c00700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 0x7fd9340ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd939200700 1 --2- 172.21.15.99:0/354297802 >> v2:172.21.15.133:3300/0 conn(0x7fd9341608e0 0x7fd9340d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321851055 0 0) 0x7fd9340cddc0 con 0x7fd934160cc0 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd91c0035b0 con 0x7fd934160cc0 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896296725 0 0) 0x7fd91c0035b0 con 0x7fd934160cc0 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 msgr2=0x7fd9340ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933400700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 0x7fd9340ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 >> v2:172.21.15.133:3300/0 conn(0x7fd9341608e0 msgr2=0x7fd9340d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.329+0000 7fd933400700 1 --2- 172.21.15.99:0/354297802 >> v2:172.21.15.133:3300/0 conn(0x7fd9341608e0 0x7fd9340d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340ce660 con 0x7fd934160cc0 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fd920001670 con 0x7fd934160cc0 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 >> v1:172.21.15.99:6789/0 conn(0x7fd934160cc0 legacy=0x7fd9340d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 0x7fd91c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c003f80 0x7fd91c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd91c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340ce0a0 con 0x7fd91c004d90 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340ce380 con 0x7fd91c003f80 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c00ba40 con 0x7fd9340d1b20 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd939c00700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd91c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd933e00700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c003f80 0x7fd91c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd939200700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 0x7fd91c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd939c00700 1 -- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 msgr2=0x7fd91c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd939c00700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9340d1b20 0x7fd91c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd939c00700 1 -- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c003f80 msgr2=0x7fd91c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd939c00700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd91c003f80 0x7fd91c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.330+0000 7fd939c00700 1 -- 172.21.15.99:0/354297802 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0035b0 con 0x7fd91c004d90 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd939c00700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd91c005210 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009f90 tx=0x7fd91c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e14646e8186ebe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00ec30 con 0x7fd91c004d90 2024-03-29T09:11:36.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c00f040 con 0x7fd91c004d90 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd933400700 1 -- 172.21.15.99:0/354297802 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c014480 con 0x7fd91c004d90 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd93bc00700 1 -- 172.21.15.99:0/354297802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 msgr2=0x7fd91c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd93bc00700 1 --2- 172.21.15.99:0/354297802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd91c005210 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009f90 tx=0x7fd91c005880 comp rx=0 tx=0).stop 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd939c00700 1 -- 172.21.15.99:0/354297802 reap_dead start 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd93bc00700 1 -- 172.21.15.99:0/354297802 shutdown_connections 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd93bc00700 1 -- 172.21.15.99:0/354297802 >> 172.21.15.99:0/354297802 conn(0x7fd93409c490 msgr2=0x7fd9340d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd93bc00700 1 -- 172.21.15.99:0/354297802 shutdown_connections 2024-03-29T09:11:36.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.331+0000 7fd93bc00700 1 -- 172.21.15.99:0/354297802 wait complete. 2024-03-29T09:11:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.333+0000 7fd93bc00700 1 Processor -- start 2024-03-29T09:11:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.333+0000 7fd93bc00700 1 -- start start 2024-03-29T09:11:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd93bc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd93415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd93bc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9341f5fb0 0x7fd9341f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd93bc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9341563c0 0x7fd934156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd93bc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341f6970 con 0x7fd91c004d90 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd93bc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341f6ea0 con 0x7fd9341563c0 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd93bc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd934156db0 con 0x7fd9341f5fb0 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd939200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd93415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd939200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd93415c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40822/0 (socket says 172.21.15.99:40822) 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.334+0000 7fd939200700 1 -- 172.21.15.99:0/2314121434 learned_addr learned my addr 172.21.15.99:0/2314121434 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd933e00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9341f5fb0 0x7fd9341f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd939c00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9341563c0 0x7fd934156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd939200700 1 -- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9341f5fb0 msgr2=0x7fd9341f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd939200700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9341f5fb0 0x7fd9341f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd939200700 1 -- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9341563c0 msgr2=0x7fd934156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:36.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd939200700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9341563c0 0x7fd934156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd939200700 1 -- 172.21.15.99:0/2314121434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93415b2a0 con 0x7fd91c004d90 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd933e00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9341f5fb0 0x7fd9341f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd939200700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd93415c340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd928011ad0 tx=0x7fd928011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd930c00700 1 -- 172.21.15.99:0/2314121434 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800f9a0 con 0x7fd91c004d90 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd930c00700 1 -- 172.21.15.99:0/2314121434 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92800fb00 con 0x7fd91c004d90 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93415b580 con 0x7fd91c004d90 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd930c00700 1 -- 172.21.15.99:0/2314121434 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800feb0 con 0x7fd91c004d90 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.335+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93415bb40 con 0x7fd91c004d90 2024-03-29T09:11:36.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.336+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9340677a0 con 0x7fd91c004d90 2024-03-29T09:11:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.337+0000 7fd930c00700 1 -- 172.21.15.99:0/2314121434 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd928014440 con 0x7fd91c004d90 2024-03-29T09:11:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.338+0000 7fd930c00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd91406ac90 0x7fd91406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.338+0000 7fd930c00700 1 -- 172.21.15.99:0/2314121434 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fd9280898e0 con 0x7fd91c004d90 2024-03-29T09:11:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.338+0000 7fd933e00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd91406ac90 0x7fd91406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:36.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.339+0000 7fd933e00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd91406ac90 0x7fd91406d150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd93414c600 tx=0x7fd920009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:36.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.342+0000 7fd930c00700 1 -- 172.21.15.99:0/2314121434 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd928054c90 con 0x7fd91c004d90 2024-03-29T09:11:36.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:36.630+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fd9340d0700 con 0x7fd91c004d90 2024-03-29T09:11:37.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.121+0000 7fd930c00700 1 -- 172.21.15.99:0/2314121434 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v164) v1 ==== 137+0+0 (secure 0 0 0) 0x7fd9280582b0 con 0x7fd91c004d90 2024-03-29T09:11:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.124+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd91406ac90 msgr2=0x7fd91406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.124+0000 7fd93bc00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd91406ac90 0x7fd91406d150 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd93414c600 tx=0x7fd920009040 comp rx=0 tx=0).stop 2024-03-29T09:11:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.124+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 msgr2=0x7fd93415c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.124 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.124+0000 7fd93bc00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd93415c340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd928011ad0 tx=0x7fd928011de0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 shutdown_connections 2024-03-29T09:11:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd91406ac90 0x7fd91406d150 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9341563c0 0x7fd934156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd91c004d90 0x7fd93415c340 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.125 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 --2- 172.21.15.99:0/2314121434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9341f5fb0 0x7fd9341f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 >> 172.21.15.99:0/2314121434 conn(0x7fd93409c490 msgr2=0x7fd9340c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:37.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 shutdown_connections 2024-03-29T09:11:37.126 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.125+0000 7fd93bc00700 1 -- 172.21.15.99:0/2314121434 wait complete. 2024-03-29T09:11:37.144 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T09:11:37.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.339+0000 7f11cb400700 1 Processor -- start 2024-03-29T09:11:37.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.339+0000 7f11cb400700 1 -- start start 2024-03-29T09:11:37.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.339+0000 7f11cb400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 0x7f11c40d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.339+0000 7f11cb400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c415d800 con 0x7f11c40d05c0 2024-03-29T09:11:37.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.339+0000 7f11cb400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c40d0a20 con 0x7f11c40d3130 2024-03-29T09:11:37.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.339+0000 7f11cb400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40d1900 con 0x7f11c415d420 2024-03-29T09:11:37.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f11c40d05c0 0x7f11c40ce760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:45432/0 (socket says 172.21.15.99:45432) 2024-03-29T09:11:37.340 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3e00700 1 -- 172.21.15.99:0/3164416510 learned_addr learned my addr 172.21.15.99:0/3164416510 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:37.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516844962 0 0) 0x7f11c415d800 con 0x7f11c40d05c0 2024-03-29T09:11:37.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c8a00700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 0x7f11c40d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ac0035b0 con 0x7f11c40d05c0 2024-03-29T09:11:37.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 912364623 0 0) 0x7f11ac0035b0 con 0x7f11c40d05c0 2024-03-29T09:11:37.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 msgr2=0x7f11c40d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 0x7f11c40d1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 >> v1:172.21.15.133:6789/0 conn(0x7f11c40d3130 legacy=0x7f11c40b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40d1be0 con 0x7f11c40d05c0 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f11b0001670 con 0x7f11c40d05c0 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.340+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 >> v1:172.21.15.99:6789/0 conn(0x7f11c40d05c0 legacy=0x7f11c40ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c3400700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11ac003a30 0x7f11ac003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c3400700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac004850 0x7f11ac004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c3400700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 0x7f11ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40d1900 con 0x7f11ac003a30 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac00bac0 con 0x7f11ac004850 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac00bc80 con 0x7f11c415d420 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c8a00700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11ac003a30 0x7f11ac003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c9400700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 0x7f11ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c3e00700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac004850 0x7f11ac004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c8a00700 1 -- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 msgr2=0x7f11ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c8a00700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c415d420 0x7f11ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c8a00700 1 -- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac004850 msgr2=0x7f11ac004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c8a00700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac004850 0x7f11ac004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c8a00700 1 -- 172.21.15.99:0/3164416510 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0035b0 con 0x7f11ac003a30 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.341+0000 7f11c8a00700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11ac003a30 0x7f11ac003e50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009b30 tx=0x7f11ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed82a95e0473d4ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800e6c0 con 0x7f11ac003a30 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b800e820 con 0x7f11ac003a30 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11c3400700 1 -- 172.21.15.99:0/3164416510 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b800eb90 con 0x7f11ac003a30 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11cb400700 1 -- 172.21.15.99:0/3164416510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11ac003a30 msgr2=0x7f11ac003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11cb400700 1 --2- 172.21.15.99:0/3164416510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11ac003a30 0x7f11ac003e50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009b30 tx=0x7f11ac005880 comp rx=0 tx=0).stop 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11c9400700 1 -- 172.21.15.99:0/3164416510 reap_dead start 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11cb400700 1 -- 172.21.15.99:0/3164416510 shutdown_connections 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11cb400700 1 -- 172.21.15.99:0/3164416510 >> 172.21.15.99:0/3164416510 conn(0x7f11c408afd0 msgr2=0x7f11c409c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:37.344 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11cb400700 1 -- 172.21.15.99:0/3164416510 shutdown_connections 2024-03-29T09:11:37.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.342+0000 7f11cb400700 1 -- 172.21.15.99:0/3164416510 wait complete. 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.344+0000 7f11cb400700 1 Processor -- start 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.344+0000 7f11cb400700 1 -- start start 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11cb400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac003a30 0x7f11c415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11cb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c4160100 0x7f11c40c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11cb400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c40ccfb0 0x7f11c40ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11cb400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415f060 con 0x7f11c4160100 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11cb400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415f340 con 0x7f11ac003a30 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11cb400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415f620 con 0x7f11c40ccfb0 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c4160100 0x7f11c40c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11c3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c4160100 0x7f11c40c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:40840/0 (socket says 172.21.15.99:40840) 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11c3e00700 1 -- 172.21.15.99:0/2810187080 learned_addr learned my addr 172.21.15.99:0/2810187080 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:37.347 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.345+0000 7f11c8a00700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac003a30 0x7f11c415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c9400700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c40ccfb0 0x7f11c40ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c3e00700 1 -- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c40ccfb0 msgr2=0x7f11c40ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c3e00700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c40ccfb0 0x7f11c40ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c3e00700 1 -- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac003a30 msgr2=0x7f11c415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c3e00700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac003a30 0x7f11c415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c3e00700 1 -- 172.21.15.99:0/2810187080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c415f900 con 0x7f11c4160100 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c3e00700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c4160100 0x7f11c40c96b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e840 tx=0x7f11b000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c0c00700 1 -- 172.21.15.99:0/2810187080 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0012590 con 0x7f11c4160100 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c0c00700 1 -- 172.21.15.99:0/2810187080 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b00126f0 con 0x7f11c4160100 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c415fbe0 con 0x7f11c4160100 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.346+0000 7f11c0c00700 1 -- 172.21.15.99:0/2810187080 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0012a80 con 0x7f11c4160100 2024-03-29T09:11:37.348 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.347+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c40ca7b0 con 0x7f11c4160100 2024-03-29T09:11:37.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.348+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c415fe20 con 0x7f11c4160100 2024-03-29T09:11:37.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.348+0000 7f11c0c00700 1 -- 172.21.15.99:0/2810187080 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f11b0012be0 con 0x7f11c4160100 2024-03-29T09:11:37.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.349+0000 7f11c0c00700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11a406ac90 0x7f11a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:37.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.349+0000 7f11c0c00700 1 -- 172.21.15.99:0/2810187080 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f11b0088f30 con 0x7f11c4160100 2024-03-29T09:11:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.349+0000 7f11c8a00700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11a406ac90 0x7f11a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:37.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.350+0000 7f11c8a00700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11a406ac90 0x7f11a406d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f11b80090b0 tx=0x7f11b801f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:37.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.354+0000 7f11c0c00700 1 -- 172.21.15.99:0/2810187080 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f11c415fe20 con 0x7f11c4160100 2024-03-29T09:11:37.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.558+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f11c415fe20 con 0x7f11c4160100 2024-03-29T09:11:37.562 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.562+0000 7f11c0c00700 1 -- 172.21.15.99:0/2810187080 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f11c415fe20 con 0x7f11c4160100 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.564+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11a406ac90 msgr2=0x7f11a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.564+0000 7f11cb400700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11a406ac90 0x7f11a406d150 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f11b80090b0 tx=0x7f11b801f040 comp rx=0 tx=0).stop 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c4160100 msgr2=0x7f11c40c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c4160100 0x7f11c40c96b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e840 tx=0x7f11b000eb50 comp rx=0 tx=0).stop 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 shutdown_connections 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f11a406ac90 0x7f11a406d150 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11c40ccfb0 0x7f11c40ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11ac003a30 0x7f11c415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.565 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 --2- 172.21.15.99:0/2810187080 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11c4160100 0x7f11c40c96b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:37.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 >> 172.21.15.99:0/2810187080 conn(0x7f11c408afd0 msgr2=0x7f11c40ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:37.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 shutdown_connections 2024-03-29T09:11:37.566 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:37.565+0000 7f11cb400700 1 -- 172.21.15.99:0/2810187080 wait complete. 2024-03-29T09:11:37.583 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T09:11:37.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:11:37.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:11:37.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T09:11:37.768 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:37.768 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:37.812 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345608k,nr_inodes=4086402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-03-29T09:11:37.812 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:11:37.812 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:37.856 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:11:37.856 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:37.856 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:37.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345608k,nr_inodes=4086402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-03-29T09:11:37.897 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:11:37.897 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:37.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:11:37.941 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T09:11:37.941 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:11:38.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.131+0000 7f03c8c00700 1 Processor -- start 2024-03-29T09:11:38.132 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.132+0000 7f03c8c00700 1 -- start start 2024-03-29T09:11:38.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c8c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03c415ae70 0x7f03c415b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c8c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03c40b0b50 0x7f03c40b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c8c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c40b14f0 con 0x7f03c40b36f0 2024-03-29T09:11:38.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c8c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c415c220 con 0x7f03c415ae70 2024-03-29T09:11:38.133 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c8c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c415b8f0 con 0x7f03c40b0b50 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c1e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03c415ae70 0x7f03c415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c1e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f03c415ae70 0x7f03c415b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53560/0 (socket says 172.21.15.99:53560) 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c1e00700 1 -- 172.21.15.99:0/1873504885 learned_addr learned my addr 172.21.15.99:0/1873504885 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c1400700 1 --2- 172.21.15.99:0/1873504885 >> v2:172.21.15.133:3300/0 conn(0x7f03c40b0b50 0x7f03c40b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c1e00700 1 -- 172.21.15.99:0/1873504885 >> v2:172.21.15.133:3300/0 conn(0x7f03c40b0b50 msgr2=0x7f03c40b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.133+0000 7f03c1e00700 1 --2- 172.21.15.99:0/1873504885 >> v2:172.21.15.133:3300/0 conn(0x7f03c40b0b50 0x7f03c40b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c1e00700 1 -- 172.21.15.99:0/1873504885 >> v1:172.21.15.99:6789/0 conn(0x7f03c40b36f0 legacy=0x7f03c40b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c1e00700 1 -- 172.21.15.99:0/1873504885 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c415bbd0 con 0x7f03c415ae70 2024-03-29T09:11:38.134 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c1400700 1 --2- 172.21.15.99:0/1873504885 >> v2:172.21.15.133:3300/0 conn(0x7f03c40b0b50 0x7f03c40b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:38.135 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c1e00700 1 --2- 172.21.15.99:0/1873504885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c415ae70 0x7f03c415b250 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f03c40ce380 tx=0x7f03b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3d92ca9fed722 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c0a00700 1 -- 172.21.15.99:0/1873504885 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400b870 con 0x7f03c415ae70 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c0a00700 1 -- 172.21.15.99:0/1873504885 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b400b9d0 con 0x7f03c415ae70 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c0a00700 1 -- 172.21.15.99:0/1873504885 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b400bd60 con 0x7f03c415ae70 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c8c00700 1 -- 172.21.15.99:0/1873504885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c415ae70 msgr2=0x7f03c415b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.134+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1873504885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c415ae70 0x7f03c415b250 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f03c40ce380 tx=0x7f03b4009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.135+0000 7f03c8c00700 1 -- 172.21.15.99:0/1873504885 shutdown_connections 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.135+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1873504885 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c415ae70 0x7f03c415b250 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.135+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1873504885 >> v2:172.21.15.133:3300/0 conn(0x7f03c40b0b50 0x7f03c40b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.135+0000 7f03c8c00700 1 -- 172.21.15.99:0/1873504885 >> 172.21.15.99:0/1873504885 conn(0x7f03c4066540 msgr2=0x7f03c4066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.135+0000 7f03c8c00700 1 -- 172.21.15.99:0/1873504885 shutdown_connections 2024-03-29T09:11:38.136 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.135+0000 7f03c8c00700 1 -- 172.21.15.99:0/1873504885 wait complete. 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.136+0000 7f03c8c00700 1 Processor -- start 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.137+0000 7f03c8c00700 1 -- start start 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.137+0000 7f03c8c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c40b0b50 0x7f03c40ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.137+0000 7f03c8c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c40b36f0 0x7f03c414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.137+0000 7f03c8c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03c415ae70 0x7f03c4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c8c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c414f2d0 con 0x7f03c40b36f0 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c8c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c4157ff0 con 0x7f03c40b0b50 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c8c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c40aded0 con 0x7f03c415ae70 2024-03-29T09:11:38.138 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c40b36f0 0x7f03c414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c40b36f0 0x7f03c414fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53562/0 (socket says 172.21.15.99:53562) 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 -- 172.21.15.99:0/1473000192 learned_addr learned my addr 172.21.15.99:0/1473000192 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1e00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c40b0b50 0x7f03c40ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c2800700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03c415ae70 0x7f03c4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 -- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03c415ae70 msgr2=0x7f03c4150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03c415ae70 0x7f03c4150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 -- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c40b0b50 msgr2=0x7f03c40ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c40b0b50 0x7f03c40ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.138+0000 7f03c1400700 1 -- 172.21.15.99:0/1473000192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4009550 con 0x7f03c40b36f0 2024-03-29T09:11:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.139+0000 7f03c1400700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c40b36f0 0x7f03c414fe00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c780 tx=0x7f03b000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.139+0000 7f03ba000700 1 -- 172.21.15.99:0/1473000192 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0011800 con 0x7f03c40b36f0 2024-03-29T09:11:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.139+0000 7f03ba000700 1 -- 172.21.15.99:0/1473000192 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0011960 con 0x7f03c40b36f0 2024-03-29T09:11:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.139+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c414eff0 con 0x7f03c40b36f0 2024-03-29T09:11:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.139+0000 7f03ba000700 1 -- 172.21.15.99:0/1473000192 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0011cd0 con 0x7f03c40b36f0 2024-03-29T09:11:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.139+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c4157c10 con 0x7f03c40b36f0 2024-03-29T09:11:38.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.140+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c4157790 con 0x7f03c40b36f0 2024-03-29T09:11:38.141 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.141+0000 7f03ba000700 1 -- 172.21.15.99:0/1473000192 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f03b0021350 con 0x7f03c40b36f0 2024-03-29T09:11:38.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.142+0000 7f03ba000700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03a406ac90 0x7f03a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.142+0000 7f03ba000700 1 -- 172.21.15.99:0/1473000192 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f03b0088fc0 con 0x7f03c40b36f0 2024-03-29T09:11:38.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.142+0000 7f03c1e00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03a406ac90 0x7f03a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.143+0000 7f03c1e00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03a406ac90 0x7f03a406d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03b4014040 tx=0x7f03b4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:38.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.146+0000 7f03ba000700 1 -- 172.21.15.99:0/1473000192 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f03c4157790 con 0x7f03c40b36f0 2024-03-29T09:11:38.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.396+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03c4158130 con 0x7f03c40b36f0 2024-03-29T09:11:38.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.397+0000 7f03ba000700 1 -- 172.21.15.99:0/1473000192 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f03c4158130 con 0x7f03c40b36f0 2024-03-29T09:11:38.397 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:38.397 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2338475025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":2338475025},{"type":"v1","addr":"172.21.15.133:6835","nonce":2338475025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5024,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/2125518501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":2125518501},{"type":"v1","addr":"172.21.15.99:6837","nonce":2125518501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5359,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3323833586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3323833586},{"type":"v1","addr":"172.21.15.133:6834","nonce":3323833586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":6024,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/1303761870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1303761870},{"type":"v1","addr":"172.21.15.99:6835","nonce":1303761870}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2024-03-29T09:11:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.399+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03a406ac90 msgr2=0x7f03a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.399+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03a406ac90 0x7f03a406d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f03b4014040 tx=0x7f03b4008030 comp rx=0 tx=0).stop 2024-03-29T09:11:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.399+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c40b36f0 msgr2=0x7f03c414fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.399+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c40b36f0 0x7f03c414fe00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c780 tx=0x7f03b000ca90 comp rx=0 tx=0).stop 2024-03-29T09:11:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 shutdown_connections 2024-03-29T09:11:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f03a406ac90 0x7f03a406d150 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f03c415ae70 0x7f03c4150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03c40b0b50 0x7f03c40ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 --2- 172.21.15.99:0/1473000192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03c40b36f0 0x7f03c414fe00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 >> 172.21.15.99:0/1473000192 conn(0x7f03c4066540 msgr2=0x7f03c415c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 shutdown_connections 2024-03-29T09:11:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.400+0000 7f03c8c00700 1 -- 172.21.15.99:0/1473000192 wait complete. 2024-03-29T09:11:38.400 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 19 2024-03-29T09:11:38.418 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T09:11:38.418 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T09:11:38.418 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T09:11:38.418 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:11:38.419 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T09:11:38.419 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T09:11:38.419 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T09:11:38.419 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:11:38.419 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T09:11:38.420 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T09:11:38.420 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T09:11:38.420 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:11:38.420 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T09:11:38.420 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T09:11:38.420 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T09:11:38.420 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:11:38.434 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T09:11:38.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T09:11:38.435 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T09:11:38.435 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T09:11:38.439 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T09:11:38.439 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T09:11:38.440 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T09:11:38.440 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T09:11:38.478 INFO:tasks.ceph.mds.a:Started 2024-03-29T09:11:38.478 DEBUG:teuthology.parallel:result is None 2024-03-29T09:11:38.480 INFO:tasks.ceph.mds.c:Started 2024-03-29T09:11:38.481 DEBUG:teuthology.parallel:result is None 2024-03-29T09:11:38.483 INFO:tasks.ceph.mds.b:Started 2024-03-29T09:11:38.483 DEBUG:teuthology.parallel:result is None 2024-03-29T09:11:38.485 INFO:tasks.ceph.mds.d:Started 2024-03-29T09:11:38.486 DEBUG:teuthology.parallel:result is None 2024-03-29T09:11:38.486 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T09:11:38.622 INFO:tasks.ceph.mds.a.smithi099.stdout:starting mds.a at 2024-03-29T09:11:38.623 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-03-29T09:11:38.628 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-03-29T09:11:38.631 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-03-29T09:11:38.693 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.695+0000 7fa967e00700 1 Processor -- start 2024-03-29T09:11:38.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.695+0000 7fa967e00700 1 -- start start 2024-03-29T09:11:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa967e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9600d1b20 0x7fa9600d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa967e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa960160cc0 0x7fa9600ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa967e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9600cddc0 con 0x7fa9601608e0 2024-03-29T09:11:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa967e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9600ce0a0 con 0x7fa9600d1b20 2024-03-29T09:11:38.695 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa967e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9600ce380 con 0x7fa960160cc0 2024-03-29T09:11:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa964a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9600d1b20 0x7fa9600d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa964a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa9600d1b20 0x7fa9600d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53596/0 (socket says 172.21.15.99:53596) 2024-03-29T09:11:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa964a00700 1 -- 172.21.15.99:0/3338637616 learned_addr learned my addr 172.21.15.99:0/3338637616 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:38.696 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.696+0000 7fa965e00700 1 --2- 172.21.15.99:0/3338637616 >> v2:172.21.15.133:3300/0 conn(0x7fa960160cc0 0x7fa9600ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa964a00700 1 -- 172.21.15.99:0/3338637616 >> v2:172.21.15.133:3300/0 conn(0x7fa960160cc0 msgr2=0x7fa9600ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa964a00700 1 --2- 172.21.15.99:0/3338637616 >> v2:172.21.15.133:3300/0 conn(0x7fa960160cc0 0x7fa9600ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa964a00700 1 -- 172.21.15.99:0/3338637616 >> v1:172.21.15.133:6789/0 conn(0x7fa9601608e0 legacy=0x7fa9600d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa964a00700 1 -- 172.21.15.99:0/3338637616 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9600ce660 con 0x7fa9600d1b20 2024-03-29T09:11:38.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa95fe00700 1 -- 172.21.15.99:0/3338637616 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165073251 0 0) 0x7fa9600cddc0 con 0x7fa9601608e0 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa964a00700 1 --2- 172.21.15.99:0/3338637616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa9600d0150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa9600b91e0 tx=0x7fa94c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b053721d2b869570 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa95fe00700 1 -- 172.21.15.99:0/3338637616 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c011830 con 0x7fa9600d1b20 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa95fe00700 1 -- 172.21.15.99:0/3338637616 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c011990 con 0x7fa9600d1b20 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.697+0000 7fa95fe00700 1 -- 172.21.15.99:0/3338637616 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c011d20 con 0x7fa9600d1b20 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 -- 172.21.15.99:0/3338637616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 msgr2=0x7fa9600d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 --2- 172.21.15.99:0/3338637616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa9600d0150 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa9600b91e0 tx=0x7fa94c009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 -- 172.21.15.99:0/3338637616 shutdown_connections 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 --2- 172.21.15.99:0/3338637616 >> v2:172.21.15.133:3300/0 conn(0x7fa960160cc0 0x7fa9600ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 --2- 172.21.15.99:0/3338637616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa9600d0150 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 -- 172.21.15.99:0/3338637616 >> 172.21.15.99:0/3338637616 conn(0x7fa96009c490 msgr2=0x7fa9600d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 -- 172.21.15.99:0/3338637616 shutdown_connections 2024-03-29T09:11:38.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.698+0000 7fa967e00700 1 -- 172.21.15.99:0/3338637616 wait complete. 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.700+0000 7fa967e00700 1 Processor -- start 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.700+0000 7fa967e00700 1 -- start start 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa967e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa960157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa967e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9601608e0 0x7fa9600b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa967e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa960160cc0 0x7fa9600c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa967e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9601587b0 con 0x7fa9600d1b20 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa967e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa960146e50 con 0x7fa9601608e0 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa967e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9601f6960 con 0x7fa960160cc0 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa960157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa960157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53608/0 (socket says 172.21.15.99:53608) 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 -- 172.21.15.99:0/2616663902 learned_addr learned my addr 172.21.15.99:0/2616663902 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa964a00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9601608e0 0x7fa9600b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.701 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965e00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa960160cc0 0x7fa9600c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 -- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa960160cc0 msgr2=0x7fa9600c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa960160cc0 0x7fa9600c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 -- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9601608e0 msgr2=0x7fa9600b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9601608e0 0x7fa9600b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 -- 172.21.15.99:0/2616663902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c009550 con 0x7fa9600d1b20 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa964a00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9601608e0 0x7fa9600b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.701+0000 7fa965400700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa960157850 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa9540011e0 tx=0x7fa95400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.702+0000 7fa95d600700 1 -- 172.21.15.99:0/2616663902 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9540054e0 con 0x7fa9600d1b20 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.702+0000 7fa95d600700 1 -- 172.21.15.99:0/2616663902 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954003680 con 0x7fa9600d1b20 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.702+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9601f6c40 con 0x7fa9600d1b20 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.702+0000 7fa95d600700 1 -- 172.21.15.99:0/2616663902 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954003a30 con 0x7fa9600d1b20 2024-03-29T09:11:38.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.702+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9601584d0 con 0x7fa9600d1b20 2024-03-29T09:11:38.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.703+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa960146fc0 con 0x7fa9600d1b20 2024-03-29T09:11:38.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.704+0000 7fa95d600700 1 -- 172.21.15.99:0/2616663902 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa954003b90 con 0x7fa9600d1b20 2024-03-29T09:11:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.705+0000 7fa95d600700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa94006ac90 0x7fa94006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.705+0000 7fa95d600700 1 -- 172.21.15.99:0/2616663902 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa9540886c0 con 0x7fa9600d1b20 2024-03-29T09:11:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.705+0000 7fa964a00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa94006ac90 0x7fa94006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:38.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.706+0000 7fa964a00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa94006ac90 0x7fa94006d150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000c00 tx=0x7fa94c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:38.707 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.709+0000 7fa95d600700 1 -- 172.21.15.99:0/2616663902 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa954053a60 con 0x7fa9600d1b20 2024-03-29T09:11:38.924 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:38.926+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa9600677a0 con 0x7fa9600d1b20 2024-03-29T09:11:39.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.672+0000 7fa95d600700 1 -- 172.21.15.99:0/2616663902 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa954017020 con 0x7fa9600d1b20 2024-03-29T09:11:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa94006ac90 msgr2=0x7fa94006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa94006ac90 0x7fa94006d150 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa94c000c00 tx=0x7fa94c008030 comp rx=0 tx=0).stop 2024-03-29T09:11:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 msgr2=0x7fa960157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:39.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa960157850 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa9540011e0 tx=0x7fa95400cae0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 shutdown_connections 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa94006ac90 0x7fa94006d150 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa960160cc0 0x7fa9600c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9600d1b20 0x7fa960157850 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 --2- 172.21.15.99:0/2616663902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9601608e0 0x7fa9600b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 >> 172.21.15.99:0/2616663902 conn(0x7fa96009c490 msgr2=0x7fa96015f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.674+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 shutdown_connections 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.675+0000 7fa967e00700 1 -- 172.21.15.99:0/2616663902 wait complete. 2024-03-29T09:11:39.674 INFO:teuthology.orchestra.run.smithi099.stderr: removed all blocklist entries 2024-03-29T09:11:39.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T09:11:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.884+0000 7f425d800700 1 Processor -- start 2024-03-29T09:11:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.884+0000 7f425d800700 1 -- start start 2024-03-29T09:11:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.885+0000 7f425d800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f42581608e0 0x7f42580d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.885+0000 7f425d800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42580cddc0 con 0x7f42580d1b20 2024-03-29T09:11:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.885+0000 7f425d800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42580ce0a0 con 0x7f4258160cc0 2024-03-29T09:11:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.885+0000 7f425d800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42580ce380 con 0x7f42581608e0 2024-03-29T09:11:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.885+0000 7f4257400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f42580d1b20 0x7f42580afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59070/0 (socket says 172.21.15.99:59070) 2024-03-29T09:11:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.885+0000 7f4257400700 1 -- 172.21.15.99:0/1410793338 learned_addr learned my addr 172.21.15.99:0/1410793338 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4256a00700 1 --2- 172.21.15.99:0/1410793338 >> v2:172.21.15.133:3300/0 conn(0x7f42581608e0 0x7f42580d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808764952 0 0) 0x7f42580cddc0 con 0x7f42580d1b20 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f423c0035b0 con 0x7f42580d1b20 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 854952722 0 0) 0x7f423c0035b0 con 0x7f42580d1b20 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 >> v2:172.21.15.133:3300/0 conn(0x7f42581608e0 msgr2=0x7f42580d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 --2- 172.21.15.99:0/1410793338 >> v2:172.21.15.133:3300/0 conn(0x7f42581608e0 0x7f42580d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 >> v1:172.21.15.133:6789/0 conn(0x7f4258160cc0 legacy=0x7f42580d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580ce660 con 0x7f42580d1b20 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f42500019f0 con 0x7f42580d1b20 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 >> v1:172.21.15.99:6789/0 conn(0x7f42580d1b20 legacy=0x7f42580afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f423c003a00 0x7f423c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c004850 0x7f423c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c005210 0x7f423c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42580ce380 con 0x7f423c005210 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c00c0c0 con 0x7f423c003a00 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c00c280 con 0x7f423c004850 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.886+0000 7f4257400700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c005210 0x7f423c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4256a00700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f423c003a00 0x7f423c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4256000700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c004850 0x7f423c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4257400700 1 -- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c004850 msgr2=0x7f423c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4257400700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c004850 0x7f423c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4257400700 1 -- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f423c003a00 msgr2=0x7f423c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4257400700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f423c003a00 0x7f423c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4257400700 1 -- 172.21.15.99:0/1410793338 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0035b0 con 0x7f423c005210 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4256a00700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f423c003a00 0x7f423c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4257400700 1 -- 172.21.15.99:0/1410793338 reap_dead start 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4257400700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c005210 0x7f423c00ba10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f425000a830 tx=0x7f423c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b7f5ba29ef4598 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:39.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.887+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000a2f0 con 0x7f423c005210 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.888+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4250012620 con 0x7f423c005210 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.888+0000 7f4255600700 1 -- 172.21.15.99:0/1410793338 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500129d0 con 0x7f423c005210 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.888+0000 7f425d800700 1 -- 172.21.15.99:0/1410793338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c005210 msgr2=0x7f423c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.888+0000 7f425d800700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c005210 0x7f423c00ba10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f425000a830 tx=0x7f423c005880 comp rx=0 tx=0).stop 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.889+0000 7f425d800700 1 -- 172.21.15.99:0/1410793338 shutdown_connections 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.889+0000 7f425d800700 1 --2- 172.21.15.99:0/1410793338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f423c005210 0x7f423c00ba10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.889+0000 7f425d800700 1 -- 172.21.15.99:0/1410793338 >> 172.21.15.99:0/1410793338 conn(0x7f425809c490 msgr2=0x7f42580d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.889+0000 7f425d800700 1 -- 172.21.15.99:0/1410793338 shutdown_connections 2024-03-29T09:11:39.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.889+0000 7f425d800700 1 -- 172.21.15.99:0/1410793338 wait complete. 2024-03-29T09:11:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.890+0000 7f425d800700 1 Processor -- start 2024-03-29T09:11:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.891+0000 7f425d800700 1 -- start start 2024-03-29T09:11:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.891+0000 7f425d800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c005210 0x7f42581558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.891+0000 7f425d800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425815ade0 0x7f4258157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f425d800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258157fc0 0x7f425815ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f425d800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258159070 con 0x7f425815ade0 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f425d800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258155de0 con 0x7f4258157fc0 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f425d800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4258158460 con 0x7f423c005210 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425815ade0 0x7f4258157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425815ade0 0x7f4258157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53634/0 (socket says 172.21.15.99:53634) 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 -- 172.21.15.99:0/1690094055 learned_addr learned my addr 172.21.15.99:0/1690094055 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256a00700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c005210 0x7f42581558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4257400700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258157fc0 0x7f425815ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 -- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c005210 msgr2=0x7f42581558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c005210 0x7f42581558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 -- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258157fc0 msgr2=0x7f425815ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258157fc0 0x7f425815ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:39.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 -- 172.21.15.99:0/1690094055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258158740 con 0x7f425815ade0 2024-03-29T09:11:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.892+0000 7f4256000700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425815ade0 0x7f4258157a80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f424400f820 tx=0x7f424400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.893+0000 7f424ea00700 1 -- 172.21.15.99:0/1690094055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440052e0 con 0x7f425815ade0 2024-03-29T09:11:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.893+0000 7f424ea00700 1 -- 172.21.15.99:0/1690094055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4244005460 con 0x7f425815ade0 2024-03-29T09:11:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.893+0000 7f424ea00700 1 -- 172.21.15.99:0/1690094055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42440038c0 con 0x7f425815ade0 2024-03-29T09:11:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.893+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4258158a20 con 0x7f425815ade0 2024-03-29T09:11:39.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.893+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4258159500 con 0x7f425815ade0 2024-03-29T09:11:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.894+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42580677a0 con 0x7f425815ade0 2024-03-29T09:11:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.895+0000 7f424ea00700 1 -- 172.21.15.99:0/1690094055 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4244003a20 con 0x7f425815ade0 2024-03-29T09:11:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.896+0000 7f424ea00700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f423806ac90 0x7f423806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.896+0000 7f424ea00700 1 -- 172.21.15.99:0/1690094055 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4244088f20 con 0x7f425815ade0 2024-03-29T09:11:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.896+0000 7f4256a00700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f423806ac90 0x7f423806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:39.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.896+0000 7f4256a00700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f423806ac90 0x7f423806d150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4248009cc0 tx=0x7f4248008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:39.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:39.900+0000 7f424ea00700 1 -- 172.21.15.99:0/1690094055 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4244054300 con 0x7f425815ade0 2024-03-29T09:11:40.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.182+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f42581591b0 con 0x7f425815ade0 2024-03-29T09:11:40.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.182+0000 7f424ea00700 1 -- 172.21.15.99:0/1690094055 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f4244057920 con 0x7f425815ade0 2024-03-29T09:11:40.181 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "auth_dump": [ 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.a", 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZmrFiLOxAAKlHfn8PZHThkmZP7F+CTSA==", 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.b", 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmQJsgFhAAXgodL3u1/qBUxO/ZydPMhQ==", 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.c", 2024-03-29T09:11:40.183 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmfXkACxAAcCfih1bN4d00TqYlKIWjLg==", 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mds.d", 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmKnAzIRAA2psjQhAebm0ZQ+d833RbsA==", 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.184 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.0", 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDwhAZmLMALHRAARJg5Ykd30R69SyDTaCBIFw==", 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.1", 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDzhAZmuL5pIBAA6jXsRZ/S6bp2Clf6SAQamg==", 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.2", 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQD2hAZmTOqEIxAAbl19sfN+WekZStEZ7F3WFQ==", 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.186 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.3", 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQD5hAZmjSHvJhAAWVvxpRqaTu+An+sB6KW0QA==", 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.4", 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQD/hAZm59SYOxAA6ZlyViyEh4T2LzLugojRlw==", 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.5", 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQADhQZmX0pCAxAAJ3tggpnioWiMCdGoDitpwg==", 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.188 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.6", 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQAGhQZmA85NBhAAeK3A7HRf5p03ImHioYPkxw==", 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.189 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "osd.7", 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQAJhQZmt9t7CRAAikERd7yLpKlPADqOxYTe4Q==", 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow profile osd", 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile osd", 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.0", 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZm8QvmKhAAtjcMwjNBLPSYOuxJAXZEnw==", 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.190 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow r", 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rw pool=cephfs_data" 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.1", 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmKIVfORAAYXf8+I1h3ctIq9tQRkNJMQ==", 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.191 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.2", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDshAZmvXccMhAALKLyc3Cfc3yjx3671P2yNw==", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.3", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDthAZmlzsGBRAA3lYwuTrdEqSvk/f3Zx4NSA==", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow r", 2024-03-29T09:11:40.192 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow rw", 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow rwx" 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "client.admin", 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZm0EgVBxAA/dalJuZ4gfwlxIjb36vqZQ==", 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "mgr": "allow *", 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow *", 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.x", 2024-03-29T09:11:40.193 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZm/LczNBAAakX4jUFzQMJdJNA14crh/A==", 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "entity": "mgr.y", 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "key": "AQDrhAZmLEdRLBAAnVR8ae1uZ/ezeRT77pc7dw==", 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": { 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": "allow *", 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "mon": "allow profile mgr", 2024-03-29T09:11:40.194 INFO:teuthology.orchestra.run.smithi099.stdout: "osd": "allow *" 2024-03-29T09:11:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:40.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-03-29T09:11:40.195 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f423806ac90 msgr2=0x7f423806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f423806ac90 0x7f423806d150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4248009cc0 tx=0x7f4248008040 comp rx=0 tx=0).stop 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425815ade0 msgr2=0x7f4258157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425815ade0 0x7f4258157a80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f424400f820 tx=0x7f424400fbe0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 shutdown_connections 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f423806ac90 0x7f423806d150 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258157fc0 0x7f425815ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f423c005210 0x7f42581558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 --2- 172.21.15.99:0/1690094055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f425815ade0 0x7f4258157a80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 >> 172.21.15.99:0/1690094055 conn(0x7f425809c490 msgr2=0x7f425806cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:40.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 shutdown_connections 2024-03-29T09:11:40.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.185+0000 7f425d800700 1 -- 172.21.15.99:0/1690094055 wait complete. 2024-03-29T09:11:40.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.391+0000 7f1313800700 1 Processor -- start 2024-03-29T09:11:40.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.391+0000 7f1313800700 1 -- start start 2024-03-29T09:11:40.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.392+0000 7f1313800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.392+0000 7f1313800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f130c0cdcb0 0x7f130c0ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.392+0000 7f1313800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c0d1be0 con 0x7f130c0cfd70 2024-03-29T09:11:40.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.392+0000 7f1313800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0b4980 con 0x7f130c0cdcb0 2024-03-29T09:11:40.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.392+0000 7f1313800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0b46a0 con 0x7f130c0d0150 2024-03-29T09:11:40.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f130be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f1311800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f130c0cdcb0 0x7f130c0ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f1310e00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f130c0cfd70 0x7f130c1615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:41052/0 (socket says 172.21.15.99:41052) 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f130be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c161cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:58100/0 (socket says 172.21.15.99:58100) 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f1310e00700 1 -- 172.21.15.99:0/3696219369 learned_addr learned my addr 172.21.15.99:0/3696219369 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f130b400700 1 -- 172.21.15.99:0/3696219369 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858463315 0 0) 0x7f130c0d1be0 con 0x7f130c0cfd70 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f130be00700 1 -- 172.21.15.99:0/3696219369 >> v2:172.21.15.133:3300/0 conn(0x7f130c0cdcb0 msgr2=0x7f130c0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f130be00700 1 --2- 172.21.15.99:0/3696219369 >> v2:172.21.15.133:3300/0 conn(0x7f130c0cdcb0 0x7f130c0ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f130be00700 1 -- 172.21.15.99:0/3696219369 >> v1:172.21.15.133:6789/0 conn(0x7f130c0cfd70 legacy=0x7f130c1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.393+0000 7f130be00700 1 -- 172.21.15.99:0/3696219369 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0d1590 con 0x7f130c0d0150 2024-03-29T09:11:40.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.394+0000 7f130be00700 1 --2- 172.21.15.99:0/3696219369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c161cb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009840 tx=0x7f12f8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b17f2f3363edc2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.394+0000 7f130b400700 1 -- 172.21.15.99:0/3696219369 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8011830 con 0x7f130c0d0150 2024-03-29T09:11:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.394+0000 7f130b400700 1 -- 172.21.15.99:0/3696219369 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f8011990 con 0x7f130c0d0150 2024-03-29T09:11:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.394+0000 7f130b400700 1 -- 172.21.15.99:0/3696219369 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8011d20 con 0x7f130c0d0150 2024-03-29T09:11:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.394+0000 7f1313800700 1 -- 172.21.15.99:0/3696219369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 msgr2=0x7f130c161cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.394 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.395+0000 7f1313800700 1 --2- 172.21.15.99:0/3696219369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c161cb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009840 tx=0x7f12f8009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.395+0000 7f1313800700 1 -- 172.21.15.99:0/3696219369 shutdown_connections 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.395+0000 7f1313800700 1 --2- 172.21.15.99:0/3696219369 >> v2:172.21.15.133:3300/0 conn(0x7f130c0cdcb0 0x7f130c0ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.395+0000 7f1313800700 1 --2- 172.21.15.99:0/3696219369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c161cb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.395+0000 7f1313800700 1 -- 172.21.15.99:0/3696219369 >> 172.21.15.99:0/3696219369 conn(0x7f130c09c4a0 msgr2=0x7f130c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.395+0000 7f1313800700 1 -- 172.21.15.99:0/3696219369 shutdown_connections 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.395+0000 7f1313800700 1 -- 172.21.15.99:0/3696219369 wait complete. 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.396+0000 7f1313800700 1 Processor -- start 2024-03-29T09:11:40.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.397+0000 7f1313800700 1 -- start start 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.397+0000 7f1313800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f130c0cdcb0 0x7f130c0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1313800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f130c0cfd70 0x7f130c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1313800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1313800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0cc890 con 0x7f130c0cfd70 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1313800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0ccb70 con 0x7f130c0cdcb0 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1313800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0cce50 con 0x7f130c0d0150 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f130c0cfd70 0x7f130c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f130c0cfd70 0x7f130c159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53640/0 (socket says 172.21.15.99:53640) 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 -- 172.21.15.99:0/405249323 learned_addr learned my addr 172.21.15.99:0/405249323 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1311800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1310e00700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f130c0cdcb0 0x7f130c0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 -- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 msgr2=0x7f130c1597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c1597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 -- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f130c0cdcb0 msgr2=0x7f130c0cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f130c0cdcb0 0x7f130c0cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f130be00700 1 -- 172.21.15.99:0/405249323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f80094a0 con 0x7f130c0cfd70 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.398+0000 7f1311800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.399+0000 7f130be00700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f130c0cfd70 0x7f130c159290 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f12f8014040 tx=0x7f12f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.399+0000 7f1308c00700 1 -- 172.21.15.99:0/405249323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f80038a0 con 0x7f130c0cfd70 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.399+0000 7f1308c00700 1 -- 172.21.15.99:0/405249323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f801f070 con 0x7f130c0cfd70 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.399+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f130c0cd130 con 0x7f130c0cfd70 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.399+0000 7f1308c00700 1 -- 172.21.15.99:0/405249323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f8003c00 con 0x7f130c0cfd70 2024-03-29T09:11:40.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.399+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c0b8060 con 0x7f130c0cfd70 2024-03-29T09:11:40.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.400+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c0d2050 con 0x7f130c0cfd70 2024-03-29T09:11:40.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.401+0000 7f1308c00700 1 -- 172.21.15.99:0/405249323 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f12f801a430 con 0x7f130c0cfd70 2024-03-29T09:11:40.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.402+0000 7f1308c00700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12ec06ac90 0x7f12ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.400 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.402+0000 7f1308c00700 1 -- 172.21.15.99:0/405249323 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f12f8021070 con 0x7f130c0cfd70 2024-03-29T09:11:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.402+0000 7f1310e00700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12ec06ac90 0x7f12ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.401 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.403+0000 7f1310e00700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12ec06ac90 0x7f12ec06d150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1300000f60 tx=0x7f1300007be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:40.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.406+0000 7f1308c00700 1 -- 172.21.15.99:0/405249323 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f130c0d2050 con 0x7f130c0cfd70 2024-03-29T09:11:40.612 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.613+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f130c0cd650 con 0x7f130c0cfd70 2024-03-29T09:11:40.613 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.614+0000 7f1308c00700 1 -- 172.21.15.99:0/405249323 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+12594 (secure 0 0 0) 0x7f130c0cd650 con 0x7f130c0cfd70 2024-03-29T09:11:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:40.613 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":51,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:39.664069+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12ec06ac90 msgr2=0x7f12ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12ec06ac90 0x7f12ec06d150 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1300000f60 tx=0x7f1300007be0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f130c0cfd70 msgr2=0x7f130c159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f130c0cfd70 0x7f130c159290 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f12f8014040 tx=0x7f12f8004750 comp rx=0 tx=0).stop 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 shutdown_connections 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12ec06ac90 0x7f12ec06d150 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f130c0d0150 0x7f130c1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f130c0cdcb0 0x7f130c0cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.616 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 --2- 172.21.15.99:0/405249323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f130c0cfd70 0x7f130c159290 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 >> 172.21.15.99:0/405249323 conn(0x7f130c09c4a0 msgr2=0x7f130c15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 shutdown_connections 2024-03-29T09:11:40.617 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.617+0000 7f1313800700 1 -- 172.21.15.99:0/405249323 wait complete. 2024-03-29T09:11:40.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:11:40.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:11:40.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.826+0000 7f85d5800700 1 Processor -- start 2024-03-29T09:11:40.825 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.826+0000 7f85d5800700 1 -- start start 2024-03-29T09:11:40.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.827+0000 7f85d5800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f85d00b3cc0 0x7f85d00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.827 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.827+0000 7f85d5800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d00d3810 con 0x7f85d0160770 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.827+0000 7f85d5800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d00d1520 con 0x7f85d00d3380 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.827+0000 7f85d5800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d00b4140 con 0x7f85d00b3cc0 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.827+0000 7f85cea00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f85d0160770 0x7f85d00d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59084/0 (socket says 172.21.15.99:59084) 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85cea00700 1 -- 172.21.15.99:0/4673655 learned_addr learned my addr 172.21.15.99:0/4673655 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85ce000700 1 --2- 172.21.15.99:0/4673655 >> v2:172.21.15.133:3300/0 conn(0x7f85d00b3cc0 0x7f85d00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85cd600700 1 -- 172.21.15.99:0/4673655 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077645282 0 0) 0x7f85d00d3810 con 0x7f85d0160770 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85cd600700 1 -- 172.21.15.99:0/4673655 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85b40035b0 con 0x7f85d0160770 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85ce000700 1 -- 172.21.15.99:0/4673655 >> v1:172.21.15.133:6789/0 conn(0x7f85d00d3380 legacy=0x7f85d00ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85ce000700 1 -- 172.21.15.99:0/4673655 >> v1:172.21.15.99:6789/0 conn(0x7f85d0160770 legacy=0x7f85d00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85ce000700 1 -- 172.21.15.99:0/4673655 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00d1800 con 0x7f85d00b3cc0 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.828+0000 7f85ce000700 1 --2- 172.21.15.99:0/4673655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 0x7f85d00d2e40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009840 tx=0x7f85bc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa0a8b43ac8457e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85cd600700 1 -- 172.21.15.99:0/4673655 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc011830 con 0x7f85d00b3cc0 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85cd600700 1 -- 172.21.15.99:0/4673655 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc011990 con 0x7f85d00b3cc0 2024-03-29T09:11:40.828 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85cd600700 1 -- 172.21.15.99:0/4673655 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc011d20 con 0x7f85d00b3cc0 2024-03-29T09:11:40.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85d5800700 1 -- 172.21.15.99:0/4673655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 msgr2=0x7f85d00d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85d5800700 1 --2- 172.21.15.99:0/4673655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 0x7f85d00d2e40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009840 tx=0x7f85bc009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:40.829 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85d5800700 1 -- 172.21.15.99:0/4673655 shutdown_connections 2024-03-29T09:11:40.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85d5800700 1 --2- 172.21.15.99:0/4673655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 0x7f85d00d2e40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85d5800700 1 -- 172.21.15.99:0/4673655 >> 172.21.15.99:0/4673655 conn(0x7f85d008afd0 msgr2=0x7f85d009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:40.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85d5800700 1 -- 172.21.15.99:0/4673655 shutdown_connections 2024-03-29T09:11:40.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.829+0000 7f85d5800700 1 -- 172.21.15.99:0/4673655 wait complete. 2024-03-29T09:11:40.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.831+0000 7f85d5800700 1 Processor -- start 2024-03-29T09:11:40.830 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.831+0000 7f85d5800700 1 -- start start 2024-03-29T09:11:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85d5800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 0x7f85d00b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85d5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d00d3380 0x7f85d01f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85d5800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d0160770 0x7f85d00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85d5800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00cb2b0 con 0x7f85d00d3380 2024-03-29T09:11:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85d5800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d01f64e0 con 0x7f85d00b3cc0 2024-03-29T09:11:40.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85d5800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00cb550 con 0x7f85d0160770 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85ce000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d00d3380 0x7f85d01f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85ce000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d00d3380 0x7f85d01f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53642/0 (socket says 172.21.15.99:53642) 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.832+0000 7f85ce000700 1 -- 172.21.15.99:0/1748216655 learned_addr learned my addr 172.21.15.99:0/1748216655 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85cea00700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 0x7f85d00b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85cf400700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d0160770 0x7f85d00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85ce000700 1 -- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d0160770 msgr2=0x7f85d00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85ce000700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d0160770 0x7f85d00b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85ce000700 1 -- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 msgr2=0x7f85d00b8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85ce000700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 0x7f85d00b8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85ce000700 1 -- 172.21.15.99:0/1748216655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc0094a0 con 0x7f85d00d3380 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85ce000700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d00d3380 0x7f85d01f5fa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0090b0 tx=0x7f85bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85c6a00700 1 -- 172.21.15.99:0/1748216655 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0036a0 con 0x7f85d00d3380 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85c6a00700 1 -- 172.21.15.99:0/1748216655 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc01f070 con 0x7f85d00d3380 2024-03-29T09:11:40.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d01f6710 con 0x7f85d00d3380 2024-03-29T09:11:40.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.833+0000 7f85c6a00700 1 -- 172.21.15.99:0/1748216655 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc003a40 con 0x7f85d00d3380 2024-03-29T09:11:40.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.834+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d01f6cd0 con 0x7f85d00d3380 2024-03-29T09:11:40.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.834+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d00b90a0 con 0x7f85d00d3380 2024-03-29T09:11:40.834 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.835+0000 7f85c6a00700 1 -- 172.21.15.99:0/1748216655 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f85bc003ba0 con 0x7f85d00d3380 2024-03-29T09:11:40.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.836+0000 7f85c6a00700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f85b006ac90 0x7f85b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:40.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.836+0000 7f85c6a00700 1 -- 172.21.15.99:0/1748216655 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f85bc021070 con 0x7f85d00d3380 2024-03-29T09:11:40.835 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.836+0000 7f85cea00700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f85b006ac90 0x7f85b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:40.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.837+0000 7f85cea00700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f85b006ac90 0x7f85b006d150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007fd0 tx=0x7f85c0007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:40.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:40.841+0000 7f85c6a00700 1 -- 172.21.15.99:0/1748216655 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f85bc055e80 con 0x7f85d00d3380 2024-03-29T09:11:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.042+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85d01f6e60 con 0x7f85d00d3380 2024-03-29T09:11:41.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.043+0000 7f85c6a00700 1 -- 172.21.15.99:0/1748216655 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85bc0150a0 con 0x7f85d00d3380 2024-03-29T09:11:41.042 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:11:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f85b006ac90 msgr2=0x7f85b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f85b006ac90 0x7f85b006d150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007fd0 tx=0x7f85c0007dc0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d00d3380 msgr2=0x7f85d01f5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d00d3380 0x7f85d01f5fa0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0090b0 tx=0x7f85bc004750 comp rx=0 tx=0).stop 2024-03-29T09:11:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 shutdown_connections 2024-03-29T09:11:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f85b006ac90 0x7f85b006d150 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.044 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d0160770 0x7f85d00b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d00b3cc0 0x7f85d00b8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 --2- 172.21.15.99:0/1748216655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f85d00d3380 0x7f85d01f5fa0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 >> 172.21.15.99:0/1748216655 conn(0x7f85d008afd0 msgr2=0x7f85d015f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 shutdown_connections 2024-03-29T09:11:41.045 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.045+0000 7f85d5800700 1 -- 172.21.15.99:0/1748216655 wait complete. 2024-03-29T09:11:41.062 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T09:11:41.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.254+0000 7f2fc4600700 1 Processor -- start 2024-03-29T09:11:41.253 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.254+0000 7f2fc4600700 1 -- start start 2024-03-29T09:11:41.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.255+0000 7f2fc4600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2fbc0d1e50 0x7f2fbc0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.255+0000 7f2fc4600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0afde0 con 0x7f2fbc15d690 2024-03-29T09:11:41.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc4600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0d01d0 con 0x7f2fbc15da70 2024-03-29T09:11:41.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc4600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc0d04b0 con 0x7f2fbc0d1e50 2024-03-29T09:11:41.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc2600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f2fbc0d1e50 0x7f2fbc0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc1c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2fbc15d690 0x7f2fbc0cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59090/0 (socket says 172.21.15.99:59090) 2024-03-29T09:11:41.255 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc1c00700 1 -- 172.21.15.99:0/3912218335 learned_addr learned my addr 172.21.15.99:0/3912218335 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc2600700 1 -- 172.21.15.99:0/3912218335 >> v1:172.21.15.133:6789/0 conn(0x7f2fbc15da70 legacy=0x7f2fbc0d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc2600700 1 -- 172.21.15.99:0/3912218335 >> v1:172.21.15.99:6789/0 conn(0x7f2fbc15d690 legacy=0x7f2fbc0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fc2600700 1 -- 172.21.15.99:0/3912218335 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0b0030 con 0x7f2fbc0d1e50 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.256+0000 7f2fbbe00700 1 -- 172.21.15.99:0/3912218335 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384778398 0 0) 0x7f2fbc0afde0 con 0x7f2fbc15d690 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fc2600700 1 --2- 172.21.15.99:0/3912218335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cfb60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009d90 tx=0x7f2fb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be57f058057ecf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fbbe00700 1 -- 172.21.15.99:0/3912218335 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400e070 con 0x7f2fbc0d1e50 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fbbe00700 1 -- 172.21.15.99:0/3912218335 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb400bbd0 con 0x7f2fbc0d1e50 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fbbe00700 1 -- 172.21.15.99:0/3912218335 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4012430 con 0x7f2fbc0d1e50 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fc4600700 1 -- 172.21.15.99:0/3912218335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc0d1e50 msgr2=0x7f2fbc0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fc4600700 1 --2- 172.21.15.99:0/3912218335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cfb60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4009d90 tx=0x7f2fb400b330 comp rx=0 tx=0).stop 2024-03-29T09:11:41.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fc4600700 1 -- 172.21.15.99:0/3912218335 shutdown_connections 2024-03-29T09:11:41.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fc4600700 1 --2- 172.21.15.99:0/3912218335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cfb60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.257+0000 7f2fc4600700 1 -- 172.21.15.99:0/3912218335 >> 172.21.15.99:0/3912218335 conn(0x7f2fbc09c480 msgr2=0x7f2fbc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:41.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.258+0000 7f2fc4600700 1 -- 172.21.15.99:0/3912218335 shutdown_connections 2024-03-29T09:11:41.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.258+0000 7f2fc4600700 1 -- 172.21.15.99:0/3912218335 wait complete. 2024-03-29T09:11:41.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.259+0000 7f2fc4600700 1 Processor -- start 2024-03-29T09:11:41.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.259+0000 7f2fc4600700 1 -- start start 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.260+0000 7f2fc4600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.260+0000 7f2fc4600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2fbc15d690 0x7f2fbc1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.260+0000 7f2fc4600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc15da70 0x7f2fbc1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.260+0000 7f2fc4600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0cc5d0 con 0x7f2fbc15da70 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.260+0000 7f2fc4600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc1f3cf0 con 0x7f2fbc0d1e50 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.260+0000 7f2fc4600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc160f00 con 0x7f2fbc15d690 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.260+0000 7f2fc2600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc15da70 0x7f2fbc1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc15da70 0x7f2fbc1609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53658/0 (socket says 172.21.15.99:53658) 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 -- 172.21.15.99:0/2357893096 learned_addr learned my addr 172.21.15.99:0/2357893096 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc1200700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2fbc15d690 0x7f2fbc1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc1c00700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 -- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2fbc15d690 msgr2=0x7f2fbc1f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2fbc15d690 0x7f2fbc1f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 -- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fbc0d1e50 msgr2=0x7f2fbc0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 -- 172.21.15.99:0/2357893096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb4009a70 con 0x7f2fbc15da70 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc1c00700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc2600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc15da70 0x7f2fbc1609c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4014040 tx=0x7f2fb4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fb9600700 1 -- 172.21.15.99:0/2357893096 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb401b810 con 0x7f2fbc15da70 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fb9600700 1 -- 172.21.15.99:0/2357893096 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4020070 con 0x7f2fbc15da70 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc0cb950 con 0x7f2fbc15da70 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.261+0000 7f2fb9600700 1 -- 172.21.15.99:0/2357893096 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb401bbb0 con 0x7f2fbc15da70 2024-03-29T09:11:41.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.262+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc0cbf10 con 0x7f2fbc15da70 2024-03-29T09:11:41.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.263+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc0cc710 con 0x7f2fbc15da70 2024-03-29T09:11:41.263 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.263+0000 7f2fb9600700 1 -- 172.21.15.99:0/2357893096 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2fb401bd10 con 0x7f2fbc15da70 2024-03-29T09:11:41.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.265+0000 7f2fb9600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2f9c06ac90 0x7f2f9c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.265+0000 7f2fb9600700 1 -- 172.21.15.99:0/2357893096 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2fb4022070 con 0x7f2fbc15da70 2024-03-29T09:11:41.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.265+0000 7f2fc1c00700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2f9c06ac90 0x7f2f9c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.264 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.265+0000 7f2fc1c00700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2f9c06ac90 0x7f2f9c06d150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0000f60 tx=0x7f2fb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:41.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.269+0000 7f2fb9600700 1 -- 172.21.15.99:0/2357893096 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2fbc0cc710 con 0x7f2fbc15da70 2024-03-29T09:11:41.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.476+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2fbc0cc710 con 0x7f2fbc15da70 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.477+0000 7f2fb9600700 1 -- 172.21.15.99:0/2357893096 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2fbc0cc710 con 0x7f2fbc15da70 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stdout:[ 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stdout: "poolnum": 1, 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stdout: "poolname": ".mgr" 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:41.476 INFO:teuthology.orchestra.run.smithi099.stdout:] 2024-03-29T09:11:41.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2f9c06ac90 msgr2=0x7f2f9c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2f9c06ac90 0x7f2f9c06d150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0000f60 tx=0x7f2fb0009040 comp rx=0 tx=0).stop 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc15da70 msgr2=0x7f2fbc1609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc15da70 0x7f2fbc1609c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4014040 tx=0x7f2fb4004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 shutdown_connections 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2f9c06ac90 0x7f2f9c06d150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2fbc15da70 0x7f2fbc1609c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fbc0d1e50 0x7f2fbc0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 --2- 172.21.15.99:0/2357893096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2fbc15d690 0x7f2fbc1f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 >> 172.21.15.99:0/2357893096 conn(0x7f2fbc09c480 msgr2=0x7f2fbc0b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 shutdown_connections 2024-03-29T09:11:41.479 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.479+0000 7f2fc4600700 1 -- 172.21.15.99:0/2357893096 wait complete. 2024-03-29T09:11:41.497 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T09:11:41.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T09:11:41.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.693+0000 7fb493400700 1 Processor -- start 2024-03-29T09:11:41.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.694+0000 7fb493400700 1 -- start start 2024-03-29T09:11:41.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb493400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 0x7fb48c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb493400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0b3e80 con 0x7fb48c160a30 2024-03-29T09:11:41.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb493400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c0d1320 con 0x7fb48c0d1e00 2024-03-29T09:11:41.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb493400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b4160 con 0x7fb48c0b43d0 2024-03-29T09:11:41.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb490a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb48c160a30 0x7fb48c0b29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59094/0 (socket says 172.21.15.99:59094) 2024-03-29T09:11:41.694 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb490a00700 1 -- 172.21.15.99:0/2604649962 learned_addr learned my addr 172.21.15.99:0/2604649962 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:41.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb48be00700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 0x7fb48c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556427957 0 0) 0x7fb48c0b3e80 con 0x7fb48c160a30 2024-03-29T09:11:41.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.695+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4740035b0 con 0x7fb48c160a30 2024-03-29T09:11:41.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112047700 0 0) 0x7fb48c0d1320 con 0x7fb48c0d1e00 2024-03-29T09:11:41.697 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb48c0b3e80 con 0x7fb48c0d1e00 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3197073167 0 0) 0x7fb4740035b0 con 0x7fb48c160a30 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 msgr2=0x7fb48c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 0x7fb48c0b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 >> v1:172.21.15.133:6789/0 conn(0x7fb48c0d1e00 legacy=0x7fb48c0b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0d1600 con 0x7fb48c160a30 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fb480001670 con 0x7fb48c160a30 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 >> v1:172.21.15.99:6789/0 conn(0x7fb48c160a30 legacy=0x7fb48c0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 0x7fb474003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474004120 0x7fb474004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004d90 0x7fb474005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b4160 con 0x7fb474004120 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47400bba0 con 0x7fb474004d90 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47400bd30 con 0x7fb48c0b43d0 2024-03-29T09:11:41.698 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb490a00700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 0x7fb474003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48be00700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474004120 0x7fb474004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb491400700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004d90 0x7fb474005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48be00700 1 -- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 msgr2=0x7fb474003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48be00700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0b43d0 0x7fb474003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48be00700 1 -- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004d90 msgr2=0x7fb474005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48be00700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004d90 0x7fb474005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.696+0000 7fb48be00700 1 -- 172.21.15.99:0/2604649962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740035b0 con 0x7fb474004120 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb48be00700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474004120 0x7fb474004850 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb478009b60 tx=0x7fb474005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d97b2d18e64f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800e6c0 con 0x7fb474004120 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47800e820 con 0x7fb474004120 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb48b400700 1 -- 172.21.15.99:0/2604649962 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47800eb90 con 0x7fb474004120 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb493400700 1 -- 172.21.15.99:0/2604649962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474004120 msgr2=0x7fb474004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb493400700 1 --2- 172.21.15.99:0/2604649962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474004120 0x7fb474004850 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb478009b60 tx=0x7fb474005880 comp rx=0 tx=0).stop 2024-03-29T09:11:41.699 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb491400700 1 -- 172.21.15.99:0/2604649962 reap_dead start 2024-03-29T09:11:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb493400700 1 -- 172.21.15.99:0/2604649962 shutdown_connections 2024-03-29T09:11:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.697+0000 7fb493400700 1 -- 172.21.15.99:0/2604649962 >> 172.21.15.99:0/2604649962 conn(0x7fb48c09c4e0 msgr2=0x7fb48c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.698+0000 7fb493400700 1 -- 172.21.15.99:0/2604649962 shutdown_connections 2024-03-29T09:11:41.700 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.698+0000 7fb493400700 1 -- 172.21.15.99:0/2604649962 wait complete. 2024-03-29T09:11:41.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.699+0000 7fb493400700 1 Processor -- start 2024-03-29T09:11:41.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.699+0000 7fb493400700 1 -- start start 2024-03-29T09:11:41.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.700+0000 7fb493400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004120 0x7fb48c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.700+0000 7fb493400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0c9590 0x7fb48c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.700+0000 7fb493400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c0b15e0 0x7fb48c0b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.702 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.700+0000 7fb493400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0c9a10 con 0x7fb48c0b15e0 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.700+0000 7fb493400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b1fa0 con 0x7fb474004120 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.700+0000 7fb493400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b6570 con 0x7fb48c0c9590 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.700+0000 7fb491400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c0b15e0 0x7fb48c0b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c0b15e0 0x7fb48c0b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53664/0 (socket says 172.21.15.99:53664) 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 -- 172.21.15.99:0/351793357 learned_addr learned my addr 172.21.15.99:0/351793357 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb490a00700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004120 0x7fb48c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb48be00700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0c9590 0x7fb48c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 -- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0c9590 msgr2=0x7fb48c152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0c9590 0x7fb48c152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 -- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004120 msgr2=0x7fb48c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004120 0x7fb48c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 -- 172.21.15.99:0/351793357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c0cb440 con 0x7fb48c0b15e0 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb491400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c0b15e0 0x7fb48c0b1a60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0cda30 tx=0x7fb48400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:41.703 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb488c00700 1 -- 172.21.15.99:0/351793357 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484003c10 con 0x7fb48c0b15e0 2024-03-29T09:11:41.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb488c00700 1 -- 172.21.15.99:0/351793357 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb484003d70 con 0x7fb48c0b15e0 2024-03-29T09:11:41.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c0cbc50 con 0x7fb48c0b15e0 2024-03-29T09:11:41.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.701+0000 7fb488c00700 1 -- 172.21.15.99:0/351793357 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400db20 con 0x7fb48c0b15e0 2024-03-29T09:11:41.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.702+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c0caba0 con 0x7fb48c0b15e0 2024-03-29T09:11:41.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.703+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c0b66b0 con 0x7fb48c0b15e0 2024-03-29T09:11:41.704 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.703+0000 7fb488c00700 1 -- 172.21.15.99:0/351793357 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb48400dc80 con 0x7fb48c0b15e0 2024-03-29T09:11:41.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.705+0000 7fb488c00700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb46c06acc0 0x7fb46c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:41.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.705+0000 7fb488c00700 1 -- 172.21.15.99:0/351793357 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb484089310 con 0x7fb48c0b15e0 2024-03-29T09:11:41.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.705+0000 7fb490a00700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb46c06acc0 0x7fb46c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:41.705 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.705+0000 7fb490a00700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb46c06acc0 0x7fb46c06d180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb480001e20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:41.708 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.709+0000 7fb488c00700 1 -- 172.21.15.99:0/351793357 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb484054720 con 0x7fb48c0b15e0 2024-03-29T09:11:41.914 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:41.912+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb48c0678b0 con 0x7fb48c0b15e0 2024-03-29T09:11:42.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.150+0000 7fb488c00700 1 -- 172.21.15.99:0/351793357 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb484057d40 con 0x7fb48c0b15e0 2024-03-29T09:11:42.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb46c06acc0 msgr2=0x7fb46c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:42.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb46c06acc0 0x7fb46c06d180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb480000c00 tx=0x7fb480001e20 comp rx=0 tx=0).stop 2024-03-29T09:11:42.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c0b15e0 msgr2=0x7fb48c0b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:42.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c0b15e0 0x7fb48c0b1a60 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0cda30 tx=0x7fb48400d430 comp rx=0 tx=0).stop 2024-03-29T09:11:42.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 shutdown_connections 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb46c06acc0 0x7fb46c06d180 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb48c0b15e0 0x7fb48c0b1a60 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004120 0x7fb48c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 --2- 172.21.15.99:0/351793357 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb48c0c9590 0x7fb48c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 >> 172.21.15.99:0/351793357 conn(0x7fb48c09c4e0 msgr2=0x7fb48c0b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 shutdown_connections 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.153+0000 7fb493400700 1 -- 172.21.15.99:0/351793357 wait complete. 2024-03-29T09:11:42.153 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' created 2024-03-29T09:11:42.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T09:11:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.374+0000 7fa488800700 1 Processor -- start 2024-03-29T09:11:42.373 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.374+0000 7fa488800700 1 -- start start 2024-03-29T09:11:42.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.375+0000 7fa488800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa48015d420 0x7fa4800d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:42.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.375+0000 7fa488800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4800b3e00 con 0x7fa4800d2eb0 2024-03-29T09:11:42.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.375+0000 7fa488800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4800b31f0 con 0x7fa4800cddb0 2024-03-29T09:11:42.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.375+0000 7fa488800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4800d12b0 con 0x7fa48015d420 2024-03-29T09:11:42.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.375+0000 7fa485e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa48015d420 0x7fa4800d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:42.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.375+0000 7fa485400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa4800d2eb0 0x7fa4800d3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59098/0 (socket says 172.21.15.99:59098) 2024-03-29T09:11:42.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.375+0000 7fa485400700 1 -- 172.21.15.99:0/3866835383 learned_addr learned my addr 172.21.15.99:0/3866835383 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:42.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa484a00700 1 -- 172.21.15.99:0/3866835383 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657104525 0 0) 0x7fa4800b3e00 con 0x7fa4800d2eb0 2024-03-29T09:11:42.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa485e00700 1 -- 172.21.15.99:0/3866835383 >> v1:172.21.15.133:6789/0 conn(0x7fa4800cddb0 legacy=0x7fa4800ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:42.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa485e00700 1 -- 172.21.15.99:0/3866835383 >> v1:172.21.15.99:6789/0 conn(0x7fa4800d2eb0 legacy=0x7fa4800d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:42.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa485e00700 1 -- 172.21.15.99:0/3866835383 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800b34d0 con 0x7fa48015d420 2024-03-29T09:11:42.376 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa485e00700 1 --2- 172.21.15.99:0/3866835383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa48015d420 0x7fa4800d0d70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa4740097c0 tx=0x7fa474009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b0e16efe8061be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa484a00700 1 -- 172.21.15.99:0/3866835383 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474004230 con 0x7fa48015d420 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa484a00700 1 -- 172.21.15.99:0/3866835383 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa474004390 con 0x7fa48015d420 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.376+0000 7fa484a00700 1 -- 172.21.15.99:0/3866835383 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740036a0 con 0x7fa48015d420 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.377+0000 7fa488800700 1 -- 172.21.15.99:0/3866835383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa48015d420 msgr2=0x7fa4800d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.377+0000 7fa488800700 1 --2- 172.21.15.99:0/3866835383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa48015d420 0x7fa4800d0d70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa4740097c0 tx=0x7fa474009ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.377+0000 7fa488800700 1 -- 172.21.15.99:0/3866835383 shutdown_connections 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.377+0000 7fa488800700 1 --2- 172.21.15.99:0/3866835383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa48015d420 0x7fa4800d0d70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.377+0000 7fa488800700 1 -- 172.21.15.99:0/3866835383 >> 172.21.15.99:0/3866835383 conn(0x7fa48008afd0 msgr2=0x7fa48009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.377+0000 7fa488800700 1 -- 172.21.15.99:0/3866835383 shutdown_connections 2024-03-29T09:11:42.377 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.377+0000 7fa488800700 1 -- 172.21.15.99:0/3866835383 wait complete. 2024-03-29T09:11:42.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.379+0000 7fa488800700 1 Processor -- start 2024-03-29T09:11:42.378 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.379+0000 7fa488800700 1 -- start start 2024-03-29T09:11:42.380 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa488800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4800cddb0 0x7fa48015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa488800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa4800d2eb0 0x7fa48014c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa488800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48015d420 0x7fa48014cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa488800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800c9670 con 0x7fa4800cddb0 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa488800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800c8a60 con 0x7fa48015d420 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa488800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800c8d40 con 0x7fa4800d2eb0 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4800cddb0 0x7fa48015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4800cddb0 0x7fa48015d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53692/0 (socket says 172.21.15.99:53692) 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 -- 172.21.15.99:0/911496056 learned_addr learned my addr 172.21.15.99:0/911496056 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa486800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48015d420 0x7fa48014cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 -- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa4800d2eb0 msgr2=0x7fa48014c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485400700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa4800d2eb0 0x7fa48014c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa4800d2eb0 0x7fa48014c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 -- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48015d420 msgr2=0x7fa48014cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:42.381 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48015d420 0x7fa48014cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485e00700 1 -- 172.21.15.99:0/911496056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740094a0 con 0x7fa4800cddb0 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa486800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48015d420 0x7fa48014cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.380+0000 7fa485400700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa4800d2eb0 0x7fa48014c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.381+0000 7fa485e00700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4800cddb0 0x7fa48015d800 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa4740090b0 tx=0x7fa474004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.381+0000 7fa47a000700 1 -- 172.21.15.99:0/911496056 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa474011630 con 0x7fa4800cddb0 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.381+0000 7fa47a000700 1 -- 172.21.15.99:0/911496056 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47401f070 con 0x7fa4800cddb0 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.381+0000 7fa47a000700 1 -- 172.21.15.99:0/911496056 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740119f0 con 0x7fa4800cddb0 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.381+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4800c9020 con 0x7fa4800cddb0 2024-03-29T09:11:42.382 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.381+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4800c86e0 con 0x7fa4800cddb0 2024-03-29T09:11:42.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.382+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4800678b0 con 0x7fa4800cddb0 2024-03-29T09:11:42.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.383+0000 7fa47a000700 1 -- 172.21.15.99:0/911496056 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa474011b50 con 0x7fa4800cddb0 2024-03-29T09:11:42.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.384+0000 7fa47a000700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa46406ac90 0x7fa46406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:42.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.384+0000 7fa47a000700 1 -- 172.21.15.99:0/911496056 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fa474021070 con 0x7fa4800cddb0 2024-03-29T09:11:42.383 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.384+0000 7fa485400700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa46406ac90 0x7fa46406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:42.384 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.385+0000 7fa485400700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa46406ac90 0x7fa46406d150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa48015ae60 tx=0x7fa470009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:42.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.389+0000 7fa47a000700 1 -- 172.21.15.99:0/911496056 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa4740560d0 con 0x7fa4800cddb0 2024-03-29T09:11:42.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:42.593+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa4800d2150 con 0x7fa4800cddb0 2024-03-29T09:11:43.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.156+0000 7fa47a000700 1 -- 172.21.15.99:0/911496056 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa47402b020 con 0x7fa4800cddb0 2024-03-29T09:11:43.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.162+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa46406ac90 msgr2=0x7fa46406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.162+0000 7fa488800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa46406ac90 0x7fa46406d150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa48015ae60 tx=0x7fa470009040 comp rx=0 tx=0).stop 2024-03-29T09:11:43.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.162+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4800cddb0 msgr2=0x7fa48015d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.162+0000 7fa488800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4800cddb0 0x7fa48015d800 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa4740090b0 tx=0x7fa474004750 comp rx=0 tx=0).stop 2024-03-29T09:11:43.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.163+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 shutdown_connections 2024-03-29T09:11:43.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.163+0000 7fa488800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa46406ac90 0x7fa46406d150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.163+0000 7fa488800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48015d420 0x7fa48014cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.163+0000 7fa488800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4800cddb0 0x7fa48015d800 secure :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa4740090b0 tx=0x7fa474004750 comp rx=0 tx=0).stop 2024-03-29T09:11:43.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.163+0000 7fa488800700 1 --2- 172.21.15.99:0/911496056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa4800d2eb0 0x7fa48014c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.164 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.163+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 >> 172.21.15.99:0/911496056 conn(0x7fa48008afd0 msgr2=0x7fa48015c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:43.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.166+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 shutdown_connections 2024-03-29T09:11:43.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.167+0000 7fa488800700 1 -- 172.21.15.99:0/911496056 wait complete. 2024-03-29T09:11:43.172 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' created 2024-03-29T09:11:43.195 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T09:11:43.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.389+0000 7f6530000700 1 Processor -- start 2024-03-29T09:11:43.388 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.389+0000 7f6530000700 1 -- start start 2024-03-29T09:11:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.390+0000 7f6530000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f65280b3cc0 0x7f65280d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.390+0000 7f6530000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f65280d3380 0x7f65280ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.390+0000 7f6530000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65280d3810 con 0x7f6528160770 2024-03-29T09:11:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f6530000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65280b4140 con 0x7f65280d3380 2024-03-29T09:11:43.390 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f6530000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65280d1520 con 0x7f65280b3cc0 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652e000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f65280d3380 0x7f65280ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652d600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6528160770 0x7f65280d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59100/0 (socket says 172.21.15.99:59100) 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652d600700 1 -- 172.21.15.99:0/498954429 learned_addr learned my addr 172.21.15.99:0/498954429 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652cc00700 1 --2- 172.21.15.99:0/498954429 >> v2:172.21.15.133:3300/0 conn(0x7f65280b3cc0 0x7f65280d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652e000700 1 -- 172.21.15.99:0/498954429 >> v2:172.21.15.133:3300/0 conn(0x7f65280b3cc0 msgr2=0x7f65280d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652e000700 1 --2- 172.21.15.99:0/498954429 >> v2:172.21.15.133:3300/0 conn(0x7f65280b3cc0 0x7f65280d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652e000700 1 -- 172.21.15.99:0/498954429 >> v1:172.21.15.99:6789/0 conn(0x7f6528160770 legacy=0x7f65280d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652e000700 1 -- 172.21.15.99:0/498954429 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65280d1800 con 0x7f65280d3380 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f6527e00700 1 -- 172.21.15.99:0/498954429 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347402900 0 0) 0x7f65280d3810 con 0x7f6528160770 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.391+0000 7f652cc00700 1 --2- 172.21.15.99:0/498954429 >> v2:172.21.15.133:3300/0 conn(0x7f65280b3cc0 0x7f65280d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:43.391 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f652e000700 1 --2- 172.21.15.99:0/498954429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65280ce080 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f65280b5cf0 tx=0x7f652000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6db450c2fba75ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:43.392 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6527e00700 1 -- 172.21.15.99:0/498954429 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652000e070 con 0x7f65280d3380 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6527e00700 1 -- 172.21.15.99:0/498954429 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000bbd0 con 0x7f65280d3380 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6527e00700 1 -- 172.21.15.99:0/498954429 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520012430 con 0x7f65280d3380 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6530000700 1 -- 172.21.15.99:0/498954429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 msgr2=0x7f65280ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6530000700 1 --2- 172.21.15.99:0/498954429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65280ce080 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f65280b5cf0 tx=0x7f652000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6530000700 1 -- 172.21.15.99:0/498954429 shutdown_connections 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6530000700 1 --2- 172.21.15.99:0/498954429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65280ce080 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6530000700 1 --2- 172.21.15.99:0/498954429 >> v2:172.21.15.133:3300/0 conn(0x7f65280b3cc0 0x7f65280d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6530000700 1 -- 172.21.15.99:0/498954429 >> 172.21.15.99:0/498954429 conn(0x7f652808afd0 msgr2=0x7f652809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.392+0000 7f6530000700 1 -- 172.21.15.99:0/498954429 shutdown_connections 2024-03-29T09:11:43.393 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.393+0000 7f6530000700 1 -- 172.21.15.99:0/498954429 wait complete. 2024-03-29T09:11:43.395 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.394+0000 7f6530000700 1 Processor -- start 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.394+0000 7f6530000700 1 -- start start 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.395+0000 7f6530000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65280b3cc0 0x7f652815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.395+0000 7f6530000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65281516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.395+0000 7f6530000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6528160770 0x7f65280ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.395+0000 7f6530000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652809cfe0 con 0x7f65280d3380 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.395+0000 7f6530000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652809d2c0 con 0x7f6528160770 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.395+0000 7f6530000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652809d5a0 con 0x7f65280b3cc0 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.395+0000 7f652cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65281516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65281516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53702/0 (socket says 172.21.15.99:53702) 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 -- 172.21.15.99:0/2855248106 learned_addr learned my addr 172.21.15.99:0/2855248106 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652e000700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6528160770 0x7f65280ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652d600700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65280b3cc0 0x7f652815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 -- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65280b3cc0 msgr2=0x7f652815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65280b3cc0 0x7f652815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 -- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6528160770 msgr2=0x7f65280ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6528160770 0x7f65280ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 -- 172.21.15.99:0/2855248106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6520009a70 con 0x7f65280d3380 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f652cc00700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65281516a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f651400c750 tx=0x7f651400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f6525600700 1 -- 172.21.15.99:0/2855248106 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514011800 con 0x7f65280d3380 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f6525600700 1 -- 172.21.15.99:0/2855248106 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6514011960 con 0x7f65280d3380 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652809d880 con 0x7f65280d3380 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.396+0000 7f6525600700 1 -- 172.21.15.99:0/2855248106 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514011cd0 con 0x7f65280d3380 2024-03-29T09:11:43.397 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.397+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6528151be0 con 0x7f65280d3380 2024-03-29T09:11:43.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.397+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65280678b0 con 0x7f65280d3380 2024-03-29T09:11:43.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.399+0000 7f6525600700 1 -- 172.21.15.99:0/2855248106 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6514021350 con 0x7f65280d3380 2024-03-29T09:11:43.398 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.399+0000 7f6525600700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650806ac90 0x7f650806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.399+0000 7f6525600700 1 -- 172.21.15.99:0/2855248106 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f65140893c0 con 0x7f65280d3380 2024-03-29T09:11:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.400+0000 7f652d600700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650806ac90 0x7f650806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.399 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.400+0000 7f652d600700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650806ac90 0x7f650806d150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f651c000f60 tx=0x7f651c007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:43.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.404+0000 7f6525600700 1 -- 172.21.15.99:0/2855248106 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f65140544c0 con 0x7f65280d3380 2024-03-29T09:11:43.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.651+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f65280d0f70 con 0x7f65280d3380 2024-03-29T09:11:43.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.659+0000 7f6525600700 1 -- 172.21.15.99:0/2855248106 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6514057ae0 con 0x7f65280d3380 2024-03-29T09:11:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.665+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650806ac90 msgr2=0x7f650806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.665+0000 7f6530000700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650806ac90 0x7f650806d150 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f651c000f60 tx=0x7f651c007c90 comp rx=0 tx=0).stop 2024-03-29T09:11:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.665+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 msgr2=0x7f65281516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.665+0000 7f6530000700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65281516a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f651400c750 tx=0x7f651400cb10 comp rx=0 tx=0).stop 2024-03-29T09:11:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.668+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 shutdown_connections 2024-03-29T09:11:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.668+0000 7f6530000700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f650806ac90 0x7f650806d150 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.668+0000 7f6530000700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6528160770 0x7f65280ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.668+0000 7f6530000700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f65280b3cc0 0x7f652815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.668+0000 7f6530000700 1 --2- 172.21.15.99:0/2855248106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65280d3380 0x7f65281516a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.668+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 >> 172.21.15.99:0/2855248106 conn(0x7f652808afd0 msgr2=0x7f652806cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:43.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.674+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 shutdown_connections 2024-03-29T09:11:43.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.675+0000 7f6530000700 1 -- 172.21.15.99:0/2855248106 wait complete. 2024-03-29T09:11:43.683 INFO:teuthology.orchestra.run.smithi099.stderr:new fs with metadata pool 8 and data pool 9 2024-03-29T09:11:43.704 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:43.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.895+0000 7fce5b600700 1 Processor -- start 2024-03-29T09:11:43.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.895+0000 7fce5b600700 1 -- start start 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce5b600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce541608e0 0x7fce540d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce5b600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 0x7fce540ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce5b600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540cddc0 con 0x7fce540d3a70 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce5b600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce540ce0a0 con 0x7fce541608e0 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce5b600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540ce380 con 0x7fce54160cc0 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce58c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fce541608e0 0x7fce540d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce59600700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fce540d3a70 0x7fce540d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59114/0 (socket says 172.21.15.99:59114) 2024-03-29T09:11:43.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce59600700 1 -- 172.21.15.99:0/2751923241 learned_addr learned my addr 172.21.15.99:0/2751923241 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:43.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.896+0000 7fce53e00700 1 --2- 172.21.15.99:0/2751923241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 0x7fce540ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce58c00700 1 -- 172.21.15.99:0/2751923241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 msgr2=0x7fce540ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:43.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce58c00700 1 --2- 172.21.15.99:0/2751923241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 0x7fce540ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce58c00700 1 -- 172.21.15.99:0/2751923241 >> v1:172.21.15.99:6789/0 conn(0x7fce540d3a70 legacy=0x7fce540d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce58c00700 1 -- 172.21.15.99:0/2751923241 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce540ce660 con 0x7fce541608e0 2024-03-29T09:11:43.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce53400700 1 -- 172.21.15.99:0/2751923241 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441047152 0 0) 0x7fce540cddc0 con 0x7fce540d3a70 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce58c00700 1 --2- 172.21.15.99:0/2751923241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce541608e0 0x7fce540d0150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fce48009840 tx=0x7fce48009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82ae72890b344e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce53400700 1 -- 172.21.15.99:0/2751923241 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48011830 con 0x7fce541608e0 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce53400700 1 -- 172.21.15.99:0/2751923241 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce48011990 con 0x7fce541608e0 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.897+0000 7fce53400700 1 -- 172.21.15.99:0/2751923241 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48011d20 con 0x7fce541608e0 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 -- 172.21.15.99:0/2751923241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce541608e0 msgr2=0x7fce540d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 --2- 172.21.15.99:0/2751923241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce541608e0 0x7fce540d0150 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fce48009840 tx=0x7fce48009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 -- 172.21.15.99:0/2751923241 shutdown_connections 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 --2- 172.21.15.99:0/2751923241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce541608e0 0x7fce540d0150 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 --2- 172.21.15.99:0/2751923241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 0x7fce540ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 -- 172.21.15.99:0/2751923241 >> 172.21.15.99:0/2751923241 conn(0x7fce5409c490 msgr2=0x7fce540d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 -- 172.21.15.99:0/2751923241 shutdown_connections 2024-03-29T09:11:43.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.898+0000 7fce5b600700 1 -- 172.21.15.99:0/2751923241 wait complete. 2024-03-29T09:11:43.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.900+0000 7fce5b600700 1 Processor -- start 2024-03-29T09:11:43.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.900+0000 7fce5b600700 1 -- start start 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce5b600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce540d3a70 0x7fce540cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce5b600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce541608e0 0x7fce540c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce5b600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 0x7fce54148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce5b600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540cd630 con 0x7fce540d3a70 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce5b600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540cc9b0 con 0x7fce541608e0 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce5b600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce540ccc90 con 0x7fce54160cc0 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce540d3a70 0x7fce540cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce540d3a70 0x7fce540cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53716/0 (socket says 172.21.15.99:53716) 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 -- 172.21.15.99:0/3469800689 learned_addr learned my addr 172.21.15.99:0/3469800689 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 -- 172.21.15.99:0/3469800689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 msgr2=0x7fce54148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 0x7fce54148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 -- 172.21.15.99:0/3469800689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce541608e0 msgr2=0x7fce540c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce541608e0 0x7fce540c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.901+0000 7fce58c00700 1 -- 172.21.15.99:0/3469800689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480094a0 con 0x7fce540d3a70 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.902+0000 7fce58c00700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce540d3a70 0x7fce540cbfb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fce48009b90 tx=0x7fce48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.902+0000 7fce50c00700 1 -- 172.21.15.99:0/3469800689 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480036a0 con 0x7fce540d3a70 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.902+0000 7fce50c00700 1 -- 172.21.15.99:0/3469800689 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4801f070 con 0x7fce540d3a70 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.902+0000 7fce50c00700 1 -- 172.21.15.99:0/3469800689 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48003a40 con 0x7fce540d3a70 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.902+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce540ccf70 con 0x7fce540d3a70 2024-03-29T09:11:43.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.902+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce540cc4f0 con 0x7fce540d3a70 2024-03-29T09:11:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.903+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54157560 con 0x7fce540d3a70 2024-03-29T09:11:43.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.904+0000 7fce50c00700 1 -- 172.21.15.99:0/3469800689 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fce48003ba0 con 0x7fce540d3a70 2024-03-29T09:11:43.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.905+0000 7fce50c00700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fce3406ac90 0x7fce3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:43.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.905+0000 7fce50c00700 1 -- 172.21.15.99:0/3469800689 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fce48021070 con 0x7fce540d3a70 2024-03-29T09:11:43.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.905+0000 7fce53e00700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fce3406ac90 0x7fce3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:43.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.906+0000 7fce53e00700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fce3406ac90 0x7fce3406d150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce5414c700 tx=0x7fce40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:43.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:43.910+0000 7fce50c00700 1 -- 172.21.15.99:0/3469800689 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fce54157560 con 0x7fce540d3a70 2024-03-29T09:11:44.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.112+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce54157560 con 0x7fce540d3a70 2024-03-29T09:11:44.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.114+0000 7fce50c00700 1 -- 172.21.15.99:0/3469800689 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16315 (secure 0 0 0) 0x7fce54157560 con 0x7fce540d3a70 2024-03-29T09:11:44.113 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:44.113 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":54,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:43.652242+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:44.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fce3406ac90 msgr2=0x7fce3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fce3406ac90 0x7fce3406d150 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce5414c700 tx=0x7fce40008040 comp rx=0 tx=0).stop 2024-03-29T09:11:44.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce540d3a70 msgr2=0x7fce540cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce540d3a70 0x7fce540cbfb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fce48009b90 tx=0x7fce48004750 comp rx=0 tx=0).stop 2024-03-29T09:11:44.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 shutdown_connections 2024-03-29T09:11:44.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fce3406ac90 0x7fce3406d150 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce54160cc0 0x7fce54148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce540d3a70 0x7fce540cbfb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 --2- 172.21.15.99:0/3469800689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce541608e0 0x7fce540c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 >> 172.21.15.99:0/3469800689 conn(0x7fce5409c490 msgr2=0x7fce5406c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:44.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 shutdown_connections 2024-03-29T09:11:44.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.116+0000 7fce5b600700 1 -- 172.21.15.99:0/3469800689 wait complete. 2024-03-29T09:11:44.134 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.324+0000 7f4b98600700 1 Processor -- start 2024-03-29T09:11:44.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.324+0000 7f4b98600700 1 -- start start 2024-03-29T09:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.325+0000 7f4b98600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4b901608e0 0x7f4b900d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.325+0000 7f4b98600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b900ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.325+0000 7f4b98600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b900cddc0 con 0x7f4b90160cc0 2024-03-29T09:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.325+0000 7f4b98600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b900ce0a0 con 0x7f4b901608e0 2024-03-29T09:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.325+0000 7f4b98600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900ce380 con 0x7f4b900d1b20 2024-03-29T09:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.325+0000 7f4b95200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4b90160cc0 0x7f4b900d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59118/0 (socket says 172.21.15.99:59118) 2024-03-29T09:11:44.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.325+0000 7f4b95200700 1 -- 172.21.15.99:0/2875333135 learned_addr learned my addr 172.21.15.99:0/2875333135 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b96600700 1 --2- 172.21.15.99:0/2875333135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b900ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b95c00700 1 --2- 172.21.15.99:0/2875333135 >> v2:172.21.15.133:3300/0 conn(0x7f4b901608e0 0x7f4b900d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b8fe00700 1 -- 172.21.15.99:0/2875333135 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186477051 0 0) 0x7f4b900cddc0 con 0x7f4b90160cc0 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b8fe00700 1 -- 172.21.15.99:0/2875333135 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b780035b0 con 0x7f4b90160cc0 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b96600700 1 -- 172.21.15.99:0/2875333135 >> v2:172.21.15.133:3300/0 conn(0x7f4b901608e0 msgr2=0x7f4b900d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b96600700 1 --2- 172.21.15.99:0/2875333135 >> v2:172.21.15.133:3300/0 conn(0x7f4b901608e0 0x7f4b900d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b96600700 1 -- 172.21.15.99:0/2875333135 >> v1:172.21.15.99:6789/0 conn(0x7f4b90160cc0 legacy=0x7f4b900d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b8fe00700 1 -- 172.21.15.99:0/2875333135 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1692483466 0 0) 0x7f4b780035b0 con 0x7f4b90160cc0 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b96600700 1 -- 172.21.15.99:0/2875333135 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900ce660 con 0x7f4b900d1b20 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b95c00700 1 --2- 172.21.15.99:0/2875333135 >> v2:172.21.15.133:3300/0 conn(0x7f4b901608e0 0x7f4b900d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.326+0000 7f4b96600700 1 --2- 172.21.15.99:0/2875333135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b900ceb80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009d90 tx=0x7f4b8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=916c811e8f724f07 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b8fe00700 1 -- 172.21.15.99:0/2875333135 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8800e070 con 0x7f4b900d1b20 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b8fe00700 1 -- 172.21.15.99:0/2875333135 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b88003a20 con 0x7f4b900d1b20 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b8fe00700 1 -- 172.21.15.99:0/2875333135 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c001670 con 0x7f4b900d1b20 2024-03-29T09:11:44.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b98600700 1 -- 172.21.15.99:0/2875333135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 msgr2=0x7f4b900ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b98600700 1 --2- 172.21.15.99:0/2875333135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b900ceb80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009d90 tx=0x7f4b8800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b98600700 1 -- 172.21.15.99:0/2875333135 shutdown_connections 2024-03-29T09:11:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b98600700 1 --2- 172.21.15.99:0/2875333135 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b900ceb80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b98600700 1 --2- 172.21.15.99:0/2875333135 >> v2:172.21.15.133:3300/0 conn(0x7f4b901608e0 0x7f4b900d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.327+0000 7f4b98600700 1 -- 172.21.15.99:0/2875333135 >> 172.21.15.99:0/2875333135 conn(0x7f4b9009c490 msgr2=0x7f4b900d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.328+0000 7f4b98600700 1 -- 172.21.15.99:0/2875333135 shutdown_connections 2024-03-29T09:11:44.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.328+0000 7f4b98600700 1 -- 172.21.15.99:0/2875333135 wait complete. 2024-03-29T09:11:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.329+0000 7f4b98600700 1 Processor -- start 2024-03-29T09:11:44.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.329+0000 7f4b98600700 1 -- start start 2024-03-29T09:11:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.330+0000 7f4b98600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b9015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.330+0000 7f4b98600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b901608e0 0x7f4b900b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.330+0000 7f4b98600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b90160cc0 0x7f4b900d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.330+0000 7f4b98600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900b5510 con 0x7f4b901608e0 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.330+0000 7f4b98600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900b57f0 con 0x7f4b90160cc0 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.330+0000 7f4b98600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b900b5ad0 con 0x7f4b900d1b20 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b901608e0 0x7f4b900b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b901608e0 0x7f4b900b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53720/0 (socket says 172.21.15.99:53720) 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 -- 172.21.15.99:0/3716823881 learned_addr learned my addr 172.21.15.99:0/3716823881 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95c00700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b9015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b96600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b90160cc0 0x7f4b900d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 -- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 msgr2=0x7f4b9015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b9015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 -- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b90160cc0 msgr2=0x7f4b900d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b90160cc0 0x7f4b900d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 -- 172.21.15.99:0/3716823881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840094a0 con 0x7f4b901608e0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.331+0000 7f4b95200700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b901608e0 0x7f4b900b4ba0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c750 tx=0x7f4b7c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.332+0000 7f4b8d600700 1 -- 172.21.15.99:0/3716823881 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c004c80 con 0x7f4b901608e0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.332+0000 7f4b8d600700 1 -- 172.21.15.99:0/3716823881 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c004de0 con 0x7f4b901608e0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.332+0000 7f4b8d600700 1 -- 172.21.15.99:0/3716823881 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0072d0 con 0x7f4b901608e0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.332+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b88009a70 con 0x7f4b901608e0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.332+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b900b6090 con 0x7f4b901608e0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.333+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b901f3020 con 0x7f4b901608e0 2024-03-29T09:11:44.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.334+0000 7f4b8d600700 1 -- 172.21.15.99:0/3716823881 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4b7c007430 con 0x7f4b901608e0 2024-03-29T09:11:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.334+0000 7f4b8d600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4b7006ac90 0x7f4b7006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.335+0000 7f4b8d600700 1 -- 172.21.15.99:0/3716823881 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4b7c0891a0 con 0x7f4b901608e0 2024-03-29T09:11:44.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.335+0000 7f4b95c00700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4b7006ac90 0x7f4b7006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.335+0000 7f4b95c00700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4b7006ac90 0x7f4b7006d150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009d70 tx=0x7f4b84009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:44.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.339+0000 7f4b8d600700 1 -- 172.21.15.99:0/3716823881 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4b901f3020 con 0x7f4b901608e0 2024-03-29T09:11:44.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.546+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b900677a0 con 0x7f4b901608e0 2024-03-29T09:11:44.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.547+0000 7f4b8d600700 1 -- 172.21.15.99:0/3716823881 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16315 (secure 0 0 0) 0x7f4b7c0541a0 con 0x7f4b901608e0 2024-03-29T09:11:44.546 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:44.546 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":54,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:43.652242+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4b7006ac90 msgr2=0x7f4b7006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4b7006ac90 0x7f4b7006d150 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009d70 tx=0x7f4b84009210 comp rx=0 tx=0).stop 2024-03-29T09:11:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b901608e0 msgr2=0x7f4b900b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b901608e0 0x7f4b900b4ba0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c750 tx=0x7f4b7c00ca60 comp rx=0 tx=0).stop 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 shutdown_connections 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4b7006ac90 0x7f4b7006d150 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b90160cc0 0x7f4b900d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b900d1b20 0x7f4b9015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 --2- 172.21.15.99:0/3716823881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b901608e0 0x7f4b900b4ba0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 >> 172.21.15.99:0/3716823881 conn(0x7f4b9009c490 msgr2=0x7f4b9015c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 shutdown_connections 2024-03-29T09:11:44.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.550+0000 7f4b98600700 1 -- 172.21.15.99:0/3716823881 wait complete. 2024-03-29T09:11:44.568 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T09:11:44.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.764+0000 7f9cf5000700 1 Processor -- start 2024-03-29T09:11:44.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.764+0000 7f9cf5000700 1 -- start start 2024-03-29T09:11:44.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.765+0000 7f9cf5000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006dda0 0x7f9cf00b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.765+0000 7f9cf5000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9cf015a240 0x7f9cf015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.765+0000 7f9cf5000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf00ac990 con 0x7f9cf0159e60 2024-03-29T09:11:44.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.765+0000 7f9cf5000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf015e230 con 0x7f9cf015a240 2024-03-29T09:11:44.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.765+0000 7f9cf5000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf015e510 con 0x7f9cf006dda0 2024-03-29T09:11:44.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.765+0000 7f9ced400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006dda0 0x7f9cf00b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.765+0000 7f9ced400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006dda0 0x7f9cf00b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:58204/0 (socket says 172.21.15.99:58204) 2024-03-29T09:11:44.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.766+0000 7f9ced400700 1 -- 172.21.15.99:0/3768904679 learned_addr learned my addr 172.21.15.99:0/3768904679 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.766+0000 7f9cee800700 1 --2- 172.21.15.99:0/3768904679 >> v2:172.21.15.133:3300/0 conn(0x7f9cf015a240 0x7f9cf015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.766+0000 7f9ced400700 1 -- 172.21.15.99:0/3768904679 >> v2:172.21.15.133:3300/0 conn(0x7f9cf015a240 msgr2=0x7f9cf015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.766+0000 7f9ced400700 1 --2- 172.21.15.99:0/3768904679 >> v2:172.21.15.133:3300/0 conn(0x7f9cf015a240 0x7f9cf015dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.766+0000 7f9ced400700 1 -- 172.21.15.99:0/3768904679 >> v1:172.21.15.133:6789/0 conn(0x7f9cf0159e60 legacy=0x7f9cf006d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.766+0000 7f9ced400700 1 -- 172.21.15.99:0/3768904679 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf015e7f0 con 0x7f9cf006dda0 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.766+0000 7f9ced400700 1 --2- 172.21.15.99:0/3768904679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006dda0 0x7f9cf00b2b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0098f0 tx=0x7f9cdc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee430a072dd0e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9ceca00700 1 -- 172.21.15.99:0/3768904679 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc011830 con 0x7f9cf006dda0 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9ceca00700 1 -- 172.21.15.99:0/3768904679 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc011990 con 0x7f9cf006dda0 2024-03-29T09:11:44.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9ceca00700 1 -- 172.21.15.99:0/3768904679 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc011d20 con 0x7f9cf006dda0 2024-03-29T09:11:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 -- 172.21.15.99:0/3768904679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006dda0 msgr2=0x7f9cf00b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 --2- 172.21.15.99:0/3768904679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006dda0 0x7f9cf00b2b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0098f0 tx=0x7f9cdc009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 -- 172.21.15.99:0/3768904679 shutdown_connections 2024-03-29T09:11:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 --2- 172.21.15.99:0/3768904679 >> v2:172.21.15.133:3300/0 conn(0x7f9cf015a240 0x7f9cf015dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 --2- 172.21.15.99:0/3768904679 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf006dda0 0x7f9cf00b2b60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 -- 172.21.15.99:0/3768904679 >> 172.21.15.99:0/3768904679 conn(0x7f9cf0066540 msgr2=0x7f9cf0066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 -- 172.21.15.99:0/3768904679 shutdown_connections 2024-03-29T09:11:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.767+0000 7f9cf5000700 1 -- 172.21.15.99:0/3768904679 wait complete. 2024-03-29T09:11:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.769+0000 7f9cf5000700 1 Processor -- start 2024-03-29T09:11:44.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.769+0000 7f9cf5000700 1 -- start start 2024-03-29T09:11:44.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.770+0000 7f9cf5000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf006dda0 0x7f9cf015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.770+0000 7f9cf5000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cf0159e60 0x7f9cf014bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.770+0000 7f9cf5000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf015a240 0x7f9cf009bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.770+0000 7f9cf5000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf014c090 con 0x7f9cf0159e60 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.770+0000 7f9cf5000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0156890 con 0x7f9cf006dda0 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.770+0000 7f9cf5000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0156b70 con 0x7f9cf015a240 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.770+0000 7f9ced400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cf0159e60 0x7f9cf014bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cf0159e60 0x7f9cf014bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53730/0 (socket says 172.21.15.99:53730) 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 -- 172.21.15.99:0/2955694766 learned_addr learned my addr 172.21.15.99:0/2955694766 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9cee800700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf015a240 0x7f9cf009bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9cede00700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf006dda0 0x7f9cf015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 -- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf015a240 msgr2=0x7f9cf009bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf015a240 0x7f9cf009bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 -- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf006dda0 msgr2=0x7f9cf015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf006dda0 0x7f9cf015a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 -- 172.21.15.99:0/2955694766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009550 con 0x7f9cf0159e60 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ced400700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cf0159e60 0x7f9cf014bb50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009c40 tx=0x7f9cdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:44.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ce6000700 1 -- 172.21.15.99:0/2955694766 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc004510 con 0x7f9cf0159e60 2024-03-29T09:11:44.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ce6000700 1 -- 172.21.15.99:0/2955694766 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc01e070 con 0x7f9cf0159e60 2024-03-29T09:11:44.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9ce6000700 1 -- 172.21.15.99:0/2955694766 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0038e0 con 0x7f9cf0159e60 2024-03-29T09:11:44.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.771+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf0156e50 con 0x7f9cf0159e60 2024-03-29T09:11:44.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.772+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf0156320 con 0x7f9cf0159e60 2024-03-29T09:11:44.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.773+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf00678b0 con 0x7f9cf0159e60 2024-03-29T09:11:44.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.773+0000 7f9ce6000700 1 -- 172.21.15.99:0/2955694766 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9cdc003a40 con 0x7f9cf0159e60 2024-03-29T09:11:44.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.775+0000 7f9ce6000700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9cd006ac90 0x7f9cd006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:44.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.775+0000 7f9ce6000700 1 -- 172.21.15.99:0/2955694766 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9cdc08a2f0 con 0x7f9cf0159e60 2024-03-29T09:11:44.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.775+0000 7f9cede00700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9cd006ac90 0x7f9cd006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:44.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.775+0000 7f9cede00700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9cd006ac90 0x7f9cd006d150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0007fd0 tx=0x7f9ce0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:44.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:44.779+0000 7f9ce6000700 1 -- 172.21.15.99:0/2955694766 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9cdc0553a0 con 0x7f9cf0159e60 2024-03-29T09:11:45.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.029+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9cf00b3cf0 con 0x7f9cf0159e60 2024-03-29T09:11:45.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.664+0000 7f9ce6000700 1 -- 172.21.15.99:0/2955694766 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9cdc0589c0 con 0x7f9cf0159e60 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.666+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9cd006ac90 msgr2=0x7f9cd006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.666+0000 7f9cf5000700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9cd006ac90 0x7f9cd006d150 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0007fd0 tx=0x7f9ce0009040 comp rx=0 tx=0).stop 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.666+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cf0159e60 msgr2=0x7f9cf014bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.666+0000 7f9cf5000700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cf0159e60 0x7f9cf014bb50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009c40 tx=0x7f9cdc004750 comp rx=0 tx=0).stop 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 shutdown_connections 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9cd006ac90 0x7f9cd006d150 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf015a240 0x7f9cf009bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf006dda0 0x7f9cf015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 --2- 172.21.15.99:0/2955694766 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9cf0159e60 0x7f9cf014bb50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 >> 172.21.15.99:0/2955694766 conn(0x7f9cf0066540 msgr2=0x7f9cf00b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 shutdown_connections 2024-03-29T09:11:45.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.667+0000 7f9cf5000700 1 -- 172.21.15.99:0/2955694766 wait complete. 2024-03-29T09:11:45.685 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T09:11:45.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T09:11:45.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.879+0000 7f4889600700 1 Processor -- start 2024-03-29T09:11:45.878 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.879+0000 7f4889600700 1 -- start start 2024-03-29T09:11:45.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.880+0000 7f4889600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 0x7f4884167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.880+0000 7f4889600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48840d1520 con 0x7f48840b3cc0 2024-03-29T09:11:45.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.880+0000 7f4889600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48840d1800 con 0x7f4884160770 2024-03-29T09:11:45.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.880+0000 7f4889600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b4140 con 0x7f48840cf090 2024-03-29T09:11:45.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4882000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f48840b3cc0 0x7f48840d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:59124/0 (socket says 172.21.15.99:59124) 2024-03-29T09:11:45.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4882000700 1 -- 172.21.15.99:0/3109747530 learned_addr learned my addr 172.21.15.99:0/3109747530 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4883400700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 0x7f4884167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716076594 0 0) 0x7f48840d1520 con 0x7f48840b3cc0 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48680035b0 con 0x7f48840b3cc0 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791721065 0 0) 0x7f48840d1800 con 0x7f4884160770 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48840d1520 con 0x7f4884160770 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2271897201 0 0) 0x7f48680035b0 con 0x7f48840b3cc0 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 msgr2=0x7f4884167f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 0x7f4884167f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 >> v1:172.21.15.133:6789/0 conn(0x7f4884160770 legacy=0x7f48840d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840d1ae0 con 0x7f48840b3cc0 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f4870001670 con 0x7f48840b3cc0 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 >> v1:172.21.15.99:6789/0 conn(0x7f48840b3cc0 legacy=0x7f48840d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.881+0000 7f4881600700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868003bc0 0x7f4868003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4881600700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 0x7f4868004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4881600700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4868004d90 0x7f4868005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486800bac0 con 0x7f4868003bc0 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486800bc00 con 0x7f4868004d90 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486800bdc0 con 0x7f48840cf090 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882a00700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868003bc0 0x7f4868003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882000700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 0x7f4868004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4883400700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4868004d90 0x7f4868005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882a00700 1 -- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 msgr2=0x7f4868004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882a00700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840cf090 0x7f4868004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882a00700 1 -- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4868004d90 msgr2=0x7f4868005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882a00700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4868004d90 0x7f4868005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882a00700 1 -- 172.21.15.99:0/3109747530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680035b0 con 0x7f4868003bc0 2024-03-29T09:11:45.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.882+0000 7f4882a00700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868003bc0 0x7f4868003fc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4874001940 tx=0x7f4868005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd132b50f6b7c78c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874001430 con 0x7f4868003bc0 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874001d70 con 0x7f4868003bc0 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4881600700 1 -- 172.21.15.99:0/3109747530 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874005510 con 0x7f4868003bc0 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4889600700 1 -- 172.21.15.99:0/3109747530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868003bc0 msgr2=0x7f4868003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4889600700 1 --2- 172.21.15.99:0/3109747530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868003bc0 0x7f4868003fc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4874001940 tx=0x7f4868005880 comp rx=0 tx=0).stop 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4883400700 1 -- 172.21.15.99:0/3109747530 reap_dead start 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4889600700 1 -- 172.21.15.99:0/3109747530 shutdown_connections 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4889600700 1 -- 172.21.15.99:0/3109747530 >> 172.21.15.99:0/3109747530 conn(0x7f488408afd0 msgr2=0x7f488409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4889600700 1 -- 172.21.15.99:0/3109747530 shutdown_connections 2024-03-29T09:11:45.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.883+0000 7f4889600700 1 -- 172.21.15.99:0/3109747530 wait complete. 2024-03-29T09:11:45.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.885+0000 7f4889600700 1 Processor -- start 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.885+0000 7f4889600700 1 -- start start 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4889600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868003bc0 0x7f4884152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4889600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4884152310 0x7f4884151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4889600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b11c0 0x7f48840b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4889600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840b1bb0 con 0x7f4884152310 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4889600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48841e6900 con 0x7f48840b11c0 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4889600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840cb460 con 0x7f4868003bc0 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4882000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4884152310 0x7f4884151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4882000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4884152310 0x7f4884151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53744/0 (socket says 172.21.15.99:53744) 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.886+0000 7f4882000700 1 -- 172.21.15.99:0/2607496072 learned_addr learned my addr 172.21.15.99:0/2607496072 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4882a00700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868003bc0 0x7f4884152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4883400700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b11c0 0x7f48840b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4882000700 1 -- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868003bc0 msgr2=0x7f4884152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4882000700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868003bc0 0x7f4884152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4882000700 1 -- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b11c0 msgr2=0x7f48840b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4882000700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b11c0 0x7f48840b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4882000700 1 -- 172.21.15.99:0/2607496072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840cb740 con 0x7f4884152310 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4882000700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4884152310 0x7f4884151b60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4870007310 tx=0x7f4870007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f487aa00700 1 -- 172.21.15.99:0/2607496072 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870011580 con 0x7f4884152310 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f487aa00700 1 -- 172.21.15.99:0/2607496072 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48700116e0 con 0x7f4884152310 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48840cba20 con 0x7f4884152310 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.887+0000 7f487aa00700 1 -- 172.21.15.99:0/2607496072 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870011a50 con 0x7f4884152310 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.888+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48841e65b0 con 0x7f4884152310 2024-03-29T09:11:45.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.889+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48840678b0 con 0x7f4884152310 2024-03-29T09:11:45.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.889+0000 7f487aa00700 1 -- 172.21.15.99:0/2607496072 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4870011bb0 con 0x7f4884152310 2024-03-29T09:11:45.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.891+0000 7f487aa00700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f486406ac90 0x7f486406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:45.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.891+0000 7f487aa00700 1 -- 172.21.15.99:0/2607496072 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4870089460 con 0x7f4884152310 2024-03-29T09:11:45.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.891+0000 7f4882a00700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f486406ac90 0x7f486406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:45.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.891+0000 7f4882a00700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f486406ac90 0x7f486406d150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c00 tx=0x7f487400a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:45.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:45.895+0000 7f487aa00700 1 -- 172.21.15.99:0/2607496072 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4870054460 con 0x7f4884152310 2024-03-29T09:11:46.146 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.147+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4884074710 con 0x7f4884152310 2024-03-29T09:11:46.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.667+0000 7f487aa00700 1 -- 172.21.15.99:0/2607496072 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4870057a80 con 0x7f4884152310 2024-03-29T09:11:46.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f486406ac90 msgr2=0x7f486406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:46.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f486406ac90 0x7f486406d150 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4874000c00 tx=0x7f487400a000 comp rx=0 tx=0).stop 2024-03-29T09:11:46.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4884152310 msgr2=0x7f4884151b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:46.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4884152310 0x7f4884151b60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4870007310 tx=0x7f4870007620 comp rx=0 tx=0).stop 2024-03-29T09:11:46.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 shutdown_connections 2024-03-29T09:11:46.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f486406ac90 0x7f486406d150 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b11c0 0x7f48840b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868003bc0 0x7f4884152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 --2- 172.21.15.99:0/2607496072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4884152310 0x7f4884151b60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 >> 172.21.15.99:0/2607496072 conn(0x7f488408afd0 msgr2=0x7f48840b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:46.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 shutdown_connections 2024-03-29T09:11:46.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.670+0000 7f4889600700 1 -- 172.21.15.99:0/2607496072 wait complete. 2024-03-29T09:11:46.689 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T09:11:46.689 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:11:46.879 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.880+0000 7f23e7200700 1 Processor -- start 2024-03-29T09:11:46.880 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.881+0000 7f23e7200700 1 -- start start 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e7200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00d0150 0x7f23e0161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e7200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f23e00cdcb0 0x7f23e00ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e7200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00d1be0 con 0x7f23e00cfd70 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e7200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e00b4980 con 0x7f23e00cdcb0 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e7200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b46a0 con 0x7f23e00d0150 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e5200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f23e00cdcb0 0x7f23e00ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e5200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f23e00cdcb0 0x7f23e00ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53758/0 (socket says 172.21.15.99:53758) 2024-03-29T09:11:46.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23e5200700 1 -- 172.21.15.99:0/1165641502 learned_addr learned my addr 172.21.15.99:0/1165641502 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.882+0000 7f23df400700 1 --2- 172.21.15.99:0/1165641502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00d0150 0x7f23e0161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23e5200700 1 -- 172.21.15.99:0/1165641502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00d0150 msgr2=0x7f23e0161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23e5200700 1 --2- 172.21.15.99:0/1165641502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00d0150 0x7f23e0161cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23e5200700 1 -- 172.21.15.99:0/1165641502 >> v1:172.21.15.133:6789/0 conn(0x7f23e00cfd70 legacy=0x7f23e01615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23e5200700 1 -- 172.21.15.99:0/1165641502 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00d1590 con 0x7f23e00cdcb0 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23e5200700 1 --2- 172.21.15.99:0/1165641502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cdcb0 0x7f23e00ce140 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009e50 tx=0x7f23d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d889f91d35075c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23dea00700 1 -- 172.21.15.99:0/1165641502 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800e070 con 0x7f23e00cdcb0 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23dea00700 1 -- 172.21.15.99:0/1165641502 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800b9c0 con 0x7f23e00cdcb0 2024-03-29T09:11:46.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.883+0000 7f23dea00700 1 -- 172.21.15.99:0/1165641502 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800bd70 con 0x7f23e00cdcb0 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 -- 172.21.15.99:0/1165641502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cdcb0 msgr2=0x7f23e00ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 --2- 172.21.15.99:0/1165641502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cdcb0 0x7f23e00ce140 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009e50 tx=0x7f23d800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 -- 172.21.15.99:0/1165641502 shutdown_connections 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 --2- 172.21.15.99:0/1165641502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cdcb0 0x7f23e00ce140 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 --2- 172.21.15.99:0/1165641502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00d0150 0x7f23e0161cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 -- 172.21.15.99:0/1165641502 >> 172.21.15.99:0/1165641502 conn(0x7f23e009c4a0 msgr2=0x7f23e00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 -- 172.21.15.99:0/1165641502 shutdown_connections 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.884+0000 7f23e7200700 1 -- 172.21.15.99:0/1165641502 wait complete. 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.886+0000 7f23e7200700 1 Processor -- start 2024-03-29T09:11:46.885 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.886+0000 7f23e7200700 1 -- start start 2024-03-29T09:11:46.887 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.886+0000 7f23e7200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00cdcb0 0x7f23e014c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.886+0000 7f23e7200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cfd70 0x7f23e01f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23e7200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23e00d0150 0x7f23e00b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23e7200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e01f71f0 con 0x7f23e00cfd70 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23e7200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e01575e0 con 0x7f23e00d0150 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23e7200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0158ff0 con 0x7f23e00cdcb0 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cfd70 0x7f23e01f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cfd70 0x7f23e01f6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53764/0 (socket says 172.21.15.99:53764) 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 -- 172.21.15.99:0/4041917634 learned_addr learned my addr 172.21.15.99:0/4041917634 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23dfe00700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00cdcb0 0x7f23e014c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23e5200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23e00d0150 0x7f23e00b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 -- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00cdcb0 msgr2=0x7f23e014c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00cdcb0 0x7f23e014c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 -- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23e00d0150 msgr2=0x7f23e00b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23e00d0150 0x7f23e00b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 -- 172.21.15.99:0/4041917634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8009b30 con 0x7f23e00cfd70 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23dfe00700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00cdcb0 0x7f23e014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23e5200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23e00d0150 0x7f23e00b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.887+0000 7f23df400700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cfd70 0x7f23e01f6cb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d800 tx=0x7f23cc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.888+0000 7f23d7e00700 1 -- 172.21.15.99:0/4041917634 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0125f0 con 0x7f23e00cfd70 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.888+0000 7f23d7e00700 1 -- 172.21.15.99:0/4041917634 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc012750 con 0x7f23e00cfd70 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.888+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e01592d0 con 0x7f23e00cfd70 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.888+0000 7f23d7e00700 1 -- 172.21.15.99:0/4041917634 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc012b00 con 0x7f23e00cfd70 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.888+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e014d000 con 0x7f23e00cfd70 2024-03-29T09:11:46.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.889+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e00677a0 con 0x7f23e00cfd70 2024-03-29T09:11:46.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.890+0000 7f23d7e00700 1 -- 172.21.15.99:0/4041917634 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f23cc012c60 con 0x7f23e00cfd70 2024-03-29T09:11:46.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.890+0000 7f23d7e00700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23c006ac90 0x7f23c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:46.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.891+0000 7f23d7e00700 1 -- 172.21.15.99:0/4041917634 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f23cc0898b0 con 0x7f23e00cfd70 2024-03-29T09:11:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.891+0000 7f23dfe00700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23c006ac90 0x7f23c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:46.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.891+0000 7f23dfe00700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23c006ac90 0x7f23c006d150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000f60 tx=0x7f23d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:46.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:46.895+0000 7f23d7e00700 1 -- 172.21.15.99:0/4041917634 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f23cc0548b0 con 0x7f23e00cfd70 2024-03-29T09:11:47.139 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.140+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23e00b1460 con 0x7f23e00cfd70 2024-03-29T09:11:47.140 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.141+0000 7f23d7e00700 1 -- 172.21.15.99:0/4041917634 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f23cc057ed0 con 0x7f23e00cfd70 2024-03-29T09:11:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:47.140 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/790352630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":790352630},{"type":"v1","addr":"172.21.15.133:6833","nonce":790352630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1265449168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1265449168},{"type":"v1","addr":"172.21.15.133:6835","nonce":1265449168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6102,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1161439746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1161439746},{"type":"v1","addr":"172.21.15.99:6836","nonce":1161439746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:11:43.652592+0000","modified":"2024-03-29T09:11:46.662472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6105":{"gid":6105,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2396189132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2396189132},{"type":"v1","addr":"172.21.15.99:6837","nonce":2396189132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T09:11:47.142 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23c006ac90 msgr2=0x7f23c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23c006ac90 0x7f23c006d150 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000f60 tx=0x7f23d0009040 comp rx=0 tx=0).stop 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cfd70 msgr2=0x7f23e01f6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cfd70 0x7f23e01f6cb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d800 tx=0x7f23cc00dbc0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 shutdown_connections 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f23c006ac90 0x7f23c006d150 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23e00d0150 0x7f23e00b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23e00cdcb0 0x7f23e014c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 --2- 172.21.15.99:0/4041917634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23e00cfd70 0x7f23e01f6cb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 >> 172.21.15.99:0/4041917634 conn(0x7f23e009c4a0 msgr2=0x7f23e00afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 shutdown_connections 2024-03-29T09:11:47.143 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.143+0000 7f23e7200700 1 -- 172.21.15.99:0/4041917634 wait complete. 2024-03-29T09:11:47.144 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-03-29T09:11:47.161 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:47.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.350+0000 7f5cbe600700 1 Processor -- start 2024-03-29T09:11:47.350 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.351+0000 7f5cbe600700 1 -- start start 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cbe600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb81608e0 0x7f5cb80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cbe600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5cb8160cc0 0x7f5cb80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cbe600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5cb80d3a70 0x7f5cb80d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cbe600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb80cddc0 con 0x7f5cb80d3a70 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cbe600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb80ce0a0 con 0x7f5cb8160cc0 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cbe600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb80ce380 con 0x7f5cb81608e0 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5cb80d3a70 0x7f5cb80d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f5cb80d3a70 0x7f5cb80d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53780/0 (socket says 172.21.15.99:53780) 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 -- 172.21.15.99:0/1851737948 learned_addr learned my addr 172.21.15.99:0/1851737948 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7400700 1 --2- 172.21.15.99:0/1851737948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb81608e0 0x7f5cb80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.353 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb6a00700 1 --2- 172.21.15.99:0/1851737948 >> v2:172.21.15.133:3300/0 conn(0x7f5cb8160cc0 0x7f5cb80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 -- 172.21.15.99:0/1851737948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb81608e0 msgr2=0x7f5cb80d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 --2- 172.21.15.99:0/1851737948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb81608e0 0x7f5cb80d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 -- 172.21.15.99:0/1851737948 >> v2:172.21.15.133:3300/0 conn(0x7f5cb8160cc0 msgr2=0x7f5cb80ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 --2- 172.21.15.99:0/1851737948 >> v2:172.21.15.133:3300/0 conn(0x7f5cb8160cc0 0x7f5cb80ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb7e00700 1 -- 172.21.15.99:0/1851737948 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb80ce660 con 0x7f5cb80d3a70 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.352+0000 7f5cb6a00700 1 --2- 172.21.15.99:0/1851737948 >> v2:172.21.15.133:3300/0 conn(0x7f5cb8160cc0 0x7f5cb80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.353+0000 7f5cb7e00700 1 --2- 172.21.15.99:0/1851737948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb80d3a70 0x7f5cb80d1410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5cb808d120 tx=0x7f5cb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3a36abdcd19731 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.353+0000 7f5cb6000700 1 -- 172.21.15.99:0/1851737948 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000e070 con 0x7f5cb80d3a70 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.353+0000 7f5cb6000700 1 -- 172.21.15.99:0/1851737948 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb000b9c0 con 0x7f5cb80d3a70 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.353+0000 7f5cb6000700 1 -- 172.21.15.99:0/1851737948 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000bd70 con 0x7f5cb80d3a70 2024-03-29T09:11:47.354 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.353+0000 7f5cbe600700 1 -- 172.21.15.99:0/1851737948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb80d3a70 msgr2=0x7f5cb80d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.353+0000 7f5cbe600700 1 --2- 172.21.15.99:0/1851737948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb80d3a70 0x7f5cb80d1410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5cb808d120 tx=0x7f5cb000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.354+0000 7f5cbe600700 1 -- 172.21.15.99:0/1851737948 shutdown_connections 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.354+0000 7f5cbe600700 1 --2- 172.21.15.99:0/1851737948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb81608e0 0x7f5cb80d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.354+0000 7f5cbe600700 1 --2- 172.21.15.99:0/1851737948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb80d3a70 0x7f5cb80d1410 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.354+0000 7f5cbe600700 1 --2- 172.21.15.99:0/1851737948 >> v2:172.21.15.133:3300/0 conn(0x7f5cb8160cc0 0x7f5cb80ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.354+0000 7f5cbe600700 1 -- 172.21.15.99:0/1851737948 >> 172.21.15.99:0/1851737948 conn(0x7f5cb809c490 msgr2=0x7f5cb80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.354+0000 7f5cbe600700 1 -- 172.21.15.99:0/1851737948 shutdown_connections 2024-03-29T09:11:47.355 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.354+0000 7f5cbe600700 1 -- 172.21.15.99:0/1851737948 wait complete. 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.355+0000 7f5cbe600700 1 Processor -- start 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.355+0000 7f5cbe600700 1 -- start start 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.356+0000 7f5cbe600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb80d3a70 0x7f5cb80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.356+0000 7f5cbe600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb81608e0 0x7f5cb80cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.356+0000 7f5cbe600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb8160cc0 0x7f5cb80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.356+0000 7f5cbe600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb815b3d0 con 0x7f5cb8160cc0 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.356+0000 7f5cbe600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb81591c0 con 0x7f5cb81608e0 2024-03-29T09:11:47.358 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.356+0000 7f5cbe600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb8159470 con 0x7f5cb80d3a70 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb8160cc0 0x7f5cb80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb8160cc0 0x7f5cb80cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53784/0 (socket says 172.21.15.99:53784) 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 -- 172.21.15.99:0/3520942655 learned_addr learned my addr 172.21.15.99:0/3520942655 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7400700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb80d3a70 0x7f5cb80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 -- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb80d3a70 msgr2=0x7f5cb80cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb6a00700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb81608e0 0x7f5cb80cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb80d3a70 0x7f5cb80cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 -- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb81608e0 msgr2=0x7f5cb80cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb81608e0 0x7f5cb80cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 -- 172.21.15.99:0/3520942655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb0009b30 con 0x7f5cb8160cc0 2024-03-29T09:11:47.359 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7400700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb80d3a70 0x7f5cb80cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb6a00700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb81608e0 0x7f5cb80cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5cb7e00700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb8160cc0 0x7f5cb80cb790 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5cb808c9a0 tx=0x7f5cb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5caf400700 1 -- 172.21.15.99:0/3520942655 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0004250 con 0x7f5cb8160cc0 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5caf400700 1 -- 172.21.15.99:0/3520942655 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb00043d0 con 0x7f5cb8160cc0 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.357+0000 7f5caf400700 1 -- 172.21.15.99:0/3520942655 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0004780 con 0x7f5cb8160cc0 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.358+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb815b640 con 0x7f5cb8160cc0 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.358+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb8159750 con 0x7f5cb8160cc0 2024-03-29T09:11:47.360 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.358+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb815b230 con 0x7f5cb8160cc0 2024-03-29T09:11:47.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.359+0000 7f5caf400700 1 -- 172.21.15.99:0/3520942655 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5cb001c440 con 0x7f5cb8160cc0 2024-03-29T09:11:47.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.360+0000 7f5caf400700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c9806ac90 0x7f5c9806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.360+0000 7f5caf400700 1 -- 172.21.15.99:0/3520942655 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5cb008aa30 con 0x7f5cb8160cc0 2024-03-29T09:11:47.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.361+0000 7f5cb7400700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c9806ac90 0x7f5c9806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.361 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.361+0000 7f5cb7400700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c9806ac90 0x7f5c9806d150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8074100 tx=0x7f5ca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:47.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.365+0000 7f5caf400700 1 -- 172.21.15.99:0/3520942655 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5cb815b230 con 0x7f5cb8160cc0 2024-03-29T09:11:47.567 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.568+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cb815bb60 con 0x7f5cb8160cc0 2024-03-29T09:11:47.568 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.570+0000 7f5caf400700 1 -- 172.21.15.99:0/3520942655 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f5cb815bb60 con 0x7f5cb8160cc0 2024-03-29T09:11:47.569 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:47.569 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:47.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c9806ac90 msgr2=0x7f5c9806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c9806ac90 0x7f5c9806d150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8074100 tx=0x7f5ca8008040 comp rx=0 tx=0).stop 2024-03-29T09:11:47.571 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb8160cc0 msgr2=0x7f5cb80cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb8160cc0 0x7f5cb80cb790 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5cb808c9a0 tx=0x7f5cb0004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 shutdown_connections 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f5c9806ac90 0x7f5c9806d150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cb8160cc0 0x7f5cb80cb790 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5cb80d3a70 0x7f5cb80cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 --2- 172.21.15.99:0/3520942655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cb81608e0 0x7f5cb80cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 >> 172.21.15.99:0/3520942655 conn(0x7f5cb809c490 msgr2=0x7f5cb80c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 shutdown_connections 2024-03-29T09:11:47.572 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.572+0000 7f5cbe600700 1 -- 172.21.15.99:0/3520942655 wait complete. 2024-03-29T09:11:47.589 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T09:11:47.589 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:11:47.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.779+0000 7fc6e5200700 1 Processor -- start 2024-03-29T09:11:47.778 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.779+0000 7fc6e5200700 1 -- start start 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.780+0000 7fc6e5200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e01608e0 0x7fc6e00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6e5200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc6e0160cc0 0x7fc6e00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6e5200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6e00d3a70 0x7fc6e00d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6e5200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e00cddc0 con 0x7fc6e00d3a70 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6e5200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e00ce0a0 con 0x7fc6e0160cc0 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6e5200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e00ce380 con 0x7fc6e01608e0 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6e00d3a70 0x7fc6e00d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc6e00d3a70 0x7fc6e00d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53798/0 (socket says 172.21.15.99:53798) 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 -- 172.21.15.99:0/3413009740 learned_addr learned my addr 172.21.15.99:0/3413009740 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6dd400700 1 --2- 172.21.15.99:0/3413009740 >> v2:172.21.15.133:3300/0 conn(0x7fc6e0160cc0 0x7fc6e00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6dde00700 1 --2- 172.21.15.99:0/3413009740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e01608e0 0x7fc6e00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 -- 172.21.15.99:0/3413009740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e01608e0 msgr2=0x7fc6e00d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:47.781 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 --2- 172.21.15.99:0/3413009740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e01608e0 0x7fc6e00d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 -- 172.21.15.99:0/3413009740 >> v2:172.21.15.133:3300/0 conn(0x7fc6e0160cc0 msgr2=0x7fc6e00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 --2- 172.21.15.99:0/3413009740 >> v2:172.21.15.133:3300/0 conn(0x7fc6e0160cc0 0x7fc6e00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6de800700 1 -- 172.21.15.99:0/3413009740 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e00ce660 con 0x7fc6e00d3a70 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.781+0000 7fc6dde00700 1 --2- 172.21.15.99:0/3413009740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e01608e0 0x7fc6e00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6de800700 1 --2- 172.21.15.99:0/3413009740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e00d3a70 0x7fc6e00d1410 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc6e008d120 tx=0x7fc6d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f2161f235000fd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6dca00700 1 -- 172.21.15.99:0/3413009740 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800e070 con 0x7fc6e00d3a70 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6dca00700 1 -- 172.21.15.99:0/3413009740 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d800b910 con 0x7fc6e00d3a70 2024-03-29T09:11:47.782 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6dca00700 1 -- 172.21.15.99:0/3413009740 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800bcc0 con 0x7fc6e00d3a70 2024-03-29T09:11:47.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6e5200700 1 -- 172.21.15.99:0/3413009740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e00d3a70 msgr2=0x7fc6e00d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3413009740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e00d3a70 0x7fc6e00d1410 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc6e008d120 tx=0x7fc6d800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:47.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6e5200700 1 -- 172.21.15.99:0/3413009740 shutdown_connections 2024-03-29T09:11:47.783 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3413009740 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e01608e0 0x7fc6e00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3413009740 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e00d3a70 0x7fc6e00d1410 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3413009740 >> v2:172.21.15.133:3300/0 conn(0x7fc6e0160cc0 0x7fc6e00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.782+0000 7fc6e5200700 1 -- 172.21.15.99:0/3413009740 >> 172.21.15.99:0/3413009740 conn(0x7fc6e009c490 msgr2=0x7fc6e00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:47.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.783+0000 7fc6e5200700 1 -- 172.21.15.99:0/3413009740 shutdown_connections 2024-03-29T09:11:47.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.783+0000 7fc6e5200700 1 -- 172.21.15.99:0/3413009740 wait complete. 2024-03-29T09:11:47.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.784+0000 7fc6e5200700 1 Processor -- start 2024-03-29T09:11:47.784 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.784+0000 7fc6e5200700 1 -- start start 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.785+0000 7fc6e5200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e00d3a70 0x7fc6e01524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.785+0000 7fc6e5200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e01608e0 0x7fc6e01f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.785+0000 7fc6e5200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e0160cc0 0x7fc6e01f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.785+0000 7fc6e5200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e009ef70 con 0x7fc6e01608e0 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.785+0000 7fc6e5200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e009f250 con 0x7fc6e00d3a70 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.785+0000 7fc6e5200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e009f530 con 0x7fc6e0160cc0 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e01608e0 0x7fc6e01f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e01608e0 0x7fc6e01f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:53806/0 (socket says 172.21.15.99:53806) 2024-03-29T09:11:47.786 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 -- 172.21.15.99:0/3393424323 learned_addr learned my addr 172.21.15.99:0/3393424323 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 -- 172.21.15.99:0/3393424323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e0160cc0 msgr2=0x7fc6e01f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e0160cc0 0x7fc6e01f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 -- 172.21.15.99:0/3393424323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e00d3a70 msgr2=0x7fc6e01524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e00d3a70 0x7fc6e01524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 -- 172.21.15.99:0/3393424323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d8009b30 con 0x7fc6e01608e0 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6dd400700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e01608e0 0x7fc6e01f60c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00e870 tx=0x7fc6cc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6d6000700 1 -- 172.21.15.99:0/3393424323 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc009590 con 0x7fc6e01608e0 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6d6000700 1 -- 172.21.15.99:0/3393424323 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc00ee70 con 0x7fc6e01608e0 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e009f810 con 0x7fc6e01608e0 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.786+0000 7fc6d6000700 1 -- 172.21.15.99:0/3393424323 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc00ca50 con 0x7fc6e01608e0 2024-03-29T09:11:47.787 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.787+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e01529f0 con 0x7fc6e01608e0 2024-03-29T09:11:47.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.788+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e00677a0 con 0x7fc6e01608e0 2024-03-29T09:11:47.788 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.789+0000 7fc6d6000700 1 -- 172.21.15.99:0/3393424323 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc6cc00cbb0 con 0x7fc6e01608e0 2024-03-29T09:11:47.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.791+0000 7fc6d6000700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc6c006ac90 0x7fc6c006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:47.789 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.791+0000 7fc6d6000700 1 -- 172.21.15.99:0/3393424323 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc6cc0893b0 con 0x7fc6e01608e0 2024-03-29T09:11:47.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.791+0000 7fc6dde00700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc6c006ac90 0x7fc6c006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:47.790 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.791+0000 7fc6dde00700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc6c006ac90 0x7fc6c006d150 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0006fd0 tx=0x7fc6d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:47.793 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.795+0000 7fc6d6000700 1 -- 172.21.15.99:0/3393424323 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fc6cc054450 con 0x7fc6e01608e0 2024-03-29T09:11:47.995 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:47.996+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6e00b22a0 con 0x7fc6c006ac90 2024-03-29T09:11:48.003 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.005+0000 7fc6d6000700 1 -- 172.21.15.99:0/3393424323 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274466 (secure 0 0 0) 0x7fc6d0004030 con 0x7fc6c006ac90 2024-03-29T09:11:48.008 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:48.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc6c006ac90 msgr2=0x7fc6c006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc6c006ac90 0x7fc6c006d150 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0006fd0 tx=0x7fc6d0008040 comp rx=0 tx=0).stop 2024-03-29T09:11:48.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e01608e0 msgr2=0x7fc6e01f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e01608e0 0x7fc6e01f60c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00e870 tx=0x7fc6cc00ec30 comp rx=0 tx=0).stop 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 shutdown_connections 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc6c006ac90 0x7fc6c006d150 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6e0160cc0 0x7fc6e01f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6e00d3a70 0x7fc6e01524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 --2- 172.21.15.99:0/3393424323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc6e01608e0 0x7fc6e01f60c0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 >> 172.21.15.99:0/3393424323 conn(0x7fc6e009c490 msgr2=0x7fc6e015f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 shutdown_connections 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.009+0000 7fc6e5200700 1 -- 172.21.15.99:0/3393424323 wait complete. 2024-03-29T09:11:48.011 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:11:48.027 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-03-29T09:11:46.486476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222056,"kb_used_data":5868,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752664,"statfs":{"total":773094113280,"available":772866727936,"internally_reserved":0,"allocated":6008832,"data_stored":3292909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.337028"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T09:11:38.651124+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:11:38.651124+0000","last_peered":"2024-03-29T09:11:38.651124+0000","last_clean":"2024-03-29T09:11:38.651124+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:11:38.651124+0000","last_undegraded":"2024-03-29T09:11:38.651124+0000","last_fullsized":"2024-03-29T09:11:38.651124+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":261979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.49399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93799999999999994,"5min":1.639,"15min":1.639},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.48599999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.83099999999999996,"5min":1.992,"15min":1.992},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.81699999999999995,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.92000000000000004,"5min":2.883,"15min":2.883},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.51800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80400000000000005,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91600000000000004,"5min":1.387,"15min":1.387},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.54400000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94499999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0589999999999999,"5min":1.367,"15min":1.367},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0209999999999999,"5min":1.458,"15min":1.458},"last":0.499},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72499999999999998,"15min":0.73799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.502,"15min":0.502},"max":{"1min":0.96299999999999997,"5min":1.413,"15min":1.413},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":261979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.81200000000000006,"15min":0.83799999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.96099999999999997,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.48999999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.45800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.74199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.79500000000000004,"15min":0.80200000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.006,"5min":1.026,"15min":1.026},"last":0.877},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.94499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1191936,"data_stored":852347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.175,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.5259999999999998,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.64900000000000002,"15min":0.68300000000000005},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.57199999999999995,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66700000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.438}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.008,"15min":0.94899999999999995},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":12.356,"5min":12.356,"15min":12.356},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.73899999999999999,"15min":0.71499999999999997},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.91700000000000004,"15min":0.93799999999999994},"min":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6279999999999999,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83399999999999996,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.903,"5min":1.99,"15min":1.99},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.0609999999999999,"15min":0.99399999999999999},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":12.669,"5min":12.669,"15min":12.669},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.874,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.79300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":261979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.51300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.51700000000000002,"5min":0.751,"15min":0.751},"last":0.377}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.495}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67800000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":1.028,"15min":1.028},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99099999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.069,"15min":1.069},"last":0.622}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.02,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":267426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.47399999999999998,"15min":0.48799999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.311},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":1.252,"15min":1.252},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61899999999999999,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66900000000000004,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.503}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.746,"5min":2.415,"15min":2.415},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":2.056,"15min":2.056},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91400000000000003,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":2.073,"15min":2.073},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92500000000000004,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95199999999999996,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.878,"15min":0.90300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96799999999999997,"5min":2.387,"15min":2.387},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.91700000000000004,"15min":0.94699999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.91500000000000004,"5min":2.996,"15min":2.996},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95499999999999996,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84999999999999998,"15min":0.874},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88800000000000001,"5min":2.165,"15min":2.165},"last":0.64100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":267426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.53200000000000003,"15min":0.55100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61599999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.435},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.496,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.53900000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76300000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99299999999999999,"5min":2.278,"15min":2.278},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96399999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.59999999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88400000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.622},{"interface":"front","average":{"1min":0.748,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.90000000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":2.649,"15min":2.649},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.034,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27628,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":610304,"data_stored":267426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1319999999999999,"5min":1.742,"15min":1.742},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.77700000000000002,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.751,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2130000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84199999999999997,"5min":0.999,"15min":0.999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91300000000000003,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.84599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.871,"15min":0.90100000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0269999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.94399999999999995,"15min":0.97599999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.073,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1191936,"data_stored":852347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81799999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85499999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.872,"15min":0.872},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83699999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.91600000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":0.873},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73499999999999999,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.81100000000000005,"5min":1.05,"15min":1.05},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.83099999999999996,"5min":1.079,"15min":1.079},"last":0.76900000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.877,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:11:48.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:11:48.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.258+0000 7f0290200700 1 Processor -- start 2024-03-29T09:11:48.257 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.258+0000 7f0290200700 1 -- start start 2024-03-29T09:11:48.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.259+0000 7f0290200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880d1690 0x7f02880d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.258 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.259+0000 7f0290200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02880b2e00 0x7f02880b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.259+0000 7f0290200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02880cdc50 con 0x7f028815d5f0 2024-03-29T09:11:48.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.259+0000 7f0290200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02880d0980 con 0x7f02880b2e00 2024-03-29T09:11:48.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.259+0000 7f0290200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02880d2050 con 0x7f02880d1690 2024-03-29T09:11:48.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028d800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f028815d5f0 0x7f02880ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48224/0 (socket says 172.21.15.99:48224) 2024-03-29T09:11:48.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028d800700 1 -- 172.21.15.99:0/706169247 learned_addr learned my addr 172.21.15.99:0/706169247 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:48.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028e200700 1 --2- 172.21.15.99:0/706169247 >> v2:172.21.15.133:3300/0 conn(0x7f02880b2e00 0x7f02880b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.259 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028ce00700 1 --2- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880d1690 0x7f02880d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f0287e00700 1 -- 172.21.15.99:0/706169247 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535111192 0 0) 0x7f02880cdc50 con 0x7f028815d5f0 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f0287e00700 1 -- 172.21.15.99:0/706169247 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02700035b0 con 0x7f028815d5f0 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028e200700 1 -- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880d1690 msgr2=0x7f02880d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028e200700 1 --2- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880d1690 0x7f02880d1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028e200700 1 -- 172.21.15.99:0/706169247 >> v1:172.21.15.99:6789/0 conn(0x7f028815d5f0 legacy=0x7f02880ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028e200700 1 -- 172.21.15.99:0/706169247 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02880cf070 con 0x7f02880b2e00 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.260+0000 7f028ce00700 1 --2- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880d1690 0x7f02880d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:48.261 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f028e200700 1 --2- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880b2e00 0x7f02880b3290 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0280009d90 tx=0x7f028000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574403a487879b29 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0287e00700 1 -- 172.21.15.99:0/706169247 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028000e070 con 0x7f02880b2e00 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0287e00700 1 -- 172.21.15.99:0/706169247 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028000bb20 con 0x7f02880b2e00 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0287e00700 1 -- 172.21.15.99:0/706169247 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280012430 con 0x7f02880b2e00 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0290200700 1 -- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880b2e00 msgr2=0x7f02880b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0290200700 1 --2- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880b2e00 0x7f02880b3290 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0280009d90 tx=0x7f028000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0290200700 1 -- 172.21.15.99:0/706169247 shutdown_connections 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0290200700 1 --2- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880b2e00 0x7f02880b3290 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0290200700 1 --2- 172.21.15.99:0/706169247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880d1690 0x7f02880d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.261+0000 7f0290200700 1 -- 172.21.15.99:0/706169247 >> 172.21.15.99:0/706169247 conn(0x7f028808cb40 msgr2=0x7f028809c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.262+0000 7f0290200700 1 -- 172.21.15.99:0/706169247 shutdown_connections 2024-03-29T09:11:48.262 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.262+0000 7f0290200700 1 -- 172.21.15.99:0/706169247 wait complete. 2024-03-29T09:11:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.264+0000 7f0290200700 1 Processor -- start 2024-03-29T09:11:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.264+0000 7f0290200700 1 -- start start 2024-03-29T09:11:48.267 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.265+0000 7f0290200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880b2e00 0x7f02881526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.265+0000 7f0290200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880d1690 0x7f02881f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.265+0000 7f0290200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028815d5f0 0x7f02880cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.265+0000 7f0290200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0288152c10 con 0x7f028815d5f0 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.265+0000 7f0290200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02881f70b0 con 0x7f02880d1690 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.265+0000 7f0290200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028815f4b0 con 0x7f02880b2e00 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028815d5f0 0x7f02880cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028815d5f0 0x7f02880cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57644/0 (socket says 172.21.15.99:57644) 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 -- 172.21.15.99:0/1772070208 learned_addr learned my addr 172.21.15.99:0/1772070208 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028d800700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880b2e00 0x7f02881526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028ce00700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880d1690 0x7f02881f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 -- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880b2e00 msgr2=0x7f02881526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880b2e00 0x7f02881526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 -- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880d1690 msgr2=0x7f02881f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880d1690 0x7f02881f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 -- 172.21.15.99:0/1772070208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740094a0 con 0x7f028815d5f0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.266+0000 7f028e200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028815d5f0 0x7f02880cc520 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0280014040 tx=0x7f0280004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.267+0000 7f0285600700 1 -- 172.21.15.99:0/1772070208 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800129f0 con 0x7f028815d5f0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.267+0000 7f0285600700 1 -- 172.21.15.99:0/1772070208 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280020070 con 0x7f028815d5f0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.267+0000 7f0285600700 1 -- 172.21.15.99:0/1772070208 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280012db0 con 0x7f028815d5f0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.267+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0280009a70 con 0x7f028815d5f0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.267+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028815fa70 con 0x7f028815d5f0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.268+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02880c78d0 con 0x7f028815d5f0 2024-03-29T09:11:48.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.269+0000 7f0285600700 1 -- 172.21.15.99:0/1772070208 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0280004040 con 0x7f028815d5f0 2024-03-29T09:11:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.269+0000 7f0285600700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f026806ac90 0x7f026806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.270+0000 7f0285600700 1 -- 172.21.15.99:0/1772070208 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0280022070 con 0x7f028815d5f0 2024-03-29T09:11:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.270+0000 7f028d800700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f026806ac90 0x7f026806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.270+0000 7f028d800700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f026806ac90 0x7f026806d150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f027c007fd0 tx=0x7f027c007dc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:48.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.274+0000 7f0285600700 1 -- 172.21.15.99:0/1772070208 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f02800560f0 con 0x7f028815d5f0 2024-03-29T09:11:48.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.472+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f02880ce160 con 0x7f026806ac90 2024-03-29T09:11:48.478 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.479+0000 7f0285600700 1 -- 172.21.15.99:0/1772070208 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274466 (secure 0 0 0) 0x7f027c0058e0 con 0x7f026806ac90 2024-03-29T09:11:48.482 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f026806ac90 msgr2=0x7f026806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f026806ac90 0x7f026806d150 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f027c007fd0 tx=0x7f027c007dc0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028815d5f0 msgr2=0x7f02880cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028815d5f0 0x7f02880cc520 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0280014040 tx=0x7f0280004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 shutdown_connections 2024-03-29T09:11:48.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f026806ac90 0x7f026806d150 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f028815d5f0 0x7f02880cc520 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02880b2e00 0x7f02881526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 --2- 172.21.15.99:0/1772070208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02880d1690 0x7f02881f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 >> 172.21.15.99:0/1772070208 conn(0x7f028808cb40 msgr2=0x7f02880cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 shutdown_connections 2024-03-29T09:11:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.484+0000 7f0290200700 1 -- 172.21.15.99:0/1772070208 wait complete. 2024-03-29T09:11:48.486 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:11:48.501 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-03-29T09:11:46.486476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222056,"kb_used_data":5868,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752664,"statfs":{"total":773094113280,"available":772866727936,"internally_reserved":0,"allocated":6008832,"data_stored":3292909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.337028"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:43.149736+0000","last_change":"2024-03-29T09:11:43.149736+0000","last_active":"2024-03-29T09:11:43.149736+0000","last_peered":"2024-03-29T09:11:43.149736+0000","last_clean":"2024-03-29T09:11:43.149736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:43.149736+0000","last_undegraded":"2024-03-29T09:11:43.149736+0000","last_fullsized":"2024-03-29T09:11:43.149736+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T09:11:42.142091+0000","last_change":"2024-03-29T09:11:42.142091+0000","last_active":"2024-03-29T09:11:42.142091+0000","last_peered":"2024-03-29T09:11:42.142091+0000","last_clean":"2024-03-29T09:11:42.142091+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T09:11:42.142091+0000","last_undegraded":"2024-03-29T09:11:42.142091+0000","last_fullsized":"2024-03-29T09:11:42.142091+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T09:11:38.651124+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:11:38.651124+0000","last_peered":"2024-03-29T09:11:38.651124+0000","last_clean":"2024-03-29T09:11:38.651124+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:11:38.651124+0000","last_undegraded":"2024-03-29T09:11:38.651124+0000","last_fullsized":"2024-03-29T09:11:38.651124+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":261979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.49399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93799999999999994,"5min":1.639,"15min":1.639},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.48599999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.83099999999999996,"5min":1.992,"15min":1.992},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.81699999999999995,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.92000000000000004,"5min":2.883,"15min":2.883},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.51800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80400000000000005,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91600000000000004,"5min":1.387,"15min":1.387},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.54400000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94499999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0589999999999999,"5min":1.367,"15min":1.367},"last":0.73899999999999999}]},{"osd":5,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0209999999999999,"5min":1.458,"15min":1.458},"last":0.499},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72499999999999998,"15min":0.73799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.502,"15min":0.502},"max":{"1min":0.96299999999999997,"5min":1.413,"15min":1.413},"last":0.56999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27604,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":598016,"data_stored":261979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.81200000000000006,"15min":0.83799999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.96099999999999997,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.48999999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.45800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.74199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.79500000000000004,"15min":0.80200000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.006,"5min":1.026,"15min":1.026},"last":0.877},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.94499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1191936,"data_stored":852347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.175,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.5259999999999998,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.64900000000000002,"15min":0.68300000000000005},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.57199999999999995,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.34000000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66700000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.438}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.008,"15min":0.94899999999999995},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":12.356,"5min":12.356,"15min":12.356},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.73899999999999999,"15min":0.71499999999999997},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.91700000000000004,"15min":0.93799999999999994},"min":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6279999999999999,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83399999999999996,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.903,"5min":1.99,"15min":1.99},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.0609999999999999,"15min":0.99399999999999999},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":12.669,"5min":12.669,"15min":12.669},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.874,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.79300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":261979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.51300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.51700000000000002,"5min":0.751,"15min":0.751},"last":0.377}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.495}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67800000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":1.028,"15min":1.028},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99099999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.069,"15min":1.069},"last":0.622}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.02,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":267426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.47399999999999998,"15min":0.48799999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.311},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":1.252,"15min":1.252},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61899999999999999,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66900000000000004,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.503}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.746,"5min":2.415,"15min":2.415},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":2.056,"15min":2.056},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91400000000000003,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":2.073,"15min":2.073},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92500000000000004,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95199999999999996,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.878,"15min":0.90300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96799999999999997,"5min":2.387,"15min":2.387},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.91700000000000004,"15min":0.94699999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.91500000000000004,"5min":2.996,"15min":2.996},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95499999999999996,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84999999999999998,"15min":0.874},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88800000000000001,"5min":2.165,"15min":2.165},"last":0.64100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":267426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.53200000000000003,"15min":0.55100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61599999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.435},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.496,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.53900000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76300000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99299999999999999,"5min":2.278,"15min":2.278},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96399999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.59999999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88400000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.622},{"interface":"front","average":{"1min":0.748,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.90000000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":2.649,"15min":2.649},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.034,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.67000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27628,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":610304,"data_stored":267426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1319999999999999,"5min":1.742,"15min":1.742},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.77700000000000002,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.751,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2130000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84199999999999997,"5min":0.999,"15min":0.999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.72399999999999998}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91300000000000003,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.70499999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.84599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.871,"15min":0.90100000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0269999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.94399999999999995,"15min":0.97599999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.073,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1191936,"data_stored":852347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81799999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85499999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.872,"15min":0.872},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83699999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.91600000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":0.873},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73499999999999999,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.81100000000000005,"5min":1.05,"15min":1.05},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.83099999999999996,"5min":1.079,"15min":1.079},"last":0.76900000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.877,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.71799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:11:48.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:11:48.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.732+0000 7f9bd8400700 1 Processor -- start 2024-03-29T09:11:48.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.733+0000 7f9bd8400700 1 -- start start 2024-03-29T09:11:48.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd8400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9bd01608e0 0x7f9bd00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd8400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd0160cc0 0x7f9bd00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd8400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd00cddc0 con 0x7f9bd00d3a70 2024-03-29T09:11:48.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd8400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd00ce0a0 con 0x7f9bd01608e0 2024-03-29T09:11:48.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd8400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00ce380 con 0x7f9bd0160cc0 2024-03-29T09:11:48.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd5000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd0160cc0 0x7f9bd00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd6400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9bd00d3a70 0x7f9bd00d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:53524/0 (socket says 172.21.15.99:53524) 2024-03-29T09:11:48.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd6400700 1 -- 172.21.15.99:0/1423972878 learned_addr learned my addr 172.21.15.99:0/1423972878 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:48.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.734+0000 7f9bd5a00700 1 --2- 172.21.15.99:0/1423972878 >> v2:172.21.15.133:3300/0 conn(0x7f9bd01608e0 0x7f9bd00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bcfe00700 1 -- 172.21.15.99:0/1423972878 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700336790 0 0) 0x7f9bd00cddc0 con 0x7f9bd00d3a70 2024-03-29T09:11:48.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bcfe00700 1 -- 172.21.15.99:0/1423972878 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb80035b0 con 0x7f9bd00d3a70 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bd5000700 1 -- 172.21.15.99:0/1423972878 >> v2:172.21.15.133:3300/0 conn(0x7f9bd01608e0 msgr2=0x7f9bd00d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bd5000700 1 --2- 172.21.15.99:0/1423972878 >> v2:172.21.15.133:3300/0 conn(0x7f9bd01608e0 0x7f9bd00d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bd5000700 1 -- 172.21.15.99:0/1423972878 >> v1:172.21.15.133:6789/0 conn(0x7f9bd00d3a70 legacy=0x7f9bd00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bd5000700 1 -- 172.21.15.99:0/1423972878 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00ce660 con 0x7f9bd0160cc0 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bd5a00700 1 --2- 172.21.15.99:0/1423972878 >> v2:172.21.15.133:3300/0 conn(0x7f9bd01608e0 0x7f9bd00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bd5000700 1 --2- 172.21.15.99:0/1423972878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd0160cc0 0x7f9bd00ceb80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009840 tx=0x7f9bbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=895c9514150eac39 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bcfe00700 1 -- 172.21.15.99:0/1423972878 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc011830 con 0x7f9bd0160cc0 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bcfe00700 1 -- 172.21.15.99:0/1423972878 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc011990 con 0x7f9bd0160cc0 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.735+0000 7f9bcfe00700 1 -- 172.21.15.99:0/1423972878 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc011d20 con 0x7f9bd0160cc0 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 -- 172.21.15.99:0/1423972878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd0160cc0 msgr2=0x7f9bd00ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1423972878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd0160cc0 0x7f9bd00ceb80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009840 tx=0x7f9bbc009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 -- 172.21.15.99:0/1423972878 shutdown_connections 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1423972878 >> v2:172.21.15.133:3300/0 conn(0x7f9bd01608e0 0x7f9bd00d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1423972878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd0160cc0 0x7f9bd00ceb80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.736 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 -- 172.21.15.99:0/1423972878 >> 172.21.15.99:0/1423972878 conn(0x7f9bd009c490 msgr2=0x7f9bd00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:48.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 -- 172.21.15.99:0/1423972878 shutdown_connections 2024-03-29T09:11:48.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.736+0000 7f9bd8400700 1 -- 172.21.15.99:0/1423972878 wait complete. 2024-03-29T09:11:48.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.738+0000 7f9bd8400700 1 Processor -- start 2024-03-29T09:11:48.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.738+0000 7f9bd8400700 1 -- start start 2024-03-29T09:11:48.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd8400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bd00d3a70 0x7f9bd00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd8400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd01608e0 0x7f9bd00b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd8400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd0160cc0 0x7f9bd01e7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd8400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00b2960 con 0x7f9bd0160cc0 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd8400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00b2b90 con 0x7f9bd00d3a70 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd8400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd00b2e70 con 0x7f9bd01608e0 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd6400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd0160cc0 0x7f9bd01e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd6400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd0160cc0 0x7f9bd01e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57654/0 (socket says 172.21.15.99:57654) 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd6400700 1 -- 172.21.15.99:0/1074068523 learned_addr learned my addr 172.21.15.99:0/1074068523 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd5000700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd01608e0 0x7f9bd00b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.739+0000 7f9bd5a00700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bd00d3a70 0x7f9bd00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd6400700 1 -- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd01608e0 msgr2=0x7f9bd00b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd6400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd01608e0 0x7f9bd00b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd6400700 1 -- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bd00d3a70 msgr2=0x7f9bd00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd6400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bd00d3a70 0x7f9bd00cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd6400700 1 -- 172.21.15.99:0/1074068523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40094a0 con 0x7f9bd0160cc0 2024-03-29T09:11:48.740 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd6400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd0160cc0 0x7f9bd01e7090 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800cb40 tx=0x7f9bc800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bcd600700 1 -- 172.21.15.99:0/1074068523 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800a9e0 con 0x7f9bd0160cc0 2024-03-29T09:11:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bcd600700 1 -- 172.21.15.99:0/1074068523 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc800ab40 con 0x7f9bd0160cc0 2024-03-29T09:11:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bcd600700 1 -- 172.21.15.99:0/1074068523 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8012400 con 0x7f9bd0160cc0 2024-03-29T09:11:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc0094a0 con 0x7f9bd0160cc0 2024-03-29T09:11:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.740+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd0154bd0 con 0x7f9bd0160cc0 2024-03-29T09:11:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.742+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd0154dc0 con 0x7f9bd0160cc0 2024-03-29T09:11:48.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.742+0000 7f9bcd600700 1 -- 172.21.15.99:0/1074068523 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9bc801b3b0 con 0x7f9bd0160cc0 2024-03-29T09:11:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.744+0000 7f9bcd600700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9bb006ac90 0x7f9bb006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:48.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.744+0000 7f9bcd600700 1 -- 172.21.15.99:0/1074068523 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9bc8088f20 con 0x7f9bd0160cc0 2024-03-29T09:11:48.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.744+0000 7f9bd5a00700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9bb006ac90 0x7f9bb006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:48.743 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.744+0000 7f9bd5a00700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9bb006ac90 0x7f9bb006d150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009d70 tx=0x7f9bc4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:48.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.748+0000 7f9bcd600700 1 -- 172.21.15.99:0/1074068523 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9bc8054100 con 0x7f9bd0160cc0 2024-03-29T09:11:48.947 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.948+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bd00b22a0 con 0x7f9bb006ac90 2024-03-29T09:11:48.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.957+0000 7f9bcd600700 1 -- 172.21.15.99:0/1074068523 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289201 (secure 0 0 0) 0x7f9bc40099c0 con 0x7f9bb006ac90 2024-03-29T09:11:48.960 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:48.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9bb006ac90 msgr2=0x7f9bb006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9bb006ac90 0x7f9bb006d150 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009d70 tx=0x7f9bc4009210 comp rx=0 tx=0).stop 2024-03-29T09:11:48.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd0160cc0 msgr2=0x7f9bd01e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:48.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd0160cc0 0x7f9bd01e7090 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800cb40 tx=0x7f9bc800ce50 comp rx=0 tx=0).stop 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 shutdown_connections 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9bb006ac90 0x7f9bb006d150 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd0160cc0 0x7f9bd01e7090 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bd00d3a70 0x7f9bd00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 --2- 172.21.15.99:0/1074068523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9bd01608e0 0x7f9bd00b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 >> 172.21.15.99:0/1074068523 conn(0x7f9bd009c490 msgr2=0x7f9bd015d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 shutdown_connections 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:48.961+0000 7f9bd8400700 1 -- 172.21.15.99:0/1074068523 wait complete. 2024-03-29T09:11:48.963 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:11:48.978 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-03-29T09:11:48.486748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222552,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752168,"statfs":{"total":773094113280,"available":772866220032,"internally_reserved":0,"allocated":6529024,"data_stored":3536444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000831"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996579+0000","last_change":"2024-03-29T09:11:43.690372+0000","last_active":"2024-03-29T09:11:44.996579+0000","last_peered":"2024-03-29T09:11:44.996579+0000","last_clean":"2024-03-29T09:11:44.996579+0000","last_became_active":"2024-03-29T09:11:43.690046+0000","last_became_peered":"2024-03-29T09:11:43.690046+0000","last_unstale":"2024-03-29T09:11:44.996579+0000","last_undegraded":"2024-03-29T09:11:44.996579+0000","last_fullsized":"2024-03-29T09:11:44.996579+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:23.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667247+0000","last_change":"2024-03-29T09:11:43.681209+0000","last_active":"2024-03-29T09:11:44.667247+0000","last_peered":"2024-03-29T09:11:44.667247+0000","last_clean":"2024-03-29T09:11:44.667247+0000","last_became_active":"2024-03-29T09:11:43.680965+0000","last_became_peered":"2024-03-29T09:11:43.680965+0000","last_unstale":"2024-03-29T09:11:44.667247+0000","last_undegraded":"2024-03-29T09:11:44.667247+0000","last_fullsized":"2024-03-29T09:11:44.667247+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:32.530881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497341+0000","last_change":"2024-03-29T09:11:43.672222+0000","last_active":"2024-03-29T09:11:45.497341+0000","last_peered":"2024-03-29T09:11:45.497341+0000","last_clean":"2024-03-29T09:11:45.497341+0000","last_became_active":"2024-03-29T09:11:43.671980+0000","last_became_peered":"2024-03-29T09:11:43.671980+0000","last_unstale":"2024-03-29T09:11:45.497341+0000","last_undegraded":"2024-03-29T09:11:45.497341+0000","last_fullsized":"2024-03-29T09:11:45.497341+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:26:11.492718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497818+0000","last_change":"2024-03-29T09:11:43.689409+0000","last_active":"2024-03-29T09:11:45.497818+0000","last_peered":"2024-03-29T09:11:45.497818+0000","last_clean":"2024-03-29T09:11:45.497818+0000","last_became_active":"2024-03-29T09:11:43.689142+0000","last_became_peered":"2024-03-29T09:11:43.689142+0000","last_unstale":"2024-03-29T09:11:45.497818+0000","last_undegraded":"2024-03-29T09:11:45.497818+0000","last_fullsized":"2024-03-29T09:11:45.497818+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:45:02.972724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996181+0000","last_change":"2024-03-29T09:11:43.685296+0000","last_active":"2024-03-29T09:11:44.996181+0000","last_peered":"2024-03-29T09:11:44.996181+0000","last_clean":"2024-03-29T09:11:44.996181+0000","last_became_active":"2024-03-29T09:11:43.682430+0000","last_became_peered":"2024-03-29T09:11:43.682430+0000","last_unstale":"2024-03-29T09:11:44.996181+0000","last_undegraded":"2024-03-29T09:11:44.996181+0000","last_fullsized":"2024-03-29T09:11:44.996181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996027+0000","last_change":"2024-03-29T09:11:43.691095+0000","last_active":"2024-03-29T09:11:44.996027+0000","last_peered":"2024-03-29T09:11:44.996027+0000","last_clean":"2024-03-29T09:11:44.996027+0000","last_became_active":"2024-03-29T09:11:43.690828+0000","last_became_peered":"2024-03-29T09:11:43.690828+0000","last_unstale":"2024-03-29T09:11:44.996027+0000","last_undegraded":"2024-03-29T09:11:44.996027+0000","last_fullsized":"2024-03-29T09:11:44.996027+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:34:41.067618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497922+0000","last_change":"2024-03-29T09:11:43.685480+0000","last_active":"2024-03-29T09:11:45.497922+0000","last_peered":"2024-03-29T09:11:45.497922+0000","last_clean":"2024-03-29T09:11:45.497922+0000","last_became_active":"2024-03-29T09:11:43.683700+0000","last_became_peered":"2024-03-29T09:11:43.683700+0000","last_unstale":"2024-03-29T09:11:45.497922+0000","last_undegraded":"2024-03-29T09:11:45.497922+0000","last_fullsized":"2024-03-29T09:11:45.497922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:00:03.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496748+0000","last_change":"2024-03-29T09:11:43.677985+0000","last_active":"2024-03-29T09:11:45.496748+0000","last_peered":"2024-03-29T09:11:45.496748+0000","last_clean":"2024-03-29T09:11:45.496748+0000","last_became_active":"2024-03-29T09:11:43.677658+0000","last_became_peered":"2024-03-29T09:11:43.677658+0000","last_unstale":"2024-03-29T09:11:45.496748+0000","last_undegraded":"2024-03-29T09:11:45.496748+0000","last_fullsized":"2024-03-29T09:11:45.496748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:04.674681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497195+0000","last_change":"2024-03-29T09:11:43.681141+0000","last_active":"2024-03-29T09:11:45.497195+0000","last_peered":"2024-03-29T09:11:45.497195+0000","last_clean":"2024-03-29T09:11:45.497195+0000","last_became_active":"2024-03-29T09:11:43.680651+0000","last_became_peered":"2024-03-29T09:11:43.680651+0000","last_unstale":"2024-03-29T09:11:45.497195+0000","last_undegraded":"2024-03-29T09:11:45.497195+0000","last_fullsized":"2024-03-29T09:11:45.497195+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:20.720413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996423+0000","last_change":"2024-03-29T09:11:43.689627+0000","last_active":"2024-03-29T09:11:44.996423+0000","last_peered":"2024-03-29T09:11:44.996423+0000","last_clean":"2024-03-29T09:11:44.996423+0000","last_became_active":"2024-03-29T09:11:43.689352+0000","last_became_peered":"2024-03-29T09:11:43.689352+0000","last_unstale":"2024-03-29T09:11:44.996423+0000","last_undegraded":"2024-03-29T09:11:44.996423+0000","last_fullsized":"2024-03-29T09:11:44.996423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:04.674681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499008+0000","last_change":"2024-03-29T09:11:43.684198+0000","last_active":"2024-03-29T09:11:45.499008+0000","last_peered":"2024-03-29T09:11:45.499008+0000","last_clean":"2024-03-29T09:11:45.499008+0000","last_became_active":"2024-03-29T09:11:43.683881+0000","last_became_peered":"2024-03-29T09:11:43.683881+0000","last_unstale":"2024-03-29T09:11:45.499008+0000","last_undegraded":"2024-03-29T09:11:45.499008+0000","last_fullsized":"2024-03-29T09:11:45.499008+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:17:07.814736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497080+0000","last_change":"2024-03-29T09:11:43.680518+0000","last_active":"2024-03-29T09:11:45.497080+0000","last_peered":"2024-03-29T09:11:45.497080+0000","last_clean":"2024-03-29T09:11:45.497080+0000","last_became_active":"2024-03-29T09:11:43.680221+0000","last_became_peered":"2024-03-29T09:11:43.680221+0000","last_unstale":"2024-03-29T09:11:45.497080+0000","last_undegraded":"2024-03-29T09:11:45.497080+0000","last_fullsized":"2024-03-29T09:11:45.497080+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:04:14.657461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667609+0000","last_change":"2024-03-29T09:11:43.688051+0000","last_active":"2024-03-29T09:11:44.667609+0000","last_peered":"2024-03-29T09:11:44.667609+0000","last_clean":"2024-03-29T09:11:44.667609+0000","last_became_active":"2024-03-29T09:11:43.687664+0000","last_became_peered":"2024-03-29T09:11:43.687664+0000","last_unstale":"2024-03-29T09:11:44.667609+0000","last_undegraded":"2024-03-29T09:11:44.667609+0000","last_fullsized":"2024-03-29T09:11:44.667609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:37.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497047+0000","last_change":"2024-03-29T09:11:43.665919+0000","last_active":"2024-03-29T09:11:45.497047+0000","last_peered":"2024-03-29T09:11:45.497047+0000","last_clean":"2024-03-29T09:11:45.497047+0000","last_became_active":"2024-03-29T09:11:43.665662+0000","last_became_peered":"2024-03-29T09:11:43.665662+0000","last_unstale":"2024-03-29T09:11:45.497047+0000","last_undegraded":"2024-03-29T09:11:45.497047+0000","last_fullsized":"2024-03-29T09:11:45.497047+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:25.840707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667467+0000","last_change":"2024-03-29T09:11:43.683560+0000","last_active":"2024-03-29T09:11:44.667467+0000","last_peered":"2024-03-29T09:11:44.667467+0000","last_clean":"2024-03-29T09:11:44.667467+0000","last_became_active":"2024-03-29T09:11:43.683227+0000","last_became_peered":"2024-03-29T09:11:43.683227+0000","last_unstale":"2024-03-29T09:11:44.667467+0000","last_undegraded":"2024-03-29T09:11:44.667467+0000","last_fullsized":"2024-03-29T09:11:44.667467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:34.499347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996022+0000","last_change":"2024-03-29T09:11:43.680836+0000","last_active":"2024-03-29T09:11:44.996022+0000","last_peered":"2024-03-29T09:11:44.996022+0000","last_clean":"2024-03-29T09:11:44.996022+0000","last_became_active":"2024-03-29T09:11:43.680568+0000","last_became_peered":"2024-03-29T09:11:43.680568+0000","last_unstale":"2024-03-29T09:11:44.996022+0000","last_undegraded":"2024-03-29T09:11:44.996022+0000","last_fullsized":"2024-03-29T09:11:44.996022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:03.217442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996654+0000","last_change":"2024-03-29T09:11:43.679345+0000","last_active":"2024-03-29T09:11:44.996654+0000","last_peered":"2024-03-29T09:11:44.996654+0000","last_clean":"2024-03-29T09:11:44.996654+0000","last_became_active":"2024-03-29T09:11:43.678923+0000","last_became_peered":"2024-03-29T09:11:43.678923+0000","last_unstale":"2024-03-29T09:11:44.996654+0000","last_undegraded":"2024-03-29T09:11:44.996654+0000","last_fullsized":"2024-03-29T09:11:44.996654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497557+0000","last_change":"2024-03-29T09:11:43.666174+0000","last_active":"2024-03-29T09:11:45.497557+0000","last_peered":"2024-03-29T09:11:45.497557+0000","last_clean":"2024-03-29T09:11:45.497557+0000","last_became_active":"2024-03-29T09:11:43.665473+0000","last_became_peered":"2024-03-29T09:11:43.665473+0000","last_unstale":"2024-03-29T09:11:45.497557+0000","last_undegraded":"2024-03-29T09:11:45.497557+0000","last_fullsized":"2024-03-29T09:11:45.497557+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667309+0000","last_change":"2024-03-29T09:11:43.671528+0000","last_active":"2024-03-29T09:11:44.667309+0000","last_peered":"2024-03-29T09:11:44.667309+0000","last_clean":"2024-03-29T09:11:44.667309+0000","last_became_active":"2024-03-29T09:11:43.670774+0000","last_became_peered":"2024-03-29T09:11:43.670774+0000","last_unstale":"2024-03-29T09:11:44.667309+0000","last_undegraded":"2024-03-29T09:11:44.667309+0000","last_fullsized":"2024-03-29T09:11:44.667309+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:20.720413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498963+0000","last_change":"2024-03-29T09:11:43.686864+0000","last_active":"2024-03-29T09:11:45.498963+0000","last_peered":"2024-03-29T09:11:45.498963+0000","last_clean":"2024-03-29T09:11:45.498963+0000","last_became_active":"2024-03-29T09:11:43.686418+0000","last_became_peered":"2024-03-29T09:11:43.686418+0000","last_unstale":"2024-03-29T09:11:45.498963+0000","last_undegraded":"2024-03-29T09:11:45.498963+0000","last_fullsized":"2024-03-29T09:11:45.498963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:17.190299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667610+0000","last_change":"2024-03-29T09:11:43.677257+0000","last_active":"2024-03-29T09:11:44.667610+0000","last_peered":"2024-03-29T09:11:44.667610+0000","last_clean":"2024-03-29T09:11:44.667610+0000","last_became_active":"2024-03-29T09:11:43.677003+0000","last_became_peered":"2024-03-29T09:11:43.677003+0000","last_unstale":"2024-03-29T09:11:44.667610+0000","last_undegraded":"2024-03-29T09:11:44.667610+0000","last_fullsized":"2024-03-29T09:11:44.667610+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:02.886449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498962+0000","last_change":"2024-03-29T09:11:43.688297+0000","last_active":"2024-03-29T09:11:45.498962+0000","last_peered":"2024-03-29T09:11:45.498962+0000","last_clean":"2024-03-29T09:11:45.498962+0000","last_became_active":"2024-03-29T09:11:43.688005+0000","last_became_peered":"2024-03-29T09:11:43.688005+0000","last_unstale":"2024-03-29T09:11:45.498962+0000","last_undegraded":"2024-03-29T09:11:45.498962+0000","last_fullsized":"2024-03-29T09:11:45.498962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:20.115771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498025+0000","last_change":"2024-03-29T09:11:43.678188+0000","last_active":"2024-03-29T09:11:45.498025+0000","last_peered":"2024-03-29T09:11:45.498025+0000","last_clean":"2024-03-29T09:11:45.498025+0000","last_became_active":"2024-03-29T09:11:43.677934+0000","last_became_peered":"2024-03-29T09:11:43.677934+0000","last_unstale":"2024-03-29T09:11:45.498025+0000","last_undegraded":"2024-03-29T09:11:45.498025+0000","last_fullsized":"2024-03-29T09:11:45.498025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:37:26.330731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496826+0000","last_change":"2024-03-29T09:11:43.675646+0000","last_active":"2024-03-29T09:11:45.496826+0000","last_peered":"2024-03-29T09:11:45.496826+0000","last_clean":"2024-03-29T09:11:45.496826+0000","last_became_active":"2024-03-29T09:11:43.675298+0000","last_became_peered":"2024-03-29T09:11:43.675298+0000","last_unstale":"2024-03-29T09:11:45.496826+0000","last_undegraded":"2024-03-29T09:11:45.496826+0000","last_fullsized":"2024-03-29T09:11:45.496826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:05.122450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667348+0000","last_change":"2024-03-29T09:11:43.680611+0000","last_active":"2024-03-29T09:11:44.667348+0000","last_peered":"2024-03-29T09:11:44.667348+0000","last_clean":"2024-03-29T09:11:44.667348+0000","last_became_active":"2024-03-29T09:11:43.680345+0000","last_became_peered":"2024-03-29T09:11:43.680345+0000","last_unstale":"2024-03-29T09:11:44.667348+0000","last_undegraded":"2024-03-29T09:11:44.667348+0000","last_fullsized":"2024-03-29T09:11:44.667348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:37.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667802+0000","last_change":"2024-03-29T09:11:43.682019+0000","last_active":"2024-03-29T09:11:44.667802+0000","last_peered":"2024-03-29T09:11:44.667802+0000","last_clean":"2024-03-29T09:11:44.667802+0000","last_became_active":"2024-03-29T09:11:43.681763+0000","last_became_peered":"2024-03-29T09:11:43.681763+0000","last_unstale":"2024-03-29T09:11:44.667802+0000","last_undegraded":"2024-03-29T09:11:44.667802+0000","last_fullsized":"2024-03-29T09:11:44.667802+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:48.040844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667365+0000","last_change":"2024-03-29T09:11:43.671717+0000","last_active":"2024-03-29T09:11:44.667365+0000","last_peered":"2024-03-29T09:11:44.667365+0000","last_clean":"2024-03-29T09:11:44.667365+0000","last_became_active":"2024-03-29T09:11:43.671231+0000","last_became_peered":"2024-03-29T09:11:43.671231+0000","last_unstale":"2024-03-29T09:11:44.667365+0000","last_undegraded":"2024-03-29T09:11:44.667365+0000","last_fullsized":"2024-03-29T09:11:44.667365+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:05.122450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496922+0000","last_change":"2024-03-29T09:11:43.680460+0000","last_active":"2024-03-29T09:11:45.496922+0000","last_peered":"2024-03-29T09:11:45.496922+0000","last_clean":"2024-03-29T09:11:45.496922+0000","last_became_active":"2024-03-29T09:11:43.680176+0000","last_became_peered":"2024-03-29T09:11:43.680176+0000","last_unstale":"2024-03-29T09:11:45.496922+0000","last_undegraded":"2024-03-29T09:11:45.496922+0000","last_fullsized":"2024-03-29T09:11:45.496922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:26.946110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496915+0000","last_change":"2024-03-29T09:11:43.679680+0000","last_active":"2024-03-29T09:11:45.496915+0000","last_peered":"2024-03-29T09:11:45.496915+0000","last_clean":"2024-03-29T09:11:45.496915+0000","last_became_active":"2024-03-29T09:11:43.679397+0000","last_became_peered":"2024-03-29T09:11:43.679397+0000","last_unstale":"2024-03-29T09:11:45.496915+0000","last_undegraded":"2024-03-29T09:11:45.496915+0000","last_fullsized":"2024-03-29T09:11:45.496915+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:16.721779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497963+0000","last_change":"2024-03-29T09:11:43.679106+0000","last_active":"2024-03-29T09:11:45.497963+0000","last_peered":"2024-03-29T09:11:45.497963+0000","last_clean":"2024-03-29T09:11:45.497963+0000","last_became_active":"2024-03-29T09:11:43.678804+0000","last_became_peered":"2024-03-29T09:11:43.678804+0000","last_unstale":"2024-03-29T09:11:45.497963+0000","last_undegraded":"2024-03-29T09:11:45.497963+0000","last_fullsized":"2024-03-29T09:11:45.497963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:48:32.315631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.666942+0000","last_change":"2024-03-29T09:11:43.669556+0000","last_active":"2024-03-29T09:11:44.666942+0000","last_peered":"2024-03-29T09:11:44.666942+0000","last_clean":"2024-03-29T09:11:44.666942+0000","last_became_active":"2024-03-29T09:11:43.669284+0000","last_became_peered":"2024-03-29T09:11:43.669284+0000","last_unstale":"2024-03-29T09:11:44.666942+0000","last_undegraded":"2024-03-29T09:11:44.666942+0000","last_fullsized":"2024-03-29T09:11:44.666942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:37.448155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498776+0000","last_change":"2024-03-29T09:11:43.688520+0000","last_active":"2024-03-29T09:11:45.498776+0000","last_peered":"2024-03-29T09:11:45.498776+0000","last_clean":"2024-03-29T09:11:45.498776+0000","last_became_active":"2024-03-29T09:11:43.688113+0000","last_became_peered":"2024-03-29T09:11:43.688113+0000","last_unstale":"2024-03-29T09:11:45.498776+0000","last_undegraded":"2024-03-29T09:11:45.498776+0000","last_fullsized":"2024-03-29T09:11:45.498776+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:25.840707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996437+0000","last_change":"2024-03-29T09:11:43.690479+0000","last_active":"2024-03-29T09:11:44.996437+0000","last_peered":"2024-03-29T09:11:44.996437+0000","last_clean":"2024-03-29T09:11:44.996437+0000","last_became_active":"2024-03-29T09:11:43.689919+0000","last_became_peered":"2024-03-29T09:11:43.689919+0000","last_unstale":"2024-03-29T09:11:44.996437+0000","last_undegraded":"2024-03-29T09:11:44.996437+0000","last_fullsized":"2024-03-29T09:11:44.996437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:54.749065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499230+0000","last_change":"2024-03-29T09:11:43.168276+0000","last_active":"2024-03-29T09:11:45.499230+0000","last_peered":"2024-03-29T09:11:45.499230+0000","last_clean":"2024-03-29T09:11:45.499230+0000","last_became_active":"2024-03-29T09:11:43.168028+0000","last_became_peered":"2024-03-29T09:11:43.168028+0000","last_unstale":"2024-03-29T09:11:45.499230+0000","last_undegraded":"2024-03-29T09:11:45.499230+0000","last_fullsized":"2024-03-29T09:11:45.499230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:48.069355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996656+0000","last_change":"2024-03-29T09:11:43.691607+0000","last_active":"2024-03-29T09:11:44.996656+0000","last_peered":"2024-03-29T09:11:44.996656+0000","last_clean":"2024-03-29T09:11:44.996656+0000","last_became_active":"2024-03-29T09:11:43.691054+0000","last_became_peered":"2024-03-29T09:11:43.691054+0000","last_unstale":"2024-03-29T09:11:44.996656+0000","last_undegraded":"2024-03-29T09:11:44.996656+0000","last_fullsized":"2024-03-29T09:11:44.996656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:37.929321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667464+0000","last_change":"2024-03-29T09:11:43.172794+0000","last_active":"2024-03-29T09:11:44.667464+0000","last_peered":"2024-03-29T09:11:44.667464+0000","last_clean":"2024-03-29T09:11:44.667464+0000","last_became_active":"2024-03-29T09:11:43.172062+0000","last_became_peered":"2024-03-29T09:11:43.172062+0000","last_unstale":"2024-03-29T09:11:44.667464+0000","last_undegraded":"2024-03-29T09:11:44.667464+0000","last_fullsized":"2024-03-29T09:11:44.667464+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:40:54.283138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496997+0000","last_change":"2024-03-29T09:11:43.685364+0000","last_active":"2024-03-29T09:11:45.496997+0000","last_peered":"2024-03-29T09:11:45.496997+0000","last_clean":"2024-03-29T09:11:45.496997+0000","last_became_active":"2024-03-29T09:11:43.685091+0000","last_became_peered":"2024-03-29T09:11:43.685091+0000","last_unstale":"2024-03-29T09:11:45.496997+0000","last_undegraded":"2024-03-29T09:11:45.496997+0000","last_fullsized":"2024-03-29T09:11:45.496997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:54.749065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996552+0000","last_change":"2024-03-29T09:11:43.173098+0000","last_active":"2024-03-29T09:11:44.996552+0000","last_peered":"2024-03-29T09:11:44.996552+0000","last_clean":"2024-03-29T09:11:44.996552+0000","last_became_active":"2024-03-29T09:11:43.172846+0000","last_became_peered":"2024-03-29T09:11:43.172846+0000","last_unstale":"2024-03-29T09:11:44.996552+0000","last_undegraded":"2024-03-29T09:11:44.996552+0000","last_fullsized":"2024-03-29T09:11:44.996552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:47.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996414+0000","last_change":"2024-03-29T09:11:43.692247+0000","last_active":"2024-03-29T09:11:44.996414+0000","last_peered":"2024-03-29T09:11:44.996414+0000","last_clean":"2024-03-29T09:11:44.996414+0000","last_became_active":"2024-03-29T09:11:43.691491+0000","last_became_peered":"2024-03-29T09:11:43.691491+0000","last_unstale":"2024-03-29T09:11:44.996414+0000","last_undegraded":"2024-03-29T09:11:44.996414+0000","last_fullsized":"2024-03-29T09:11:44.996414+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.953214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497899+0000","last_change":"2024-03-29T09:11:43.177133+0000","last_active":"2024-03-29T09:11:45.497899+0000","last_peered":"2024-03-29T09:11:45.497899+0000","last_clean":"2024-03-29T09:11:45.497899+0000","last_became_active":"2024-03-29T09:11:43.176799+0000","last_became_peered":"2024-03-29T09:11:43.176799+0000","last_unstale":"2024-03-29T09:11:45.497899+0000","last_undegraded":"2024-03-29T09:11:45.497899+0000","last_fullsized":"2024-03-29T09:11:45.497899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:45:07.789102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497028+0000","last_change":"2024-03-29T09:11:43.683820+0000","last_active":"2024-03-29T09:11:45.497028+0000","last_peered":"2024-03-29T09:11:45.497028+0000","last_clean":"2024-03-29T09:11:45.497028+0000","last_became_active":"2024-03-29T09:11:43.683490+0000","last_became_peered":"2024-03-29T09:11:43.683490+0000","last_unstale":"2024-03-29T09:11:45.497028+0000","last_undegraded":"2024-03-29T09:11:45.497028+0000","last_fullsized":"2024-03-29T09:11:45.497028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:03.217442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497615+0000","last_change":"2024-03-29T09:11:43.180832+0000","last_active":"2024-03-29T09:11:45.497615+0000","last_peered":"2024-03-29T09:11:45.497615+0000","last_clean":"2024-03-29T09:11:45.497615+0000","last_became_active":"2024-03-29T09:11:43.180518+0000","last_became_peered":"2024-03-29T09:11:43.180518+0000","last_unstale":"2024-03-29T09:11:45.497615+0000","last_undegraded":"2024-03-29T09:11:45.497615+0000","last_fullsized":"2024-03-29T09:11:45.497615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:24:19.226534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498945+0000","last_change":"2024-03-29T09:11:43.688382+0000","last_active":"2024-03-29T09:11:45.498945+0000","last_peered":"2024-03-29T09:11:45.498945+0000","last_clean":"2024-03-29T09:11:45.498945+0000","last_became_active":"2024-03-29T09:11:43.688094+0000","last_became_peered":"2024-03-29T09:11:43.688094+0000","last_unstale":"2024-03-29T09:11:45.498945+0000","last_undegraded":"2024-03-29T09:11:45.498945+0000","last_fullsized":"2024-03-29T09:11:45.498945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:21.218363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996277+0000","last_change":"2024-03-29T09:11:43.183057+0000","last_active":"2024-03-29T09:11:44.996277+0000","last_peered":"2024-03-29T09:11:44.996277+0000","last_clean":"2024-03-29T09:11:44.996277+0000","last_became_active":"2024-03-29T09:11:43.182589+0000","last_became_peered":"2024-03-29T09:11:43.182589+0000","last_unstale":"2024-03-29T09:11:44.996277+0000","last_undegraded":"2024-03-29T09:11:44.996277+0000","last_fullsized":"2024-03-29T09:11:44.996277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:01.878804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496947+0000","last_change":"2024-03-29T09:11:43.676496+0000","last_active":"2024-03-29T09:11:45.496947+0000","last_peered":"2024-03-29T09:11:45.496947+0000","last_clean":"2024-03-29T09:11:45.496947+0000","last_became_active":"2024-03-29T09:11:43.676208+0000","last_became_peered":"2024-03-29T09:11:43.676208+0000","last_unstale":"2024-03-29T09:11:45.496947+0000","last_undegraded":"2024-03-29T09:11:45.496947+0000","last_fullsized":"2024-03-29T09:11:45.496947+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:16.327235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497050+0000","last_change":"2024-03-29T09:11:43.173125+0000","last_active":"2024-03-29T09:11:45.497050+0000","last_peered":"2024-03-29T09:11:45.497050+0000","last_clean":"2024-03-29T09:11:45.497050+0000","last_became_active":"2024-03-29T09:11:43.172897+0000","last_became_peered":"2024-03-29T09:11:43.172897+0000","last_unstale":"2024-03-29T09:11:45.497050+0000","last_undegraded":"2024-03-29T09:11:45.497050+0000","last_fullsized":"2024-03-29T09:11:45.497050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:51.992855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996071+0000","last_change":"2024-03-29T09:11:43.681720+0000","last_active":"2024-03-29T09:11:44.996071+0000","last_peered":"2024-03-29T09:11:44.996071+0000","last_clean":"2024-03-29T09:11:44.996071+0000","last_became_active":"2024-03-29T09:11:43.681366+0000","last_became_peered":"2024-03-29T09:11:43.681366+0000","last_unstale":"2024-03-29T09:11:44.996071+0000","last_undegraded":"2024-03-29T09:11:44.996071+0000","last_fullsized":"2024-03-29T09:11:44.996071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:34.234919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667404+0000","last_change":"2024-03-29T09:11:43.176057+0000","last_active":"2024-03-29T09:11:44.667404+0000","last_peered":"2024-03-29T09:11:44.667404+0000","last_clean":"2024-03-29T09:11:44.667404+0000","last_became_active":"2024-03-29T09:11:43.175783+0000","last_became_peered":"2024-03-29T09:11:43.175783+0000","last_unstale":"2024-03-29T09:11:44.667404+0000","last_undegraded":"2024-03-29T09:11:44.667404+0000","last_fullsized":"2024-03-29T09:11:44.667404+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:31.151934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667835+0000","last_change":"2024-03-29T09:11:43.681530+0000","last_active":"2024-03-29T09:11:44.667835+0000","last_peered":"2024-03-29T09:11:44.667835+0000","last_clean":"2024-03-29T09:11:44.667835+0000","last_became_active":"2024-03-29T09:11:43.679577+0000","last_became_peered":"2024-03-29T09:11:43.679577+0000","last_unstale":"2024-03-29T09:11:44.667835+0000","last_undegraded":"2024-03-29T09:11:44.667835+0000","last_fullsized":"2024-03-29T09:11:44.667835+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:32.530881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497211+0000","last_change":"2024-03-29T09:11:43.165270+0000","last_active":"2024-03-29T09:11:45.497211+0000","last_peered":"2024-03-29T09:11:45.497211+0000","last_clean":"2024-03-29T09:11:45.497211+0000","last_became_active":"2024-03-29T09:11:43.164467+0000","last_became_peered":"2024-03-29T09:11:43.164467+0000","last_unstale":"2024-03-29T09:11:45.497211+0000","last_undegraded":"2024-03-29T09:11:45.497211+0000","last_fullsized":"2024-03-29T09:11:45.497211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:29.110475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497883+0000","last_change":"2024-03-29T09:11:43.666694+0000","last_active":"2024-03-29T09:11:45.497883+0000","last_peered":"2024-03-29T09:11:45.497883+0000","last_clean":"2024-03-29T09:11:45.497883+0000","last_became_active":"2024-03-29T09:11:43.666396+0000","last_became_peered":"2024-03-29T09:11:43.666396+0000","last_unstale":"2024-03-29T09:11:45.497883+0000","last_undegraded":"2024-03-29T09:11:45.497883+0000","last_fullsized":"2024-03-29T09:11:45.497883+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:20.115771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497219+0000","last_change":"2024-03-29T09:11:43.173205+0000","last_active":"2024-03-29T09:11:45.497219+0000","last_peered":"2024-03-29T09:11:45.497219+0000","last_clean":"2024-03-29T09:11:45.497219+0000","last_became_active":"2024-03-29T09:11:43.172920+0000","last_became_peered":"2024-03-29T09:11:43.172920+0000","last_unstale":"2024-03-29T09:11:45.497219+0000","last_undegraded":"2024-03-29T09:11:45.497219+0000","last_fullsized":"2024-03-29T09:11:45.497219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:33.491702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497202+0000","last_change":"2024-03-29T09:11:43.685373+0000","last_active":"2024-03-29T09:11:45.497202+0000","last_peered":"2024-03-29T09:11:45.497202+0000","last_clean":"2024-03-29T09:11:45.497202+0000","last_became_active":"2024-03-29T09:11:43.685092+0000","last_became_peered":"2024-03-29T09:11:43.685092+0000","last_unstale":"2024-03-29T09:11:45.497202+0000","last_undegraded":"2024-03-29T09:11:45.497202+0000","last_fullsized":"2024-03-29T09:11:45.497202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498123+0000","last_change":"2024-03-29T09:11:43.181613+0000","last_active":"2024-03-29T09:11:45.498123+0000","last_peered":"2024-03-29T09:11:45.498123+0000","last_clean":"2024-03-29T09:11:45.498123+0000","last_became_active":"2024-03-29T09:11:43.181326+0000","last_became_peered":"2024-03-29T09:11:43.181326+0000","last_unstale":"2024-03-29T09:11:45.498123+0000","last_undegraded":"2024-03-29T09:11:45.498123+0000","last_fullsized":"2024-03-29T09:11:45.498123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:12.031715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497714+0000","last_change":"2024-03-29T09:11:43.675831+0000","last_active":"2024-03-29T09:11:45.497714+0000","last_peered":"2024-03-29T09:11:45.497714+0000","last_clean":"2024-03-29T09:11:45.497714+0000","last_became_active":"2024-03-29T09:11:43.675418+0000","last_became_peered":"2024-03-29T09:11:43.675418+0000","last_unstale":"2024-03-29T09:11:45.497714+0000","last_undegraded":"2024-03-29T09:11:45.497714+0000","last_fullsized":"2024-03-29T09:11:45.497714+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:06:28.079801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667829+0000","last_change":"2024-03-29T09:11:43.181637+0000","last_active":"2024-03-29T09:11:44.667829+0000","last_peered":"2024-03-29T09:11:44.667829+0000","last_clean":"2024-03-29T09:11:44.667829+0000","last_became_active":"2024-03-29T09:11:43.181327+0000","last_became_peered":"2024-03-29T09:11:43.181327+0000","last_unstale":"2024-03-29T09:11:44.667829+0000","last_undegraded":"2024-03-29T09:11:44.667829+0000","last_fullsized":"2024-03-29T09:11:44.667829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:34.000096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497070+0000","last_change":"2024-03-29T09:11:43.677020+0000","last_active":"2024-03-29T09:11:45.497070+0000","last_peered":"2024-03-29T09:11:45.497070+0000","last_clean":"2024-03-29T09:11:45.497070+0000","last_became_active":"2024-03-29T09:11:43.676671+0000","last_became_peered":"2024-03-29T09:11:43.676671+0000","last_unstale":"2024-03-29T09:11:45.497070+0000","last_undegraded":"2024-03-29T09:11:45.497070+0000","last_fullsized":"2024-03-29T09:11:45.497070+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:00:54.297415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497048+0000","last_change":"2024-03-29T09:11:43.176006+0000","last_active":"2024-03-29T09:11:45.497048+0000","last_peered":"2024-03-29T09:11:45.497048+0000","last_clean":"2024-03-29T09:11:45.497048+0000","last_became_active":"2024-03-29T09:11:43.175716+0000","last_became_peered":"2024-03-29T09:11:43.175716+0000","last_unstale":"2024-03-29T09:11:45.497048+0000","last_undegraded":"2024-03-29T09:11:45.497048+0000","last_fullsized":"2024-03-29T09:11:45.497048+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:47.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497524+0000","last_change":"2024-03-29T09:11:43.673170+0000","last_active":"2024-03-29T09:11:45.497524+0000","last_peered":"2024-03-29T09:11:45.497524+0000","last_clean":"2024-03-29T09:11:45.497524+0000","last_became_active":"2024-03-29T09:11:43.672886+0000","last_became_peered":"2024-03-29T09:11:43.672886+0000","last_unstale":"2024-03-29T09:11:45.497524+0000","last_undegraded":"2024-03-29T09:11:45.497524+0000","last_fullsized":"2024-03-29T09:11:45.497524+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:16.721779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498004+0000","last_change":"2024-03-29T09:11:43.177733+0000","last_active":"2024-03-29T09:11:45.498004+0000","last_peered":"2024-03-29T09:11:45.498004+0000","last_clean":"2024-03-29T09:11:45.498004+0000","last_became_active":"2024-03-29T09:11:43.177447+0000","last_became_peered":"2024-03-29T09:11:43.177447+0000","last_unstale":"2024-03-29T09:11:45.498004+0000","last_undegraded":"2024-03-29T09:11:45.498004+0000","last_fullsized":"2024-03-29T09:11:45.498004+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:31:51.585476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496889+0000","last_change":"2024-03-29T09:11:43.681087+0000","last_active":"2024-03-29T09:11:45.496889+0000","last_peered":"2024-03-29T09:11:45.496889+0000","last_clean":"2024-03-29T09:11:45.496889+0000","last_became_active":"2024-03-29T09:11:43.680814+0000","last_became_peered":"2024-03-29T09:11:43.680814+0000","last_unstale":"2024-03-29T09:11:45.496889+0000","last_undegraded":"2024-03-29T09:11:45.496889+0000","last_fullsized":"2024-03-29T09:11:45.496889+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:37.448155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498207+0000","last_change":"2024-03-29T09:11:43.170448+0000","last_active":"2024-03-29T09:11:45.498207+0000","last_peered":"2024-03-29T09:11:45.498207+0000","last_clean":"2024-03-29T09:11:45.498207+0000","last_became_active":"2024-03-29T09:11:43.170170+0000","last_became_peered":"2024-03-29T09:11:43.170170+0000","last_unstale":"2024-03-29T09:11:45.498207+0000","last_undegraded":"2024-03-29T09:11:45.498207+0000","last_fullsized":"2024-03-29T09:11:45.498207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:21.777400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498121+0000","last_change":"2024-03-29T09:11:43.679100+0000","last_active":"2024-03-29T09:11:45.498121+0000","last_peered":"2024-03-29T09:11:45.498121+0000","last_clean":"2024-03-29T09:11:45.498121+0000","last_became_active":"2024-03-29T09:11:43.678319+0000","last_became_peered":"2024-03-29T09:11:43.678319+0000","last_unstale":"2024-03-29T09:11:45.498121+0000","last_undegraded":"2024-03-29T09:11:45.498121+0000","last_fullsized":"2024-03-29T09:11:45.498121+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:58:28.504247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":19,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497725+0000","last_change":"2024-03-29T09:11:43.173897+0000","last_active":"2024-03-29T09:11:45.497725+0000","last_peered":"2024-03-29T09:11:45.497725+0000","last_clean":"2024-03-29T09:11:45.497725+0000","last_became_active":"2024-03-29T09:11:43.172759+0000","last_became_peered":"2024-03-29T09:11:43.172759+0000","last_unstale":"2024-03-29T09:11:45.497725+0000","last_undegraded":"2024-03-29T09:11:45.497725+0000","last_fullsized":"2024-03-29T09:11:45.497725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.415296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497652+0000","last_change":"2024-03-29T09:11:43.681216+0000","last_active":"2024-03-29T09:11:45.497652+0000","last_peered":"2024-03-29T09:11:45.497652+0000","last_clean":"2024-03-29T09:11:45.497652+0000","last_became_active":"2024-03-29T09:11:43.680966+0000","last_became_peered":"2024-03-29T09:11:43.680966+0000","last_unstale":"2024-03-29T09:11:45.497652+0000","last_undegraded":"2024-03-29T09:11:45.497652+0000","last_fullsized":"2024-03-29T09:11:45.497652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:06.103995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667467+0000","last_change":"2024-03-29T09:11:43.183881+0000","last_active":"2024-03-29T09:11:44.667467+0000","last_peered":"2024-03-29T09:11:44.667467+0000","last_clean":"2024-03-29T09:11:44.667467+0000","last_became_active":"2024-03-29T09:11:43.183567+0000","last_became_peered":"2024-03-29T09:11:43.183567+0000","last_unstale":"2024-03-29T09:11:44.667467+0000","last_undegraded":"2024-03-29T09:11:44.667467+0000","last_fullsized":"2024-03-29T09:11:44.667467+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:44.603438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498039+0000","last_change":"2024-03-29T09:11:43.665968+0000","last_active":"2024-03-29T09:11:45.498039+0000","last_peered":"2024-03-29T09:11:45.498039+0000","last_clean":"2024-03-29T09:11:45.498039+0000","last_became_active":"2024-03-29T09:11:43.665714+0000","last_became_peered":"2024-03-29T09:11:43.665714+0000","last_unstale":"2024-03-29T09:11:45.498039+0000","last_undegraded":"2024-03-29T09:11:45.498039+0000","last_fullsized":"2024-03-29T09:11:45.498039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:21.218363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497372+0000","last_change":"2024-03-29T09:11:43.173582+0000","last_active":"2024-03-29T09:11:45.497372+0000","last_peered":"2024-03-29T09:11:45.497372+0000","last_clean":"2024-03-29T09:11:45.497372+0000","last_became_active":"2024-03-29T09:11:43.173300+0000","last_became_peered":"2024-03-29T09:11:43.173300+0000","last_unstale":"2024-03-29T09:11:45.497372+0000","last_undegraded":"2024-03-29T09:11:45.497372+0000","last_fullsized":"2024-03-29T09:11:45.497372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:50.765164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.666945+0000","last_change":"2024-03-29T09:11:43.685214+0000","last_active":"2024-03-29T09:11:44.666945+0000","last_peered":"2024-03-29T09:11:44.666945+0000","last_clean":"2024-03-29T09:11:44.666945+0000","last_became_active":"2024-03-29T09:11:43.684951+0000","last_became_peered":"2024-03-29T09:11:43.684951+0000","last_unstale":"2024-03-29T09:11:44.666945+0000","last_undegraded":"2024-03-29T09:11:44.666945+0000","last_fullsized":"2024-03-29T09:11:44.666945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:21.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497581+0000","last_change":"2024-03-29T09:11:43.171045+0000","last_active":"2024-03-29T09:11:45.497581+0000","last_peered":"2024-03-29T09:11:45.497581+0000","last_clean":"2024-03-29T09:11:45.497581+0000","last_became_active":"2024-03-29T09:11:43.170833+0000","last_became_peered":"2024-03-29T09:11:43.170833+0000","last_unstale":"2024-03-29T09:11:45.497581+0000","last_undegraded":"2024-03-29T09:11:45.497581+0000","last_fullsized":"2024-03-29T09:11:45.497581+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:28.792665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497822+0000","last_change":"2024-03-29T09:11:43.682084+0000","last_active":"2024-03-29T09:11:45.497822+0000","last_peered":"2024-03-29T09:11:45.497822+0000","last_clean":"2024-03-29T09:11:45.497822+0000","last_became_active":"2024-03-29T09:11:43.681789+0000","last_became_peered":"2024-03-29T09:11:43.681789+0000","last_unstale":"2024-03-29T09:11:45.497822+0000","last_undegraded":"2024-03-29T09:11:45.497822+0000","last_fullsized":"2024-03-29T09:11:45.497822+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:41:59.385726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497200+0000","last_change":"2024-03-29T09:11:43.179870+0000","last_active":"2024-03-29T09:11:45.497200+0000","last_peered":"2024-03-29T09:11:45.497200+0000","last_clean":"2024-03-29T09:11:45.497200+0000","last_became_active":"2024-03-29T09:11:43.179590+0000","last_became_peered":"2024-03-29T09:11:43.179590+0000","last_unstale":"2024-03-29T09:11:45.497200+0000","last_undegraded":"2024-03-29T09:11:45.497200+0000","last_fullsized":"2024-03-29T09:11:45.497200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:13:28.738464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497186+0000","last_change":"2024-03-29T09:11:43.684574+0000","last_active":"2024-03-29T09:11:45.497186+0000","last_peered":"2024-03-29T09:11:45.497186+0000","last_clean":"2024-03-29T09:11:45.497186+0000","last_became_active":"2024-03-29T09:11:43.684273+0000","last_became_peered":"2024-03-29T09:11:43.684273+0000","last_unstale":"2024-03-29T09:11:45.497186+0000","last_undegraded":"2024-03-29T09:11:45.497186+0000","last_fullsized":"2024-03-29T09:11:45.497186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:21.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499015+0000","last_change":"2024-03-29T09:11:43.177261+0000","last_active":"2024-03-29T09:11:45.499015+0000","last_peered":"2024-03-29T09:11:45.499015+0000","last_clean":"2024-03-29T09:11:45.499015+0000","last_became_active":"2024-03-29T09:11:43.176994+0000","last_became_peered":"2024-03-29T09:11:43.176994+0000","last_unstale":"2024-03-29T09:11:45.499015+0000","last_undegraded":"2024-03-29T09:11:45.499015+0000","last_fullsized":"2024-03-29T09:11:45.499015+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:33.227274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497092+0000","last_change":"2024-03-29T09:11:43.677761+0000","last_active":"2024-03-29T09:11:45.497092+0000","last_peered":"2024-03-29T09:11:45.497092+0000","last_clean":"2024-03-29T09:11:45.497092+0000","last_became_active":"2024-03-29T09:11:43.677445+0000","last_became_peered":"2024-03-29T09:11:43.677445+0000","last_unstale":"2024-03-29T09:11:45.497092+0000","last_undegraded":"2024-03-29T09:11:45.497092+0000","last_fullsized":"2024-03-29T09:11:45.497092+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:34.234919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496977+0000","last_change":"2024-03-29T09:11:43.174858+0000","last_active":"2024-03-29T09:11:45.496977+0000","last_peered":"2024-03-29T09:11:45.496977+0000","last_clean":"2024-03-29T09:11:45.496977+0000","last_became_active":"2024-03-29T09:11:43.174559+0000","last_became_peered":"2024-03-29T09:11:43.174559+0000","last_unstale":"2024-03-29T09:11:45.496977+0000","last_undegraded":"2024-03-29T09:11:45.496977+0000","last_fullsized":"2024-03-29T09:11:45.496977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:01.878804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496861+0000","last_change":"2024-03-29T09:11:43.677801+0000","last_active":"2024-03-29T09:11:45.496861+0000","last_peered":"2024-03-29T09:11:45.496861+0000","last_clean":"2024-03-29T09:11:45.496861+0000","last_became_active":"2024-03-29T09:11:43.677410+0000","last_became_peered":"2024-03-29T09:11:43.677410+0000","last_unstale":"2024-03-29T09:11:45.496861+0000","last_undegraded":"2024-03-29T09:11:45.496861+0000","last_fullsized":"2024-03-29T09:11:45.496861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497264+0000","last_change":"2024-03-29T09:11:43.178139+0000","last_active":"2024-03-29T09:11:45.497264+0000","last_peered":"2024-03-29T09:11:45.497264+0000","last_clean":"2024-03-29T09:11:45.497264+0000","last_became_active":"2024-03-29T09:11:43.176620+0000","last_became_peered":"2024-03-29T09:11:43.176620+0000","last_unstale":"2024-03-29T09:11:45.497264+0000","last_undegraded":"2024-03-29T09:11:45.497264+0000","last_fullsized":"2024-03-29T09:11:45.497264+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:44.603438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497741+0000","last_change":"2024-03-29T09:11:43.666170+0000","last_active":"2024-03-29T09:11:45.497741+0000","last_peered":"2024-03-29T09:11:45.497741+0000","last_clean":"2024-03-29T09:11:45.497741+0000","last_became_active":"2024-03-29T09:11:43.665889+0000","last_became_peered":"2024-03-29T09:11:43.665889+0000","last_unstale":"2024-03-29T09:11:45.497741+0000","last_undegraded":"2024-03-29T09:11:45.497741+0000","last_fullsized":"2024-03-29T09:11:45.497741+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667507+0000","last_change":"2024-03-29T09:11:43.176436+0000","last_active":"2024-03-29T09:11:44.667507+0000","last_peered":"2024-03-29T09:11:44.667507+0000","last_clean":"2024-03-29T09:11:44.667507+0000","last_became_active":"2024-03-29T09:11:43.176164+0000","last_became_peered":"2024-03-29T09:11:43.176164+0000","last_unstale":"2024-03-29T09:11:44.667507+0000","last_undegraded":"2024-03-29T09:11:44.667507+0000","last_fullsized":"2024-03-29T09:11:44.667507+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:54:51.708243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497972+0000","last_change":"2024-03-29T09:11:43.677579+0000","last_active":"2024-03-29T09:11:45.497972+0000","last_peered":"2024-03-29T09:11:45.497972+0000","last_clean":"2024-03-29T09:11:45.497972+0000","last_became_active":"2024-03-29T09:11:43.676104+0000","last_became_peered":"2024-03-29T09:11:43.676104+0000","last_unstale":"2024-03-29T09:11:45.497972+0000","last_undegraded":"2024-03-29T09:11:45.497972+0000","last_fullsized":"2024-03-29T09:11:45.497972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:26.946110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996502+0000","last_change":"2024-03-29T09:11:43.182966+0000","last_active":"2024-03-29T09:11:44.996502+0000","last_peered":"2024-03-29T09:11:44.996502+0000","last_clean":"2024-03-29T09:11:44.996502+0000","last_became_active":"2024-03-29T09:11:43.182644+0000","last_became_peered":"2024-03-29T09:11:43.182644+0000","last_unstale":"2024-03-29T09:11:44.996502+0000","last_undegraded":"2024-03-29T09:11:44.996502+0000","last_fullsized":"2024-03-29T09:11:44.996502+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:36.913030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667050+0000","last_change":"2024-03-29T09:11:43.676032+0000","last_active":"2024-03-29T09:11:44.667050+0000","last_peered":"2024-03-29T09:11:44.667050+0000","last_clean":"2024-03-29T09:11:44.667050+0000","last_became_active":"2024-03-29T09:11:43.675680+0000","last_became_peered":"2024-03-29T09:11:43.675680+0000","last_unstale":"2024-03-29T09:11:44.667050+0000","last_undegraded":"2024-03-29T09:11:44.667050+0000","last_fullsized":"2024-03-29T09:11:44.667050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:04:14.657461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497378+0000","last_change":"2024-03-29T09:11:43.171894+0000","last_active":"2024-03-29T09:11:45.497378+0000","last_peered":"2024-03-29T09:11:45.497378+0000","last_clean":"2024-03-29T09:11:45.497378+0000","last_became_active":"2024-03-29T09:11:43.171636+0000","last_became_peered":"2024-03-29T09:11:43.171636+0000","last_unstale":"2024-03-29T09:11:45.497378+0000","last_undegraded":"2024-03-29T09:11:45.497378+0000","last_fullsized":"2024-03-29T09:11:45.497378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:27.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497284+0000","last_change":"2024-03-29T09:11:43.678609+0000","last_active":"2024-03-29T09:11:45.497284+0000","last_peered":"2024-03-29T09:11:45.497284+0000","last_clean":"2024-03-29T09:11:45.497284+0000","last_became_active":"2024-03-29T09:11:43.678256+0000","last_became_peered":"2024-03-29T09:11:43.678256+0000","last_unstale":"2024-03-29T09:11:45.497284+0000","last_undegraded":"2024-03-29T09:11:45.497284+0000","last_fullsized":"2024-03-29T09:11:45.497284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:23.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497444+0000","last_change":"2024-03-29T09:11:43.175598+0000","last_active":"2024-03-29T09:11:45.497444+0000","last_peered":"2024-03-29T09:11:45.497444+0000","last_clean":"2024-03-29T09:11:45.497444+0000","last_became_active":"2024-03-29T09:11:43.175354+0000","last_became_peered":"2024-03-29T09:11:43.175354+0000","last_unstale":"2024-03-29T09:11:45.497444+0000","last_undegraded":"2024-03-29T09:11:45.497444+0000","last_fullsized":"2024-03-29T09:11:45.497444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:23:19.175144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996580+0000","last_change":"2024-03-29T09:11:43.691405+0000","last_active":"2024-03-29T09:11:44.996580+0000","last_peered":"2024-03-29T09:11:44.996580+0000","last_clean":"2024-03-29T09:11:44.996580+0000","last_became_active":"2024-03-29T09:11:43.691072+0000","last_became_peered":"2024-03-29T09:11:43.691072+0000","last_unstale":"2024-03-29T09:11:44.996580+0000","last_undegraded":"2024-03-29T09:11:44.996580+0000","last_fullsized":"2024-03-29T09:11:44.996580+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:32:46.071316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667321+0000","last_change":"2024-03-29T09:11:43.179767+0000","last_active":"2024-03-29T09:11:44.667321+0000","last_peered":"2024-03-29T09:11:44.667321+0000","last_clean":"2024-03-29T09:11:44.667321+0000","last_became_active":"2024-03-29T09:11:43.179487+0000","last_became_peered":"2024-03-29T09:11:43.179487+0000","last_unstale":"2024-03-29T09:11:44.667321+0000","last_undegraded":"2024-03-29T09:11:44.667321+0000","last_fullsized":"2024-03-29T09:11:44.667321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:27.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667726+0000","last_change":"2024-03-29T09:11:43.687900+0000","last_active":"2024-03-29T09:11:44.667726+0000","last_peered":"2024-03-29T09:11:44.667726+0000","last_clean":"2024-03-29T09:11:44.667726+0000","last_became_active":"2024-03-29T09:11:43.687446+0000","last_became_peered":"2024-03-29T09:11:43.687446+0000","last_unstale":"2024-03-29T09:11:44.667726+0000","last_undegraded":"2024-03-29T09:11:44.667726+0000","last_fullsized":"2024-03-29T09:11:44.667726+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:51.772809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497118+0000","last_change":"2024-03-29T09:11:43.175422+0000","last_active":"2024-03-29T09:11:45.497118+0000","last_peered":"2024-03-29T09:11:45.497118+0000","last_clean":"2024-03-29T09:11:45.497118+0000","last_became_active":"2024-03-29T09:11:43.175145+0000","last_became_peered":"2024-03-29T09:11:43.175145+0000","last_unstale":"2024-03-29T09:11:45.497118+0000","last_undegraded":"2024-03-29T09:11:45.497118+0000","last_fullsized":"2024-03-29T09:11:45.497118+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:04.086596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496735+0000","last_change":"2024-03-29T09:11:43.683642+0000","last_active":"2024-03-29T09:11:45.496735+0000","last_peered":"2024-03-29T09:11:45.496735+0000","last_clean":"2024-03-29T09:11:45.496735+0000","last_became_active":"2024-03-29T09:11:43.680296+0000","last_became_peered":"2024-03-29T09:11:43.680296+0000","last_unstale":"2024-03-29T09:11:45.496735+0000","last_undegraded":"2024-03-29T09:11:45.496735+0000","last_fullsized":"2024-03-29T09:11:45.496735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:06.103995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996134+0000","last_change":"2024-03-29T09:11:43.177125+0000","last_active":"2024-03-29T09:11:44.996134+0000","last_peered":"2024-03-29T09:11:44.996134+0000","last_clean":"2024-03-29T09:11:44.996134+0000","last_became_active":"2024-03-29T09:11:43.176802+0000","last_became_peered":"2024-03-29T09:11:43.176802+0000","last_unstale":"2024-03-29T09:11:44.996134+0000","last_undegraded":"2024-03-29T09:11:44.996134+0000","last_fullsized":"2024-03-29T09:11:44.996134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:31.523236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667170+0000","last_change":"2024-03-29T09:11:43.675090+0000","last_active":"2024-03-29T09:11:44.667170+0000","last_peered":"2024-03-29T09:11:44.667170+0000","last_clean":"2024-03-29T09:11:44.667170+0000","last_became_active":"2024-03-29T09:11:43.674631+0000","last_became_peered":"2024-03-29T09:11:43.674631+0000","last_unstale":"2024-03-29T09:11:44.667170+0000","last_undegraded":"2024-03-29T09:11:44.667170+0000","last_fullsized":"2024-03-29T09:11:44.667170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:00:54.297415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496892+0000","last_change":"2024-03-29T09:11:43.174863+0000","last_active":"2024-03-29T09:11:45.496892+0000","last_peered":"2024-03-29T09:11:45.496892+0000","last_clean":"2024-03-29T09:11:45.496892+0000","last_became_active":"2024-03-29T09:11:43.174568+0000","last_became_peered":"2024-03-29T09:11:43.174568+0000","last_unstale":"2024-03-29T09:11:45.496892+0000","last_undegraded":"2024-03-29T09:11:45.496892+0000","last_fullsized":"2024-03-29T09:11:45.496892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:48:29.840684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497110+0000","last_change":"2024-03-29T09:11:43.667721+0000","last_active":"2024-03-29T09:11:45.497110+0000","last_peered":"2024-03-29T09:11:45.497110+0000","last_clean":"2024-03-29T09:11:45.497110+0000","last_became_active":"2024-03-29T09:11:43.667442+0000","last_became_peered":"2024-03-29T09:11:43.667442+0000","last_unstale":"2024-03-29T09:11:45.497110+0000","last_undegraded":"2024-03-29T09:11:45.497110+0000","last_fullsized":"2024-03-29T09:11:45.497110+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:37.929321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667517+0000","last_change":"2024-03-29T09:11:43.181790+0000","last_active":"2024-03-29T09:11:44.667517+0000","last_peered":"2024-03-29T09:11:44.667517+0000","last_clean":"2024-03-29T09:11:44.667517+0000","last_became_active":"2024-03-29T09:11:43.181311+0000","last_became_peered":"2024-03-29T09:11:43.181311+0000","last_unstale":"2024-03-29T09:11:44.667517+0000","last_undegraded":"2024-03-29T09:11:44.667517+0000","last_fullsized":"2024-03-29T09:11:44.667517+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:06:19.992268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-29T09:11:43.663437+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:11:43.663437+0000","last_peered":"2024-03-29T09:11:43.663437+0000","last_clean":"2024-03-29T09:11:43.663437+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:11:43.663437+0000","last_undegraded":"2024-03-29T09:11:43.663437+0000","last_fullsized":"2024-03-29T09:11:43.663437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":293965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.498},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.49399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93799999999999994,"5min":1.639,"15min":1.639},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.48599999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.83099999999999996,"5min":1.992,"15min":1.992},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.81699999999999995,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.92000000000000004,"5min":2.883,"15min":2.883},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.51800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80400000000000005,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91600000000000004,"5min":1.387,"15min":1.387},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.54400000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94499999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0589999999999999,"5min":1.367,"15min":1.367},"last":0.873}]},{"osd":5,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0209999999999999,"5min":1.458,"15min":1.458},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72499999999999998,"15min":0.73799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.502,"15min":0.502},"max":{"1min":0.96299999999999997,"5min":1.413,"15min":1.413},"last":0.72799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":293895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.97799999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.84799999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.81200000000000006,"15min":0.83799999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.96099999999999997,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.91000000000000003}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.48999999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.45800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.74199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.745}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.79500000000000004,"15min":0.80200000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.006,"5min":1.026,"15min":1.026},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":884823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.175,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.5259999999999998,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.64900000000000002,"15min":0.68300000000000005},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.57199999999999995,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66700000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.008,"15min":0.94899999999999995},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":12.356,"5min":12.356,"15min":12.356},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.73899999999999999,"15min":0.71499999999999997},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.91700000000000004,"15min":0.93799999999999994},"min":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6279999999999999,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83399999999999996,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.903,"5min":1.99,"15min":1.99},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.0609999999999999,"15min":0.99399999999999999},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":12.669,"5min":12.669,"15min":12.669},"last":0.627}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.874,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.75700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":295437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.51300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.51700000000000002,"5min":0.751,"15min":0.751},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.436},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67800000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":1.028,"15min":1.028},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99099999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.748}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.069,"15min":1.069},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.02,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.82599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":295597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.47399999999999998,"15min":0.48799999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":1.252,"15min":1.252},"last":0.47699999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61899999999999999,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66900000000000004,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.746,"5min":2.415,"15min":2.415},"last":0.496},{"interface":"front","average":{"1min":0.5,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":2.056,"15min":2.056},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91400000000000003,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":2.073,"15min":2.073},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92500000000000004,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95199999999999996,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.878,"15min":0.90300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96799999999999997,"5min":2.387,"15min":2.387},"last":0.628},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.91700000000000004,"15min":0.94699999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.91500000000000004,"5min":2.996,"15min":2.996},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95499999999999996,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84999999999999998,"15min":0.874},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88800000000000001,"5min":2.165,"15min":2.165},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":293963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.53200000000000003,"15min":0.55100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61599999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.496,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.53900000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76300000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99299999999999999,"5min":2.278,"15min":2.278},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96399999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.59999999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88400000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.745}]},{"osd":6,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.90000000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":2.649,"15min":2.649},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.034,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":294489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1319999999999999,"5min":1.742,"15min":1.742},"last":0.873},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.77700000000000002,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.751,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2130000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84199999999999997,"5min":0.999,"15min":0.999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91300000000000003,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.84599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.871,"15min":0.90100000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0269999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.94399999999999995,"15min":0.97599999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.073,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.72399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":884275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81799999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85499999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.872,"15min":0.872},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83699999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.91600000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":0.873},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73499999999999999,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.81100000000000005,"5min":1.05,"15min":1.05},"last":0.502}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.83099999999999996,"5min":1.079,"15min":1.079},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.877,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.755},{"interface":"front","average":{"1min":0.752,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:11:48.984 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-29T09:11:49.138 INFO:tasks.ceph.mds.c.smithi099.stderr:2024-03-29T09:11:49.139+0000 7f385a400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T09:11:49.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.208+0000 7fa517400700 1 Processor -- start 2024-03-29T09:11:49.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.208+0000 7fa517400700 1 -- start start 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa517400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 0x7fa5100ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa517400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa51015ad20 0x7fa5100b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa517400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51015b170 con 0x7fa5100b0b50 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa517400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5100b1510 con 0x7fa51015ad20 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa517400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51015b3e0 con 0x7fa51013d100 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa50fe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa5100b0b50 0x7fa51015a610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48234/0 (socket says 172.21.15.99:48234) 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa50fe00700 1 -- 172.21.15.99:0/2342588945 learned_addr learned my addr 172.21.15.99:0/2342588945 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:49.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa514a00700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 0x7fa5100ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa515400700 1 --2- 172.21.15.99:0/2342588945 >> v2:172.21.15.133:3300/0 conn(0x7fa51015ad20 0x7fa5100b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448689704 0 0) 0x7fa51015b170 con 0x7fa5100b0b50 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.210+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f80057c0 con 0x7fa5100b0b50 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 870537620 0 0) 0x7fa4f80057c0 con 0x7fa5100b0b50 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 msgr2=0x7fa5100ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 0x7fa5100ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 >> v2:172.21.15.133:3300/0 conn(0x7fa51015ad20 msgr2=0x7fa5100b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 --2- 172.21.15.99:0/2342588945 >> v2:172.21.15.133:3300/0 conn(0x7fa51015ad20 0x7fa5100b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100cdb60 con 0x7fa5100b0b50 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fa4fc001670 con 0x7fa5100b0b50 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 >> v1:172.21.15.99:6789/0 conn(0x7fa5100b0b50 legacy=0x7fa51015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 0x7fa4f8005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4f80061e0 0x7fa4f8006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa4f8007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100b1510 con 0x7fa4f8006fa0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51015b3e0 con 0x7fa4f80061e0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f800dc20 con 0x7fa51013d100 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa515400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa4f8007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa514a00700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 0x7fa4f8005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa50fe00700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4f80061e0 0x7fa4f8006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa515400700 1 -- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 msgr2=0x7fa4f8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa515400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013d100 0x7fa4f8005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa515400700 1 -- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4f80061e0 msgr2=0x7fa4f8006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa515400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4f80061e0 0x7fa4f8006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.211+0000 7fa515400700 1 -- 172.21.15.99:0/2342588945 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f80057c0 con 0x7fa4f8006fa0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa515400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa4f8007420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e170 tx=0x7fa4f8007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225db49e6283b638 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800ec90 con 0x7fa4f8006fa0 2024-03-29T09:11:49.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50800f040 con 0x7fa4f8006fa0 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa50f400700 1 -- 172.21.15.99:0/2342588945 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa508014480 con 0x7fa4f8006fa0 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa517400700 1 -- 172.21.15.99:0/2342588945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 msgr2=0x7fa4f8007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa517400700 1 --2- 172.21.15.99:0/2342588945 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa4f8007420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa50800e170 tx=0x7fa4f8007a90 comp rx=0 tx=0).stop 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa515400700 1 -- 172.21.15.99:0/2342588945 reap_dead start 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa517400700 1 -- 172.21.15.99:0/2342588945 shutdown_connections 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa517400700 1 -- 172.21.15.99:0/2342588945 >> 172.21.15.99:0/2342588945 conn(0x7fa510066540 msgr2=0x7fa510066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa517400700 1 -- 172.21.15.99:0/2342588945 shutdown_connections 2024-03-29T09:11:49.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.212+0000 7fa517400700 1 -- 172.21.15.99:0/2342588945 wait complete. 2024-03-29T09:11:49.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.215+0000 7fa517400700 1 Processor -- start 2024-03-29T09:11:49.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.215+0000 7fa517400700 1 -- start start 2024-03-29T09:11:49.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa517400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa5101fa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa517400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5101f7160 0x7fa51015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa517400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013bda0 0x7fa5100aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa517400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101fb6b0 con 0x7fa4f8006fa0 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa517400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101fae10 con 0x7fa5101f7160 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa517400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5101fb0f0 con 0x7fa51013bda0 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa5101fa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa5101fa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57666/0 (socket says 172.21.15.99:57666) 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 -- 172.21.15.99:0/479249931 learned_addr learned my addr 172.21.15.99:0/479249931 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa515400700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013bda0 0x7fa5100aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa50fe00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5101f7160 0x7fa51015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 -- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013bda0 msgr2=0x7fa5100aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013bda0 0x7fa5100aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 -- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5101f7160 msgr2=0x7fa51015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5101f7160 0x7fa51015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.217 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa514a00700 1 -- 172.21.15.99:0/479249931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5101fb3d0 con 0x7fa4f8006fa0 2024-03-29T09:11:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.216+0000 7fa50fe00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5101f7160 0x7fa51015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.217+0000 7fa514a00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa5101fa780 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa504011660 tx=0x7fa504011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.217+0000 7fa50cc00700 1 -- 172.21.15.99:0/479249931 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504015870 con 0x7fa4f8006fa0 2024-03-29T09:11:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.217+0000 7fa50cc00700 1 -- 172.21.15.99:0/479249931 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5040159d0 con 0x7fa4f8006fa0 2024-03-29T09:11:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.217+0000 7fa50cc00700 1 -- 172.21.15.99:0/479249931 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa504015d40 con 0x7fa4f8006fa0 2024-03-29T09:11:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.217+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa510154da0 con 0x7fa4f8006fa0 2024-03-29T09:11:49.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.217+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51013c410 con 0x7fa4f8006fa0 2024-03-29T09:11:49.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.218+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5100678b0 con 0x7fa4f8006fa0 2024-03-29T09:11:49.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.219+0000 7fa50cc00700 1 -- 172.21.15.99:0/479249931 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa504013400 con 0x7fa4f8006fa0 2024-03-29T09:11:49.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.220+0000 7fa50cc00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa4f006ac90 0x7fa4f006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.220+0000 7fa50cc00700 1 -- 172.21.15.99:0/479249931 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa504089860 con 0x7fa4f8006fa0 2024-03-29T09:11:49.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.220+0000 7fa50fe00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa4f006ac90 0x7fa4f006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.221+0000 7fa50fe00700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa4f006ac90 0x7fa4f006d150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc000c00 tx=0x7fa4fc000fd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:49.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.225+0000 7fa50cc00700 1 -- 172.21.15.99:0/479249931 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa5040548f0 con 0x7fa4f8006fa0 2024-03-29T09:11:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.510+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa510073510 con 0x7fa4f8006fa0 2024-03-29T09:11:49.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.511+0000 7fa50cc00700 1 -- 172.21.15.99:0/479249931 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1173 (secure 0 0 0) 0x7fa504057f10 con 0x7fa4f8006fa0 2024-03-29T09:11:49.510 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:49.510 INFO:teuthology.orchestra.run.smithi099.stdout:{"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":196,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1711703325,"num_in_osds":8,"osd_in_since":1711703316,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":227893248,"bytes_avail":772866220032,"bytes_total":773094113280,"write_bytes_sec":2389,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6105}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-03-29T09:11:36.485105+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-29T09:11:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa4f006ac90 msgr2=0x7fa4f006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa4f006ac90 0x7fa4f006d150 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc000c00 tx=0x7fa4fc000fd0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 msgr2=0x7fa5101fa780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa5101fa780 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa504011660 tx=0x7fa504011a20 comp rx=0 tx=0).stop 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 shutdown_connections 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa4f006ac90 0x7fa4f006d150 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa51013bda0 0x7fa5100aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4f8006fa0 0x7fa5101fa780 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 --2- 172.21.15.99:0/479249931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa5101f7160 0x7fa51015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 >> 172.21.15.99:0/479249931 conn(0x7fa510066540 msgr2=0x7fa51015b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 shutdown_connections 2024-03-29T09:11:49.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.513+0000 7fa517400700 1 -- 172.21.15.99:0/479249931 wait complete. 2024-03-29T09:11:49.530 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-29T09:11:49.530 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:11:49.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.721+0000 7fcd45e00700 1 Processor -- start 2024-03-29T09:11:49.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.721+0000 7fcd45e00700 1 -- start start 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd45e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcd4015e7a0 0x7fcd400d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd45e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd400cfad0 0x7fcd400cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd45e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd400cdf80 con 0x7fcd400d12f0 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd45e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400d0420 con 0x7fcd400cfad0 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd45e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4015eb80 con 0x7fcd4015e7a0 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd3e000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcd400d12f0 0x7fcd400cecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48238/0 (socket says 172.21.15.99:48238) 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd3e000700 1 -- 172.21.15.99:0/2898169307 learned_addr learned my addr 172.21.15.99:0/2898169307 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd3f400700 1 --2- 172.21.15.99:0/2898169307 >> v2:172.21.15.99:3300/0 conn(0x7fcd400cfad0 0x7fcd400cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.722+0000 7fcd3ea00700 1 --2- 172.21.15.99:0/2898169307 >> v2:172.21.15.133:3300/0 conn(0x7fcd4015e7a0 0x7fcd400d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3d600700 1 -- 172.21.15.99:0/2898169307 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287662584 0 0) 0x7fcd400cdf80 con 0x7fcd400d12f0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3d600700 1 -- 172.21.15.99:0/2898169307 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd240035b0 con 0x7fcd400d12f0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3f400700 1 -- 172.21.15.99:0/2898169307 >> v2:172.21.15.133:3300/0 conn(0x7fcd4015e7a0 msgr2=0x7fcd400d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3f400700 1 --2- 172.21.15.99:0/2898169307 >> v2:172.21.15.133:3300/0 conn(0x7fcd4015e7a0 0x7fcd400d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3f400700 1 -- 172.21.15.99:0/2898169307 >> v1:172.21.15.99:6789/0 conn(0x7fcd400d12f0 legacy=0x7fcd400cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3f400700 1 -- 172.21.15.99:0/2898169307 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400d2140 con 0x7fcd400cfad0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3f400700 1 --2- 172.21.15.99:0/2898169307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400cfad0 0x7fcd400cfee0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcd400d30b0 tx=0x7fcd3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c961bd96ba48300f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3d600700 1 -- 172.21.15.99:0/2898169307 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3800e070 con 0x7fcd400cfad0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3d600700 1 -- 172.21.15.99:0/2898169307 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3800bb20 con 0x7fcd400cfad0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.723+0000 7fcd3d600700 1 -- 172.21.15.99:0/2898169307 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38012430 con 0x7fcd400cfad0 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 -- 172.21.15.99:0/2898169307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400cfad0 msgr2=0x7fcd400cfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 --2- 172.21.15.99:0/2898169307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400cfad0 0x7fcd400cfee0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcd400d30b0 tx=0x7fcd3800b330 comp rx=0 tx=0).stop 2024-03-29T09:11:49.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 -- 172.21.15.99:0/2898169307 shutdown_connections 2024-03-29T09:11:49.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 --2- 172.21.15.99:0/2898169307 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd400cfad0 0x7fcd400cfee0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 --2- 172.21.15.99:0/2898169307 >> v2:172.21.15.133:3300/0 conn(0x7fcd4015e7a0 0x7fcd400d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 -- 172.21.15.99:0/2898169307 >> 172.21.15.99:0/2898169307 conn(0x7fcd4009c480 msgr2=0x7fcd400d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:49.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 -- 172.21.15.99:0/2898169307 shutdown_connections 2024-03-29T09:11:49.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.724+0000 7fcd45e00700 1 -- 172.21.15.99:0/2898169307 wait complete. 2024-03-29T09:11:49.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.726+0000 7fcd45e00700 1 Processor -- start 2024-03-29T09:11:49.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.726+0000 7fcd45e00700 1 -- start start 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd45e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd400cfad0 0x7fcd4014d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd45e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd400d12f0 0x7fcd400b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd45e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd4015e7a0 0x7fcd400b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd45e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400b2b80 con 0x7fcd4015e7a0 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd45e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4014cec0 con 0x7fcd400cfad0 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd45e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd401f70e0 con 0x7fcd400d12f0 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd4015e7a0 0x7fcd400b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd4015e7a0 0x7fcd400b8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57678/0 (socket says 172.21.15.99:57678) 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 -- 172.21.15.99:0/1984763027 learned_addr learned my addr 172.21.15.99:0/1984763027 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3ea00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd400cfad0 0x7fcd4014d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3e000700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd400d12f0 0x7fcd400b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 -- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd400d12f0 msgr2=0x7fcd400b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd400d12f0 0x7fcd400b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 -- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd400cfad0 msgr2=0x7fcd4014d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd400cfad0 0x7fcd4014d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3f400700 1 -- 172.21.15.99:0/1984763027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd38009a70 con 0x7fcd4015e7a0 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.727+0000 7fcd3e000700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd400d12f0 0x7fcd400b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.728+0000 7fcd3f400700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd4015e7a0 0x7fcd400b8410 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcd38014040 tx=0x7fcd38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.728+0000 7fcd37400700 1 -- 172.21.15.99:0/1984763027 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38012430 con 0x7fcd4015e7a0 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.728+0000 7fcd37400700 1 -- 172.21.15.99:0/1984763027 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd38020070 con 0x7fcd4015e7a0 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.728+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd400b2250 con 0x7fcd4015e7a0 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.728+0000 7fcd37400700 1 -- 172.21.15.99:0/1984763027 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38004280 con 0x7fcd4015e7a0 2024-03-29T09:11:49.729 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.728+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd400b2810 con 0x7fcd4015e7a0 2024-03-29T09:11:49.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.729+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd400b2d30 con 0x7fcd4015e7a0 2024-03-29T09:11:49.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.730+0000 7fcd37400700 1 -- 172.21.15.99:0/1984763027 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcd380043e0 con 0x7fcd4015e7a0 2024-03-29T09:11:49.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.731+0000 7fcd37400700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcd2006ac90 0x7fcd2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:49.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.731+0000 7fcd37400700 1 -- 172.21.15.99:0/1984763027 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd38022070 con 0x7fcd4015e7a0 2024-03-29T09:11:49.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.731+0000 7fcd3ea00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcd2006ac90 0x7fcd2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:49.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.732+0000 7fcd3ea00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcd2006ac90 0x7fcd2006d150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009ad0 tx=0x7fcd30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:49.734 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.736+0000 7fcd37400700 1 -- 172.21.15.99:0/1984763027 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcd400b2d30 con 0x7fcd4015e7a0 2024-03-29T09:11:49.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.937+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd400ce7b0 con 0x7fcd2006ac90 2024-03-29T09:11:49.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.946+0000 7fcd37400700 1 -- 172.21.15.99:0/1984763027 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289201 (secure 0 0 0) 0x7fcd30011750 con 0x7fcd2006ac90 2024-03-29T09:11:49.949 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcd2006ac90 msgr2=0x7fcd2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcd2006ac90 0x7fcd2006d150 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009ad0 tx=0x7fcd30008040 comp rx=0 tx=0).stop 2024-03-29T09:11:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd4015e7a0 msgr2=0x7fcd400b8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd4015e7a0 0x7fcd400b8410 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcd38014040 tx=0x7fcd38004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 shutdown_connections 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcd2006ac90 0x7fcd2006d150 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd4015e7a0 0x7fcd400b8410 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd400cfad0 0x7fcd4014d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.950+0000 7fcd45e00700 1 --2- 172.21.15.99:0/1984763027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd400d12f0 0x7fcd400b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.951+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 >> 172.21.15.99:0/1984763027 conn(0x7fcd4009c480 msgr2=0x7fcd400cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.951+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 shutdown_connections 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:49.951+0000 7fcd45e00700 1 -- 172.21.15.99:0/1984763027 wait complete. 2024-03-29T09:11:49.952 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:11:49.967 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-03-29T09:11:48.486748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222552,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752168,"statfs":{"total":773094113280,"available":772866220032,"internally_reserved":0,"allocated":6529024,"data_stored":3536444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000831"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996579+0000","last_change":"2024-03-29T09:11:43.690372+0000","last_active":"2024-03-29T09:11:44.996579+0000","last_peered":"2024-03-29T09:11:44.996579+0000","last_clean":"2024-03-29T09:11:44.996579+0000","last_became_active":"2024-03-29T09:11:43.690046+0000","last_became_peered":"2024-03-29T09:11:43.690046+0000","last_unstale":"2024-03-29T09:11:44.996579+0000","last_undegraded":"2024-03-29T09:11:44.996579+0000","last_fullsized":"2024-03-29T09:11:44.996579+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:23.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667247+0000","last_change":"2024-03-29T09:11:43.681209+0000","last_active":"2024-03-29T09:11:44.667247+0000","last_peered":"2024-03-29T09:11:44.667247+0000","last_clean":"2024-03-29T09:11:44.667247+0000","last_became_active":"2024-03-29T09:11:43.680965+0000","last_became_peered":"2024-03-29T09:11:43.680965+0000","last_unstale":"2024-03-29T09:11:44.667247+0000","last_undegraded":"2024-03-29T09:11:44.667247+0000","last_fullsized":"2024-03-29T09:11:44.667247+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:32.530881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497341+0000","last_change":"2024-03-29T09:11:43.672222+0000","last_active":"2024-03-29T09:11:45.497341+0000","last_peered":"2024-03-29T09:11:45.497341+0000","last_clean":"2024-03-29T09:11:45.497341+0000","last_became_active":"2024-03-29T09:11:43.671980+0000","last_became_peered":"2024-03-29T09:11:43.671980+0000","last_unstale":"2024-03-29T09:11:45.497341+0000","last_undegraded":"2024-03-29T09:11:45.497341+0000","last_fullsized":"2024-03-29T09:11:45.497341+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:26:11.492718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497818+0000","last_change":"2024-03-29T09:11:43.689409+0000","last_active":"2024-03-29T09:11:45.497818+0000","last_peered":"2024-03-29T09:11:45.497818+0000","last_clean":"2024-03-29T09:11:45.497818+0000","last_became_active":"2024-03-29T09:11:43.689142+0000","last_became_peered":"2024-03-29T09:11:43.689142+0000","last_unstale":"2024-03-29T09:11:45.497818+0000","last_undegraded":"2024-03-29T09:11:45.497818+0000","last_fullsized":"2024-03-29T09:11:45.497818+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:45:02.972724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996181+0000","last_change":"2024-03-29T09:11:43.685296+0000","last_active":"2024-03-29T09:11:44.996181+0000","last_peered":"2024-03-29T09:11:44.996181+0000","last_clean":"2024-03-29T09:11:44.996181+0000","last_became_active":"2024-03-29T09:11:43.682430+0000","last_became_peered":"2024-03-29T09:11:43.682430+0000","last_unstale":"2024-03-29T09:11:44.996181+0000","last_undegraded":"2024-03-29T09:11:44.996181+0000","last_fullsized":"2024-03-29T09:11:44.996181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996027+0000","last_change":"2024-03-29T09:11:43.691095+0000","last_active":"2024-03-29T09:11:44.996027+0000","last_peered":"2024-03-29T09:11:44.996027+0000","last_clean":"2024-03-29T09:11:44.996027+0000","last_became_active":"2024-03-29T09:11:43.690828+0000","last_became_peered":"2024-03-29T09:11:43.690828+0000","last_unstale":"2024-03-29T09:11:44.996027+0000","last_undegraded":"2024-03-29T09:11:44.996027+0000","last_fullsized":"2024-03-29T09:11:44.996027+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:34:41.067618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497922+0000","last_change":"2024-03-29T09:11:43.685480+0000","last_active":"2024-03-29T09:11:45.497922+0000","last_peered":"2024-03-29T09:11:45.497922+0000","last_clean":"2024-03-29T09:11:45.497922+0000","last_became_active":"2024-03-29T09:11:43.683700+0000","last_became_peered":"2024-03-29T09:11:43.683700+0000","last_unstale":"2024-03-29T09:11:45.497922+0000","last_undegraded":"2024-03-29T09:11:45.497922+0000","last_fullsized":"2024-03-29T09:11:45.497922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:00:03.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496748+0000","last_change":"2024-03-29T09:11:43.677985+0000","last_active":"2024-03-29T09:11:45.496748+0000","last_peered":"2024-03-29T09:11:45.496748+0000","last_clean":"2024-03-29T09:11:45.496748+0000","last_became_active":"2024-03-29T09:11:43.677658+0000","last_became_peered":"2024-03-29T09:11:43.677658+0000","last_unstale":"2024-03-29T09:11:45.496748+0000","last_undegraded":"2024-03-29T09:11:45.496748+0000","last_fullsized":"2024-03-29T09:11:45.496748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:04.674681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497195+0000","last_change":"2024-03-29T09:11:43.681141+0000","last_active":"2024-03-29T09:11:45.497195+0000","last_peered":"2024-03-29T09:11:45.497195+0000","last_clean":"2024-03-29T09:11:45.497195+0000","last_became_active":"2024-03-29T09:11:43.680651+0000","last_became_peered":"2024-03-29T09:11:43.680651+0000","last_unstale":"2024-03-29T09:11:45.497195+0000","last_undegraded":"2024-03-29T09:11:45.497195+0000","last_fullsized":"2024-03-29T09:11:45.497195+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:20.720413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996423+0000","last_change":"2024-03-29T09:11:43.689627+0000","last_active":"2024-03-29T09:11:44.996423+0000","last_peered":"2024-03-29T09:11:44.996423+0000","last_clean":"2024-03-29T09:11:44.996423+0000","last_became_active":"2024-03-29T09:11:43.689352+0000","last_became_peered":"2024-03-29T09:11:43.689352+0000","last_unstale":"2024-03-29T09:11:44.996423+0000","last_undegraded":"2024-03-29T09:11:44.996423+0000","last_fullsized":"2024-03-29T09:11:44.996423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:04.674681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499008+0000","last_change":"2024-03-29T09:11:43.684198+0000","last_active":"2024-03-29T09:11:45.499008+0000","last_peered":"2024-03-29T09:11:45.499008+0000","last_clean":"2024-03-29T09:11:45.499008+0000","last_became_active":"2024-03-29T09:11:43.683881+0000","last_became_peered":"2024-03-29T09:11:43.683881+0000","last_unstale":"2024-03-29T09:11:45.499008+0000","last_undegraded":"2024-03-29T09:11:45.499008+0000","last_fullsized":"2024-03-29T09:11:45.499008+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:17:07.814736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497080+0000","last_change":"2024-03-29T09:11:43.680518+0000","last_active":"2024-03-29T09:11:45.497080+0000","last_peered":"2024-03-29T09:11:45.497080+0000","last_clean":"2024-03-29T09:11:45.497080+0000","last_became_active":"2024-03-29T09:11:43.680221+0000","last_became_peered":"2024-03-29T09:11:43.680221+0000","last_unstale":"2024-03-29T09:11:45.497080+0000","last_undegraded":"2024-03-29T09:11:45.497080+0000","last_fullsized":"2024-03-29T09:11:45.497080+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:04:14.657461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667609+0000","last_change":"2024-03-29T09:11:43.688051+0000","last_active":"2024-03-29T09:11:44.667609+0000","last_peered":"2024-03-29T09:11:44.667609+0000","last_clean":"2024-03-29T09:11:44.667609+0000","last_became_active":"2024-03-29T09:11:43.687664+0000","last_became_peered":"2024-03-29T09:11:43.687664+0000","last_unstale":"2024-03-29T09:11:44.667609+0000","last_undegraded":"2024-03-29T09:11:44.667609+0000","last_fullsized":"2024-03-29T09:11:44.667609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:37.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497047+0000","last_change":"2024-03-29T09:11:43.665919+0000","last_active":"2024-03-29T09:11:45.497047+0000","last_peered":"2024-03-29T09:11:45.497047+0000","last_clean":"2024-03-29T09:11:45.497047+0000","last_became_active":"2024-03-29T09:11:43.665662+0000","last_became_peered":"2024-03-29T09:11:43.665662+0000","last_unstale":"2024-03-29T09:11:45.497047+0000","last_undegraded":"2024-03-29T09:11:45.497047+0000","last_fullsized":"2024-03-29T09:11:45.497047+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:25.840707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667467+0000","last_change":"2024-03-29T09:11:43.683560+0000","last_active":"2024-03-29T09:11:44.667467+0000","last_peered":"2024-03-29T09:11:44.667467+0000","last_clean":"2024-03-29T09:11:44.667467+0000","last_became_active":"2024-03-29T09:11:43.683227+0000","last_became_peered":"2024-03-29T09:11:43.683227+0000","last_unstale":"2024-03-29T09:11:44.667467+0000","last_undegraded":"2024-03-29T09:11:44.667467+0000","last_fullsized":"2024-03-29T09:11:44.667467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:34.499347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996022+0000","last_change":"2024-03-29T09:11:43.680836+0000","last_active":"2024-03-29T09:11:44.996022+0000","last_peered":"2024-03-29T09:11:44.996022+0000","last_clean":"2024-03-29T09:11:44.996022+0000","last_became_active":"2024-03-29T09:11:43.680568+0000","last_became_peered":"2024-03-29T09:11:43.680568+0000","last_unstale":"2024-03-29T09:11:44.996022+0000","last_undegraded":"2024-03-29T09:11:44.996022+0000","last_fullsized":"2024-03-29T09:11:44.996022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:03.217442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996654+0000","last_change":"2024-03-29T09:11:43.679345+0000","last_active":"2024-03-29T09:11:44.996654+0000","last_peered":"2024-03-29T09:11:44.996654+0000","last_clean":"2024-03-29T09:11:44.996654+0000","last_became_active":"2024-03-29T09:11:43.678923+0000","last_became_peered":"2024-03-29T09:11:43.678923+0000","last_unstale":"2024-03-29T09:11:44.996654+0000","last_undegraded":"2024-03-29T09:11:44.996654+0000","last_fullsized":"2024-03-29T09:11:44.996654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497557+0000","last_change":"2024-03-29T09:11:43.666174+0000","last_active":"2024-03-29T09:11:45.497557+0000","last_peered":"2024-03-29T09:11:45.497557+0000","last_clean":"2024-03-29T09:11:45.497557+0000","last_became_active":"2024-03-29T09:11:43.665473+0000","last_became_peered":"2024-03-29T09:11:43.665473+0000","last_unstale":"2024-03-29T09:11:45.497557+0000","last_undegraded":"2024-03-29T09:11:45.497557+0000","last_fullsized":"2024-03-29T09:11:45.497557+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667309+0000","last_change":"2024-03-29T09:11:43.671528+0000","last_active":"2024-03-29T09:11:44.667309+0000","last_peered":"2024-03-29T09:11:44.667309+0000","last_clean":"2024-03-29T09:11:44.667309+0000","last_became_active":"2024-03-29T09:11:43.670774+0000","last_became_peered":"2024-03-29T09:11:43.670774+0000","last_unstale":"2024-03-29T09:11:44.667309+0000","last_undegraded":"2024-03-29T09:11:44.667309+0000","last_fullsized":"2024-03-29T09:11:44.667309+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:20.720413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498963+0000","last_change":"2024-03-29T09:11:43.686864+0000","last_active":"2024-03-29T09:11:45.498963+0000","last_peered":"2024-03-29T09:11:45.498963+0000","last_clean":"2024-03-29T09:11:45.498963+0000","last_became_active":"2024-03-29T09:11:43.686418+0000","last_became_peered":"2024-03-29T09:11:43.686418+0000","last_unstale":"2024-03-29T09:11:45.498963+0000","last_undegraded":"2024-03-29T09:11:45.498963+0000","last_fullsized":"2024-03-29T09:11:45.498963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:17.190299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667610+0000","last_change":"2024-03-29T09:11:43.677257+0000","last_active":"2024-03-29T09:11:44.667610+0000","last_peered":"2024-03-29T09:11:44.667610+0000","last_clean":"2024-03-29T09:11:44.667610+0000","last_became_active":"2024-03-29T09:11:43.677003+0000","last_became_peered":"2024-03-29T09:11:43.677003+0000","last_unstale":"2024-03-29T09:11:44.667610+0000","last_undegraded":"2024-03-29T09:11:44.667610+0000","last_fullsized":"2024-03-29T09:11:44.667610+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:02.886449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498962+0000","last_change":"2024-03-29T09:11:43.688297+0000","last_active":"2024-03-29T09:11:45.498962+0000","last_peered":"2024-03-29T09:11:45.498962+0000","last_clean":"2024-03-29T09:11:45.498962+0000","last_became_active":"2024-03-29T09:11:43.688005+0000","last_became_peered":"2024-03-29T09:11:43.688005+0000","last_unstale":"2024-03-29T09:11:45.498962+0000","last_undegraded":"2024-03-29T09:11:45.498962+0000","last_fullsized":"2024-03-29T09:11:45.498962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:20.115771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498025+0000","last_change":"2024-03-29T09:11:43.678188+0000","last_active":"2024-03-29T09:11:45.498025+0000","last_peered":"2024-03-29T09:11:45.498025+0000","last_clean":"2024-03-29T09:11:45.498025+0000","last_became_active":"2024-03-29T09:11:43.677934+0000","last_became_peered":"2024-03-29T09:11:43.677934+0000","last_unstale":"2024-03-29T09:11:45.498025+0000","last_undegraded":"2024-03-29T09:11:45.498025+0000","last_fullsized":"2024-03-29T09:11:45.498025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:37:26.330731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496826+0000","last_change":"2024-03-29T09:11:43.675646+0000","last_active":"2024-03-29T09:11:45.496826+0000","last_peered":"2024-03-29T09:11:45.496826+0000","last_clean":"2024-03-29T09:11:45.496826+0000","last_became_active":"2024-03-29T09:11:43.675298+0000","last_became_peered":"2024-03-29T09:11:43.675298+0000","last_unstale":"2024-03-29T09:11:45.496826+0000","last_undegraded":"2024-03-29T09:11:45.496826+0000","last_fullsized":"2024-03-29T09:11:45.496826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:05.122450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667348+0000","last_change":"2024-03-29T09:11:43.680611+0000","last_active":"2024-03-29T09:11:44.667348+0000","last_peered":"2024-03-29T09:11:44.667348+0000","last_clean":"2024-03-29T09:11:44.667348+0000","last_became_active":"2024-03-29T09:11:43.680345+0000","last_became_peered":"2024-03-29T09:11:43.680345+0000","last_unstale":"2024-03-29T09:11:44.667348+0000","last_undegraded":"2024-03-29T09:11:44.667348+0000","last_fullsized":"2024-03-29T09:11:44.667348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:37.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667802+0000","last_change":"2024-03-29T09:11:43.682019+0000","last_active":"2024-03-29T09:11:44.667802+0000","last_peered":"2024-03-29T09:11:44.667802+0000","last_clean":"2024-03-29T09:11:44.667802+0000","last_became_active":"2024-03-29T09:11:43.681763+0000","last_became_peered":"2024-03-29T09:11:43.681763+0000","last_unstale":"2024-03-29T09:11:44.667802+0000","last_undegraded":"2024-03-29T09:11:44.667802+0000","last_fullsized":"2024-03-29T09:11:44.667802+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:48.040844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667365+0000","last_change":"2024-03-29T09:11:43.671717+0000","last_active":"2024-03-29T09:11:44.667365+0000","last_peered":"2024-03-29T09:11:44.667365+0000","last_clean":"2024-03-29T09:11:44.667365+0000","last_became_active":"2024-03-29T09:11:43.671231+0000","last_became_peered":"2024-03-29T09:11:43.671231+0000","last_unstale":"2024-03-29T09:11:44.667365+0000","last_undegraded":"2024-03-29T09:11:44.667365+0000","last_fullsized":"2024-03-29T09:11:44.667365+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:05.122450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496922+0000","last_change":"2024-03-29T09:11:43.680460+0000","last_active":"2024-03-29T09:11:45.496922+0000","last_peered":"2024-03-29T09:11:45.496922+0000","last_clean":"2024-03-29T09:11:45.496922+0000","last_became_active":"2024-03-29T09:11:43.680176+0000","last_became_peered":"2024-03-29T09:11:43.680176+0000","last_unstale":"2024-03-29T09:11:45.496922+0000","last_undegraded":"2024-03-29T09:11:45.496922+0000","last_fullsized":"2024-03-29T09:11:45.496922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:26.946110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496915+0000","last_change":"2024-03-29T09:11:43.679680+0000","last_active":"2024-03-29T09:11:45.496915+0000","last_peered":"2024-03-29T09:11:45.496915+0000","last_clean":"2024-03-29T09:11:45.496915+0000","last_became_active":"2024-03-29T09:11:43.679397+0000","last_became_peered":"2024-03-29T09:11:43.679397+0000","last_unstale":"2024-03-29T09:11:45.496915+0000","last_undegraded":"2024-03-29T09:11:45.496915+0000","last_fullsized":"2024-03-29T09:11:45.496915+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:16.721779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497963+0000","last_change":"2024-03-29T09:11:43.679106+0000","last_active":"2024-03-29T09:11:45.497963+0000","last_peered":"2024-03-29T09:11:45.497963+0000","last_clean":"2024-03-29T09:11:45.497963+0000","last_became_active":"2024-03-29T09:11:43.678804+0000","last_became_peered":"2024-03-29T09:11:43.678804+0000","last_unstale":"2024-03-29T09:11:45.497963+0000","last_undegraded":"2024-03-29T09:11:45.497963+0000","last_fullsized":"2024-03-29T09:11:45.497963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:48:32.315631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.666942+0000","last_change":"2024-03-29T09:11:43.669556+0000","last_active":"2024-03-29T09:11:44.666942+0000","last_peered":"2024-03-29T09:11:44.666942+0000","last_clean":"2024-03-29T09:11:44.666942+0000","last_became_active":"2024-03-29T09:11:43.669284+0000","last_became_peered":"2024-03-29T09:11:43.669284+0000","last_unstale":"2024-03-29T09:11:44.666942+0000","last_undegraded":"2024-03-29T09:11:44.666942+0000","last_fullsized":"2024-03-29T09:11:44.666942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:37.448155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498776+0000","last_change":"2024-03-29T09:11:43.688520+0000","last_active":"2024-03-29T09:11:45.498776+0000","last_peered":"2024-03-29T09:11:45.498776+0000","last_clean":"2024-03-29T09:11:45.498776+0000","last_became_active":"2024-03-29T09:11:43.688113+0000","last_became_peered":"2024-03-29T09:11:43.688113+0000","last_unstale":"2024-03-29T09:11:45.498776+0000","last_undegraded":"2024-03-29T09:11:45.498776+0000","last_fullsized":"2024-03-29T09:11:45.498776+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:25.840707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996437+0000","last_change":"2024-03-29T09:11:43.690479+0000","last_active":"2024-03-29T09:11:44.996437+0000","last_peered":"2024-03-29T09:11:44.996437+0000","last_clean":"2024-03-29T09:11:44.996437+0000","last_became_active":"2024-03-29T09:11:43.689919+0000","last_became_peered":"2024-03-29T09:11:43.689919+0000","last_unstale":"2024-03-29T09:11:44.996437+0000","last_undegraded":"2024-03-29T09:11:44.996437+0000","last_fullsized":"2024-03-29T09:11:44.996437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:54.749065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499230+0000","last_change":"2024-03-29T09:11:43.168276+0000","last_active":"2024-03-29T09:11:45.499230+0000","last_peered":"2024-03-29T09:11:45.499230+0000","last_clean":"2024-03-29T09:11:45.499230+0000","last_became_active":"2024-03-29T09:11:43.168028+0000","last_became_peered":"2024-03-29T09:11:43.168028+0000","last_unstale":"2024-03-29T09:11:45.499230+0000","last_undegraded":"2024-03-29T09:11:45.499230+0000","last_fullsized":"2024-03-29T09:11:45.499230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:48.069355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996656+0000","last_change":"2024-03-29T09:11:43.691607+0000","last_active":"2024-03-29T09:11:44.996656+0000","last_peered":"2024-03-29T09:11:44.996656+0000","last_clean":"2024-03-29T09:11:44.996656+0000","last_became_active":"2024-03-29T09:11:43.691054+0000","last_became_peered":"2024-03-29T09:11:43.691054+0000","last_unstale":"2024-03-29T09:11:44.996656+0000","last_undegraded":"2024-03-29T09:11:44.996656+0000","last_fullsized":"2024-03-29T09:11:44.996656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:37.929321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667464+0000","last_change":"2024-03-29T09:11:43.172794+0000","last_active":"2024-03-29T09:11:44.667464+0000","last_peered":"2024-03-29T09:11:44.667464+0000","last_clean":"2024-03-29T09:11:44.667464+0000","last_became_active":"2024-03-29T09:11:43.172062+0000","last_became_peered":"2024-03-29T09:11:43.172062+0000","last_unstale":"2024-03-29T09:11:44.667464+0000","last_undegraded":"2024-03-29T09:11:44.667464+0000","last_fullsized":"2024-03-29T09:11:44.667464+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:40:54.283138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496997+0000","last_change":"2024-03-29T09:11:43.685364+0000","last_active":"2024-03-29T09:11:45.496997+0000","last_peered":"2024-03-29T09:11:45.496997+0000","last_clean":"2024-03-29T09:11:45.496997+0000","last_became_active":"2024-03-29T09:11:43.685091+0000","last_became_peered":"2024-03-29T09:11:43.685091+0000","last_unstale":"2024-03-29T09:11:45.496997+0000","last_undegraded":"2024-03-29T09:11:45.496997+0000","last_fullsized":"2024-03-29T09:11:45.496997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:54.749065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996552+0000","last_change":"2024-03-29T09:11:43.173098+0000","last_active":"2024-03-29T09:11:44.996552+0000","last_peered":"2024-03-29T09:11:44.996552+0000","last_clean":"2024-03-29T09:11:44.996552+0000","last_became_active":"2024-03-29T09:11:43.172846+0000","last_became_peered":"2024-03-29T09:11:43.172846+0000","last_unstale":"2024-03-29T09:11:44.996552+0000","last_undegraded":"2024-03-29T09:11:44.996552+0000","last_fullsized":"2024-03-29T09:11:44.996552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:47.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996414+0000","last_change":"2024-03-29T09:11:43.692247+0000","last_active":"2024-03-29T09:11:44.996414+0000","last_peered":"2024-03-29T09:11:44.996414+0000","last_clean":"2024-03-29T09:11:44.996414+0000","last_became_active":"2024-03-29T09:11:43.691491+0000","last_became_peered":"2024-03-29T09:11:43.691491+0000","last_unstale":"2024-03-29T09:11:44.996414+0000","last_undegraded":"2024-03-29T09:11:44.996414+0000","last_fullsized":"2024-03-29T09:11:44.996414+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.953214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497899+0000","last_change":"2024-03-29T09:11:43.177133+0000","last_active":"2024-03-29T09:11:45.497899+0000","last_peered":"2024-03-29T09:11:45.497899+0000","last_clean":"2024-03-29T09:11:45.497899+0000","last_became_active":"2024-03-29T09:11:43.176799+0000","last_became_peered":"2024-03-29T09:11:43.176799+0000","last_unstale":"2024-03-29T09:11:45.497899+0000","last_undegraded":"2024-03-29T09:11:45.497899+0000","last_fullsized":"2024-03-29T09:11:45.497899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:45:07.789102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497028+0000","last_change":"2024-03-29T09:11:43.683820+0000","last_active":"2024-03-29T09:11:45.497028+0000","last_peered":"2024-03-29T09:11:45.497028+0000","last_clean":"2024-03-29T09:11:45.497028+0000","last_became_active":"2024-03-29T09:11:43.683490+0000","last_became_peered":"2024-03-29T09:11:43.683490+0000","last_unstale":"2024-03-29T09:11:45.497028+0000","last_undegraded":"2024-03-29T09:11:45.497028+0000","last_fullsized":"2024-03-29T09:11:45.497028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:03.217442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497615+0000","last_change":"2024-03-29T09:11:43.180832+0000","last_active":"2024-03-29T09:11:45.497615+0000","last_peered":"2024-03-29T09:11:45.497615+0000","last_clean":"2024-03-29T09:11:45.497615+0000","last_became_active":"2024-03-29T09:11:43.180518+0000","last_became_peered":"2024-03-29T09:11:43.180518+0000","last_unstale":"2024-03-29T09:11:45.497615+0000","last_undegraded":"2024-03-29T09:11:45.497615+0000","last_fullsized":"2024-03-29T09:11:45.497615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:24:19.226534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498945+0000","last_change":"2024-03-29T09:11:43.688382+0000","last_active":"2024-03-29T09:11:45.498945+0000","last_peered":"2024-03-29T09:11:45.498945+0000","last_clean":"2024-03-29T09:11:45.498945+0000","last_became_active":"2024-03-29T09:11:43.688094+0000","last_became_peered":"2024-03-29T09:11:43.688094+0000","last_unstale":"2024-03-29T09:11:45.498945+0000","last_undegraded":"2024-03-29T09:11:45.498945+0000","last_fullsized":"2024-03-29T09:11:45.498945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:21.218363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996277+0000","last_change":"2024-03-29T09:11:43.183057+0000","last_active":"2024-03-29T09:11:44.996277+0000","last_peered":"2024-03-29T09:11:44.996277+0000","last_clean":"2024-03-29T09:11:44.996277+0000","last_became_active":"2024-03-29T09:11:43.182589+0000","last_became_peered":"2024-03-29T09:11:43.182589+0000","last_unstale":"2024-03-29T09:11:44.996277+0000","last_undegraded":"2024-03-29T09:11:44.996277+0000","last_fullsized":"2024-03-29T09:11:44.996277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:01.878804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496947+0000","last_change":"2024-03-29T09:11:43.676496+0000","last_active":"2024-03-29T09:11:45.496947+0000","last_peered":"2024-03-29T09:11:45.496947+0000","last_clean":"2024-03-29T09:11:45.496947+0000","last_became_active":"2024-03-29T09:11:43.676208+0000","last_became_peered":"2024-03-29T09:11:43.676208+0000","last_unstale":"2024-03-29T09:11:45.496947+0000","last_undegraded":"2024-03-29T09:11:45.496947+0000","last_fullsized":"2024-03-29T09:11:45.496947+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:16.327235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497050+0000","last_change":"2024-03-29T09:11:43.173125+0000","last_active":"2024-03-29T09:11:45.497050+0000","last_peered":"2024-03-29T09:11:45.497050+0000","last_clean":"2024-03-29T09:11:45.497050+0000","last_became_active":"2024-03-29T09:11:43.172897+0000","last_became_peered":"2024-03-29T09:11:43.172897+0000","last_unstale":"2024-03-29T09:11:45.497050+0000","last_undegraded":"2024-03-29T09:11:45.497050+0000","last_fullsized":"2024-03-29T09:11:45.497050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:51.992855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996071+0000","last_change":"2024-03-29T09:11:43.681720+0000","last_active":"2024-03-29T09:11:44.996071+0000","last_peered":"2024-03-29T09:11:44.996071+0000","last_clean":"2024-03-29T09:11:44.996071+0000","last_became_active":"2024-03-29T09:11:43.681366+0000","last_became_peered":"2024-03-29T09:11:43.681366+0000","last_unstale":"2024-03-29T09:11:44.996071+0000","last_undegraded":"2024-03-29T09:11:44.996071+0000","last_fullsized":"2024-03-29T09:11:44.996071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:34.234919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667404+0000","last_change":"2024-03-29T09:11:43.176057+0000","last_active":"2024-03-29T09:11:44.667404+0000","last_peered":"2024-03-29T09:11:44.667404+0000","last_clean":"2024-03-29T09:11:44.667404+0000","last_became_active":"2024-03-29T09:11:43.175783+0000","last_became_peered":"2024-03-29T09:11:43.175783+0000","last_unstale":"2024-03-29T09:11:44.667404+0000","last_undegraded":"2024-03-29T09:11:44.667404+0000","last_fullsized":"2024-03-29T09:11:44.667404+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:31.151934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667835+0000","last_change":"2024-03-29T09:11:43.681530+0000","last_active":"2024-03-29T09:11:44.667835+0000","last_peered":"2024-03-29T09:11:44.667835+0000","last_clean":"2024-03-29T09:11:44.667835+0000","last_became_active":"2024-03-29T09:11:43.679577+0000","last_became_peered":"2024-03-29T09:11:43.679577+0000","last_unstale":"2024-03-29T09:11:44.667835+0000","last_undegraded":"2024-03-29T09:11:44.667835+0000","last_fullsized":"2024-03-29T09:11:44.667835+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:32.530881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497211+0000","last_change":"2024-03-29T09:11:43.165270+0000","last_active":"2024-03-29T09:11:45.497211+0000","last_peered":"2024-03-29T09:11:45.497211+0000","last_clean":"2024-03-29T09:11:45.497211+0000","last_became_active":"2024-03-29T09:11:43.164467+0000","last_became_peered":"2024-03-29T09:11:43.164467+0000","last_unstale":"2024-03-29T09:11:45.497211+0000","last_undegraded":"2024-03-29T09:11:45.497211+0000","last_fullsized":"2024-03-29T09:11:45.497211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:29.110475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497883+0000","last_change":"2024-03-29T09:11:43.666694+0000","last_active":"2024-03-29T09:11:45.497883+0000","last_peered":"2024-03-29T09:11:45.497883+0000","last_clean":"2024-03-29T09:11:45.497883+0000","last_became_active":"2024-03-29T09:11:43.666396+0000","last_became_peered":"2024-03-29T09:11:43.666396+0000","last_unstale":"2024-03-29T09:11:45.497883+0000","last_undegraded":"2024-03-29T09:11:45.497883+0000","last_fullsized":"2024-03-29T09:11:45.497883+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:20.115771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497219+0000","last_change":"2024-03-29T09:11:43.173205+0000","last_active":"2024-03-29T09:11:45.497219+0000","last_peered":"2024-03-29T09:11:45.497219+0000","last_clean":"2024-03-29T09:11:45.497219+0000","last_became_active":"2024-03-29T09:11:43.172920+0000","last_became_peered":"2024-03-29T09:11:43.172920+0000","last_unstale":"2024-03-29T09:11:45.497219+0000","last_undegraded":"2024-03-29T09:11:45.497219+0000","last_fullsized":"2024-03-29T09:11:45.497219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:33.491702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497202+0000","last_change":"2024-03-29T09:11:43.685373+0000","last_active":"2024-03-29T09:11:45.497202+0000","last_peered":"2024-03-29T09:11:45.497202+0000","last_clean":"2024-03-29T09:11:45.497202+0000","last_became_active":"2024-03-29T09:11:43.685092+0000","last_became_peered":"2024-03-29T09:11:43.685092+0000","last_unstale":"2024-03-29T09:11:45.497202+0000","last_undegraded":"2024-03-29T09:11:45.497202+0000","last_fullsized":"2024-03-29T09:11:45.497202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498123+0000","last_change":"2024-03-29T09:11:43.181613+0000","last_active":"2024-03-29T09:11:45.498123+0000","last_peered":"2024-03-29T09:11:45.498123+0000","last_clean":"2024-03-29T09:11:45.498123+0000","last_became_active":"2024-03-29T09:11:43.181326+0000","last_became_peered":"2024-03-29T09:11:43.181326+0000","last_unstale":"2024-03-29T09:11:45.498123+0000","last_undegraded":"2024-03-29T09:11:45.498123+0000","last_fullsized":"2024-03-29T09:11:45.498123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:12.031715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497714+0000","last_change":"2024-03-29T09:11:43.675831+0000","last_active":"2024-03-29T09:11:45.497714+0000","last_peered":"2024-03-29T09:11:45.497714+0000","last_clean":"2024-03-29T09:11:45.497714+0000","last_became_active":"2024-03-29T09:11:43.675418+0000","last_became_peered":"2024-03-29T09:11:43.675418+0000","last_unstale":"2024-03-29T09:11:45.497714+0000","last_undegraded":"2024-03-29T09:11:45.497714+0000","last_fullsized":"2024-03-29T09:11:45.497714+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:06:28.079801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667829+0000","last_change":"2024-03-29T09:11:43.181637+0000","last_active":"2024-03-29T09:11:44.667829+0000","last_peered":"2024-03-29T09:11:44.667829+0000","last_clean":"2024-03-29T09:11:44.667829+0000","last_became_active":"2024-03-29T09:11:43.181327+0000","last_became_peered":"2024-03-29T09:11:43.181327+0000","last_unstale":"2024-03-29T09:11:44.667829+0000","last_undegraded":"2024-03-29T09:11:44.667829+0000","last_fullsized":"2024-03-29T09:11:44.667829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:34.000096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497070+0000","last_change":"2024-03-29T09:11:43.677020+0000","last_active":"2024-03-29T09:11:45.497070+0000","last_peered":"2024-03-29T09:11:45.497070+0000","last_clean":"2024-03-29T09:11:45.497070+0000","last_became_active":"2024-03-29T09:11:43.676671+0000","last_became_peered":"2024-03-29T09:11:43.676671+0000","last_unstale":"2024-03-29T09:11:45.497070+0000","last_undegraded":"2024-03-29T09:11:45.497070+0000","last_fullsized":"2024-03-29T09:11:45.497070+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:00:54.297415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497048+0000","last_change":"2024-03-29T09:11:43.176006+0000","last_active":"2024-03-29T09:11:45.497048+0000","last_peered":"2024-03-29T09:11:45.497048+0000","last_clean":"2024-03-29T09:11:45.497048+0000","last_became_active":"2024-03-29T09:11:43.175716+0000","last_became_peered":"2024-03-29T09:11:43.175716+0000","last_unstale":"2024-03-29T09:11:45.497048+0000","last_undegraded":"2024-03-29T09:11:45.497048+0000","last_fullsized":"2024-03-29T09:11:45.497048+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:47.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497524+0000","last_change":"2024-03-29T09:11:43.673170+0000","last_active":"2024-03-29T09:11:45.497524+0000","last_peered":"2024-03-29T09:11:45.497524+0000","last_clean":"2024-03-29T09:11:45.497524+0000","last_became_active":"2024-03-29T09:11:43.672886+0000","last_became_peered":"2024-03-29T09:11:43.672886+0000","last_unstale":"2024-03-29T09:11:45.497524+0000","last_undegraded":"2024-03-29T09:11:45.497524+0000","last_fullsized":"2024-03-29T09:11:45.497524+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:16.721779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498004+0000","last_change":"2024-03-29T09:11:43.177733+0000","last_active":"2024-03-29T09:11:45.498004+0000","last_peered":"2024-03-29T09:11:45.498004+0000","last_clean":"2024-03-29T09:11:45.498004+0000","last_became_active":"2024-03-29T09:11:43.177447+0000","last_became_peered":"2024-03-29T09:11:43.177447+0000","last_unstale":"2024-03-29T09:11:45.498004+0000","last_undegraded":"2024-03-29T09:11:45.498004+0000","last_fullsized":"2024-03-29T09:11:45.498004+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:31:51.585476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496889+0000","last_change":"2024-03-29T09:11:43.681087+0000","last_active":"2024-03-29T09:11:45.496889+0000","last_peered":"2024-03-29T09:11:45.496889+0000","last_clean":"2024-03-29T09:11:45.496889+0000","last_became_active":"2024-03-29T09:11:43.680814+0000","last_became_peered":"2024-03-29T09:11:43.680814+0000","last_unstale":"2024-03-29T09:11:45.496889+0000","last_undegraded":"2024-03-29T09:11:45.496889+0000","last_fullsized":"2024-03-29T09:11:45.496889+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:37.448155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498207+0000","last_change":"2024-03-29T09:11:43.170448+0000","last_active":"2024-03-29T09:11:45.498207+0000","last_peered":"2024-03-29T09:11:45.498207+0000","last_clean":"2024-03-29T09:11:45.498207+0000","last_became_active":"2024-03-29T09:11:43.170170+0000","last_became_peered":"2024-03-29T09:11:43.170170+0000","last_unstale":"2024-03-29T09:11:45.498207+0000","last_undegraded":"2024-03-29T09:11:45.498207+0000","last_fullsized":"2024-03-29T09:11:45.498207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:21.777400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498121+0000","last_change":"2024-03-29T09:11:43.679100+0000","last_active":"2024-03-29T09:11:45.498121+0000","last_peered":"2024-03-29T09:11:45.498121+0000","last_clean":"2024-03-29T09:11:45.498121+0000","last_became_active":"2024-03-29T09:11:43.678319+0000","last_became_peered":"2024-03-29T09:11:43.678319+0000","last_unstale":"2024-03-29T09:11:45.498121+0000","last_undegraded":"2024-03-29T09:11:45.498121+0000","last_fullsized":"2024-03-29T09:11:45.498121+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:58:28.504247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":19,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497725+0000","last_change":"2024-03-29T09:11:43.173897+0000","last_active":"2024-03-29T09:11:45.497725+0000","last_peered":"2024-03-29T09:11:45.497725+0000","last_clean":"2024-03-29T09:11:45.497725+0000","last_became_active":"2024-03-29T09:11:43.172759+0000","last_became_peered":"2024-03-29T09:11:43.172759+0000","last_unstale":"2024-03-29T09:11:45.497725+0000","last_undegraded":"2024-03-29T09:11:45.497725+0000","last_fullsized":"2024-03-29T09:11:45.497725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.415296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497652+0000","last_change":"2024-03-29T09:11:43.681216+0000","last_active":"2024-03-29T09:11:45.497652+0000","last_peered":"2024-03-29T09:11:45.497652+0000","last_clean":"2024-03-29T09:11:45.497652+0000","last_became_active":"2024-03-29T09:11:43.680966+0000","last_became_peered":"2024-03-29T09:11:43.680966+0000","last_unstale":"2024-03-29T09:11:45.497652+0000","last_undegraded":"2024-03-29T09:11:45.497652+0000","last_fullsized":"2024-03-29T09:11:45.497652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:06.103995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667467+0000","last_change":"2024-03-29T09:11:43.183881+0000","last_active":"2024-03-29T09:11:44.667467+0000","last_peered":"2024-03-29T09:11:44.667467+0000","last_clean":"2024-03-29T09:11:44.667467+0000","last_became_active":"2024-03-29T09:11:43.183567+0000","last_became_peered":"2024-03-29T09:11:43.183567+0000","last_unstale":"2024-03-29T09:11:44.667467+0000","last_undegraded":"2024-03-29T09:11:44.667467+0000","last_fullsized":"2024-03-29T09:11:44.667467+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:44.603438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498039+0000","last_change":"2024-03-29T09:11:43.665968+0000","last_active":"2024-03-29T09:11:45.498039+0000","last_peered":"2024-03-29T09:11:45.498039+0000","last_clean":"2024-03-29T09:11:45.498039+0000","last_became_active":"2024-03-29T09:11:43.665714+0000","last_became_peered":"2024-03-29T09:11:43.665714+0000","last_unstale":"2024-03-29T09:11:45.498039+0000","last_undegraded":"2024-03-29T09:11:45.498039+0000","last_fullsized":"2024-03-29T09:11:45.498039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:21.218363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497372+0000","last_change":"2024-03-29T09:11:43.173582+0000","last_active":"2024-03-29T09:11:45.497372+0000","last_peered":"2024-03-29T09:11:45.497372+0000","last_clean":"2024-03-29T09:11:45.497372+0000","last_became_active":"2024-03-29T09:11:43.173300+0000","last_became_peered":"2024-03-29T09:11:43.173300+0000","last_unstale":"2024-03-29T09:11:45.497372+0000","last_undegraded":"2024-03-29T09:11:45.497372+0000","last_fullsized":"2024-03-29T09:11:45.497372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:50.765164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.666945+0000","last_change":"2024-03-29T09:11:43.685214+0000","last_active":"2024-03-29T09:11:44.666945+0000","last_peered":"2024-03-29T09:11:44.666945+0000","last_clean":"2024-03-29T09:11:44.666945+0000","last_became_active":"2024-03-29T09:11:43.684951+0000","last_became_peered":"2024-03-29T09:11:43.684951+0000","last_unstale":"2024-03-29T09:11:44.666945+0000","last_undegraded":"2024-03-29T09:11:44.666945+0000","last_fullsized":"2024-03-29T09:11:44.666945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:21.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497581+0000","last_change":"2024-03-29T09:11:43.171045+0000","last_active":"2024-03-29T09:11:45.497581+0000","last_peered":"2024-03-29T09:11:45.497581+0000","last_clean":"2024-03-29T09:11:45.497581+0000","last_became_active":"2024-03-29T09:11:43.170833+0000","last_became_peered":"2024-03-29T09:11:43.170833+0000","last_unstale":"2024-03-29T09:11:45.497581+0000","last_undegraded":"2024-03-29T09:11:45.497581+0000","last_fullsized":"2024-03-29T09:11:45.497581+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:28.792665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497822+0000","last_change":"2024-03-29T09:11:43.682084+0000","last_active":"2024-03-29T09:11:45.497822+0000","last_peered":"2024-03-29T09:11:45.497822+0000","last_clean":"2024-03-29T09:11:45.497822+0000","last_became_active":"2024-03-29T09:11:43.681789+0000","last_became_peered":"2024-03-29T09:11:43.681789+0000","last_unstale":"2024-03-29T09:11:45.497822+0000","last_undegraded":"2024-03-29T09:11:45.497822+0000","last_fullsized":"2024-03-29T09:11:45.497822+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:41:59.385726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497200+0000","last_change":"2024-03-29T09:11:43.179870+0000","last_active":"2024-03-29T09:11:45.497200+0000","last_peered":"2024-03-29T09:11:45.497200+0000","last_clean":"2024-03-29T09:11:45.497200+0000","last_became_active":"2024-03-29T09:11:43.179590+0000","last_became_peered":"2024-03-29T09:11:43.179590+0000","last_unstale":"2024-03-29T09:11:45.497200+0000","last_undegraded":"2024-03-29T09:11:45.497200+0000","last_fullsized":"2024-03-29T09:11:45.497200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:13:28.738464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497186+0000","last_change":"2024-03-29T09:11:43.684574+0000","last_active":"2024-03-29T09:11:45.497186+0000","last_peered":"2024-03-29T09:11:45.497186+0000","last_clean":"2024-03-29T09:11:45.497186+0000","last_became_active":"2024-03-29T09:11:43.684273+0000","last_became_peered":"2024-03-29T09:11:43.684273+0000","last_unstale":"2024-03-29T09:11:45.497186+0000","last_undegraded":"2024-03-29T09:11:45.497186+0000","last_fullsized":"2024-03-29T09:11:45.497186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:21.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499015+0000","last_change":"2024-03-29T09:11:43.177261+0000","last_active":"2024-03-29T09:11:45.499015+0000","last_peered":"2024-03-29T09:11:45.499015+0000","last_clean":"2024-03-29T09:11:45.499015+0000","last_became_active":"2024-03-29T09:11:43.176994+0000","last_became_peered":"2024-03-29T09:11:43.176994+0000","last_unstale":"2024-03-29T09:11:45.499015+0000","last_undegraded":"2024-03-29T09:11:45.499015+0000","last_fullsized":"2024-03-29T09:11:45.499015+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:33.227274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497092+0000","last_change":"2024-03-29T09:11:43.677761+0000","last_active":"2024-03-29T09:11:45.497092+0000","last_peered":"2024-03-29T09:11:45.497092+0000","last_clean":"2024-03-29T09:11:45.497092+0000","last_became_active":"2024-03-29T09:11:43.677445+0000","last_became_peered":"2024-03-29T09:11:43.677445+0000","last_unstale":"2024-03-29T09:11:45.497092+0000","last_undegraded":"2024-03-29T09:11:45.497092+0000","last_fullsized":"2024-03-29T09:11:45.497092+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:34.234919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496977+0000","last_change":"2024-03-29T09:11:43.174858+0000","last_active":"2024-03-29T09:11:45.496977+0000","last_peered":"2024-03-29T09:11:45.496977+0000","last_clean":"2024-03-29T09:11:45.496977+0000","last_became_active":"2024-03-29T09:11:43.174559+0000","last_became_peered":"2024-03-29T09:11:43.174559+0000","last_unstale":"2024-03-29T09:11:45.496977+0000","last_undegraded":"2024-03-29T09:11:45.496977+0000","last_fullsized":"2024-03-29T09:11:45.496977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:01.878804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496861+0000","last_change":"2024-03-29T09:11:43.677801+0000","last_active":"2024-03-29T09:11:45.496861+0000","last_peered":"2024-03-29T09:11:45.496861+0000","last_clean":"2024-03-29T09:11:45.496861+0000","last_became_active":"2024-03-29T09:11:43.677410+0000","last_became_peered":"2024-03-29T09:11:43.677410+0000","last_unstale":"2024-03-29T09:11:45.496861+0000","last_undegraded":"2024-03-29T09:11:45.496861+0000","last_fullsized":"2024-03-29T09:11:45.496861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497264+0000","last_change":"2024-03-29T09:11:43.178139+0000","last_active":"2024-03-29T09:11:45.497264+0000","last_peered":"2024-03-29T09:11:45.497264+0000","last_clean":"2024-03-29T09:11:45.497264+0000","last_became_active":"2024-03-29T09:11:43.176620+0000","last_became_peered":"2024-03-29T09:11:43.176620+0000","last_unstale":"2024-03-29T09:11:45.497264+0000","last_undegraded":"2024-03-29T09:11:45.497264+0000","last_fullsized":"2024-03-29T09:11:45.497264+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:44.603438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497741+0000","last_change":"2024-03-29T09:11:43.666170+0000","last_active":"2024-03-29T09:11:45.497741+0000","last_peered":"2024-03-29T09:11:45.497741+0000","last_clean":"2024-03-29T09:11:45.497741+0000","last_became_active":"2024-03-29T09:11:43.665889+0000","last_became_peered":"2024-03-29T09:11:43.665889+0000","last_unstale":"2024-03-29T09:11:45.497741+0000","last_undegraded":"2024-03-29T09:11:45.497741+0000","last_fullsized":"2024-03-29T09:11:45.497741+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667507+0000","last_change":"2024-03-29T09:11:43.176436+0000","last_active":"2024-03-29T09:11:44.667507+0000","last_peered":"2024-03-29T09:11:44.667507+0000","last_clean":"2024-03-29T09:11:44.667507+0000","last_became_active":"2024-03-29T09:11:43.176164+0000","last_became_peered":"2024-03-29T09:11:43.176164+0000","last_unstale":"2024-03-29T09:11:44.667507+0000","last_undegraded":"2024-03-29T09:11:44.667507+0000","last_fullsized":"2024-03-29T09:11:44.667507+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:54:51.708243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497972+0000","last_change":"2024-03-29T09:11:43.677579+0000","last_active":"2024-03-29T09:11:45.497972+0000","last_peered":"2024-03-29T09:11:45.497972+0000","last_clean":"2024-03-29T09:11:45.497972+0000","last_became_active":"2024-03-29T09:11:43.676104+0000","last_became_peered":"2024-03-29T09:11:43.676104+0000","last_unstale":"2024-03-29T09:11:45.497972+0000","last_undegraded":"2024-03-29T09:11:45.497972+0000","last_fullsized":"2024-03-29T09:11:45.497972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:26.946110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996502+0000","last_change":"2024-03-29T09:11:43.182966+0000","last_active":"2024-03-29T09:11:44.996502+0000","last_peered":"2024-03-29T09:11:44.996502+0000","last_clean":"2024-03-29T09:11:44.996502+0000","last_became_active":"2024-03-29T09:11:43.182644+0000","last_became_peered":"2024-03-29T09:11:43.182644+0000","last_unstale":"2024-03-29T09:11:44.996502+0000","last_undegraded":"2024-03-29T09:11:44.996502+0000","last_fullsized":"2024-03-29T09:11:44.996502+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:36.913030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667050+0000","last_change":"2024-03-29T09:11:43.676032+0000","last_active":"2024-03-29T09:11:44.667050+0000","last_peered":"2024-03-29T09:11:44.667050+0000","last_clean":"2024-03-29T09:11:44.667050+0000","last_became_active":"2024-03-29T09:11:43.675680+0000","last_became_peered":"2024-03-29T09:11:43.675680+0000","last_unstale":"2024-03-29T09:11:44.667050+0000","last_undegraded":"2024-03-29T09:11:44.667050+0000","last_fullsized":"2024-03-29T09:11:44.667050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:04:14.657461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497378+0000","last_change":"2024-03-29T09:11:43.171894+0000","last_active":"2024-03-29T09:11:45.497378+0000","last_peered":"2024-03-29T09:11:45.497378+0000","last_clean":"2024-03-29T09:11:45.497378+0000","last_became_active":"2024-03-29T09:11:43.171636+0000","last_became_peered":"2024-03-29T09:11:43.171636+0000","last_unstale":"2024-03-29T09:11:45.497378+0000","last_undegraded":"2024-03-29T09:11:45.497378+0000","last_fullsized":"2024-03-29T09:11:45.497378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:27.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497284+0000","last_change":"2024-03-29T09:11:43.678609+0000","last_active":"2024-03-29T09:11:45.497284+0000","last_peered":"2024-03-29T09:11:45.497284+0000","last_clean":"2024-03-29T09:11:45.497284+0000","last_became_active":"2024-03-29T09:11:43.678256+0000","last_became_peered":"2024-03-29T09:11:43.678256+0000","last_unstale":"2024-03-29T09:11:45.497284+0000","last_undegraded":"2024-03-29T09:11:45.497284+0000","last_fullsized":"2024-03-29T09:11:45.497284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:23.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497444+0000","last_change":"2024-03-29T09:11:43.175598+0000","last_active":"2024-03-29T09:11:45.497444+0000","last_peered":"2024-03-29T09:11:45.497444+0000","last_clean":"2024-03-29T09:11:45.497444+0000","last_became_active":"2024-03-29T09:11:43.175354+0000","last_became_peered":"2024-03-29T09:11:43.175354+0000","last_unstale":"2024-03-29T09:11:45.497444+0000","last_undegraded":"2024-03-29T09:11:45.497444+0000","last_fullsized":"2024-03-29T09:11:45.497444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:23:19.175144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996580+0000","last_change":"2024-03-29T09:11:43.691405+0000","last_active":"2024-03-29T09:11:44.996580+0000","last_peered":"2024-03-29T09:11:44.996580+0000","last_clean":"2024-03-29T09:11:44.996580+0000","last_became_active":"2024-03-29T09:11:43.691072+0000","last_became_peered":"2024-03-29T09:11:43.691072+0000","last_unstale":"2024-03-29T09:11:44.996580+0000","last_undegraded":"2024-03-29T09:11:44.996580+0000","last_fullsized":"2024-03-29T09:11:44.996580+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:32:46.071316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667321+0000","last_change":"2024-03-29T09:11:43.179767+0000","last_active":"2024-03-29T09:11:44.667321+0000","last_peered":"2024-03-29T09:11:44.667321+0000","last_clean":"2024-03-29T09:11:44.667321+0000","last_became_active":"2024-03-29T09:11:43.179487+0000","last_became_peered":"2024-03-29T09:11:43.179487+0000","last_unstale":"2024-03-29T09:11:44.667321+0000","last_undegraded":"2024-03-29T09:11:44.667321+0000","last_fullsized":"2024-03-29T09:11:44.667321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:27.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667726+0000","last_change":"2024-03-29T09:11:43.687900+0000","last_active":"2024-03-29T09:11:44.667726+0000","last_peered":"2024-03-29T09:11:44.667726+0000","last_clean":"2024-03-29T09:11:44.667726+0000","last_became_active":"2024-03-29T09:11:43.687446+0000","last_became_peered":"2024-03-29T09:11:43.687446+0000","last_unstale":"2024-03-29T09:11:44.667726+0000","last_undegraded":"2024-03-29T09:11:44.667726+0000","last_fullsized":"2024-03-29T09:11:44.667726+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:51.772809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497118+0000","last_change":"2024-03-29T09:11:43.175422+0000","last_active":"2024-03-29T09:11:45.497118+0000","last_peered":"2024-03-29T09:11:45.497118+0000","last_clean":"2024-03-29T09:11:45.497118+0000","last_became_active":"2024-03-29T09:11:43.175145+0000","last_became_peered":"2024-03-29T09:11:43.175145+0000","last_unstale":"2024-03-29T09:11:45.497118+0000","last_undegraded":"2024-03-29T09:11:45.497118+0000","last_fullsized":"2024-03-29T09:11:45.497118+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:04.086596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496735+0000","last_change":"2024-03-29T09:11:43.683642+0000","last_active":"2024-03-29T09:11:45.496735+0000","last_peered":"2024-03-29T09:11:45.496735+0000","last_clean":"2024-03-29T09:11:45.496735+0000","last_became_active":"2024-03-29T09:11:43.680296+0000","last_became_peered":"2024-03-29T09:11:43.680296+0000","last_unstale":"2024-03-29T09:11:45.496735+0000","last_undegraded":"2024-03-29T09:11:45.496735+0000","last_fullsized":"2024-03-29T09:11:45.496735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:06.103995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996134+0000","last_change":"2024-03-29T09:11:43.177125+0000","last_active":"2024-03-29T09:11:44.996134+0000","last_peered":"2024-03-29T09:11:44.996134+0000","last_clean":"2024-03-29T09:11:44.996134+0000","last_became_active":"2024-03-29T09:11:43.176802+0000","last_became_peered":"2024-03-29T09:11:43.176802+0000","last_unstale":"2024-03-29T09:11:44.996134+0000","last_undegraded":"2024-03-29T09:11:44.996134+0000","last_fullsized":"2024-03-29T09:11:44.996134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:31.523236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667170+0000","last_change":"2024-03-29T09:11:43.675090+0000","last_active":"2024-03-29T09:11:44.667170+0000","last_peered":"2024-03-29T09:11:44.667170+0000","last_clean":"2024-03-29T09:11:44.667170+0000","last_became_active":"2024-03-29T09:11:43.674631+0000","last_became_peered":"2024-03-29T09:11:43.674631+0000","last_unstale":"2024-03-29T09:11:44.667170+0000","last_undegraded":"2024-03-29T09:11:44.667170+0000","last_fullsized":"2024-03-29T09:11:44.667170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:00:54.297415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496892+0000","last_change":"2024-03-29T09:11:43.174863+0000","last_active":"2024-03-29T09:11:45.496892+0000","last_peered":"2024-03-29T09:11:45.496892+0000","last_clean":"2024-03-29T09:11:45.496892+0000","last_became_active":"2024-03-29T09:11:43.174568+0000","last_became_peered":"2024-03-29T09:11:43.174568+0000","last_unstale":"2024-03-29T09:11:45.496892+0000","last_undegraded":"2024-03-29T09:11:45.496892+0000","last_fullsized":"2024-03-29T09:11:45.496892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:48:29.840684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497110+0000","last_change":"2024-03-29T09:11:43.667721+0000","last_active":"2024-03-29T09:11:45.497110+0000","last_peered":"2024-03-29T09:11:45.497110+0000","last_clean":"2024-03-29T09:11:45.497110+0000","last_became_active":"2024-03-29T09:11:43.667442+0000","last_became_peered":"2024-03-29T09:11:43.667442+0000","last_unstale":"2024-03-29T09:11:45.497110+0000","last_undegraded":"2024-03-29T09:11:45.497110+0000","last_fullsized":"2024-03-29T09:11:45.497110+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:37.929321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667517+0000","last_change":"2024-03-29T09:11:43.181790+0000","last_active":"2024-03-29T09:11:44.667517+0000","last_peered":"2024-03-29T09:11:44.667517+0000","last_clean":"2024-03-29T09:11:44.667517+0000","last_became_active":"2024-03-29T09:11:43.181311+0000","last_became_peered":"2024-03-29T09:11:43.181311+0000","last_unstale":"2024-03-29T09:11:44.667517+0000","last_undegraded":"2024-03-29T09:11:44.667517+0000","last_fullsized":"2024-03-29T09:11:44.667517+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:06:19.992268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-29T09:11:43.663437+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:11:43.663437+0000","last_peered":"2024-03-29T09:11:43.663437+0000","last_clean":"2024-03-29T09:11:43.663437+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:11:43.663437+0000","last_undegraded":"2024-03-29T09:11:43.663437+0000","last_fullsized":"2024-03-29T09:11:43.663437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":293965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.40500000000000003,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76000000000000001,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.498},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.49399999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93799999999999994,"5min":1.639,"15min":1.639},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.84899999999999998,"15min":0.88100000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.753,"15min":0.77600000000000002},"min":{"1min":0.48599999999999999,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.83099999999999996,"5min":1.992,"15min":1.992},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.81699999999999995,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.85799999999999998,"15min":0.88200000000000001},"min":{"1min":0.52600000000000002,"5min":0.495,"15min":0.495},"max":{"1min":0.92000000000000004,"5min":2.883,"15min":2.883},"last":0.85099999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.51800000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80400000000000005,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.745,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91600000000000004,"5min":1.387,"15min":1.387},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.54400000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94499999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.57799999999999996,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.0589999999999999,"5min":1.367,"15min":1.367},"last":0.873}]},{"osd":5,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.755,"15min":0.76900000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0209999999999999,"5min":1.458,"15min":1.458},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72499999999999998,"15min":0.73799999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.84499999999999997,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:10:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76800000000000002,"15min":0.77700000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.88400000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.53300000000000003,"5min":0.502,"15min":0.502},"max":{"1min":0.96299999999999997,"5min":1.413,"15min":1.413},"last":0.72799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":293895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.97799999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.84799999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.81200000000000006,"15min":0.83799999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.96099999999999997,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.91000000000000003}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.48999999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.45800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.74199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.745}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.79500000000000004,"15min":0.80200000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.006,"5min":1.026,"15min":1.026},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":884823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.175,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.5259999999999998,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.64900000000000002,"15min":0.68300000000000005},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.57199999999999995,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66700000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.008,"15min":0.94899999999999995},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":12.356,"5min":12.356,"15min":12.356},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.73899999999999999,"15min":0.71499999999999997},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.91700000000000004,"15min":0.93799999999999994},"min":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6279999999999999,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83399999999999996,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.903,"5min":1.99,"15min":1.99},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.0609999999999999,"15min":0.99399999999999999},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":12.669,"5min":12.669,"15min":12.669},"last":0.627}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.874,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.75700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":295437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.51300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.51700000000000002,"5min":0.751,"15min":0.751},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.436},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67800000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":1.028,"15min":1.028},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99099999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.748}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.069,"15min":1.069},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.02,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.82599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":295597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.47399999999999998,"15min":0.48799999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":1.252,"15min":1.252},"last":0.47699999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61899999999999999,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66900000000000004,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.746,"5min":2.415,"15min":2.415},"last":0.496},{"interface":"front","average":{"1min":0.5,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":2.056,"15min":2.056},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91400000000000003,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":2.073,"15min":2.073},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92500000000000004,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95199999999999996,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.878,"15min":0.90300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96799999999999997,"5min":2.387,"15min":2.387},"last":0.628},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.91700000000000004,"15min":0.94699999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.91500000000000004,"5min":2.996,"15min":2.996},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95499999999999996,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84999999999999998,"15min":0.874},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88800000000000001,"5min":2.165,"15min":2.165},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":293963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.53200000000000003,"15min":0.55100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61599999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.496,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.53900000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76300000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99299999999999999,"5min":2.278,"15min":2.278},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96399999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.59999999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88400000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.745}]},{"osd":6,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.90000000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":2.649,"15min":2.649},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.034,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.71999999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":294489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1319999999999999,"5min":1.742,"15min":1.742},"last":0.873},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.77700000000000002,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.751,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2130000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.65800000000000003}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84199999999999997,"5min":0.999,"15min":0.999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91300000000000003,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.84599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.871,"15min":0.90100000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0269999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.94399999999999995,"15min":0.97599999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.073,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.72399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":884275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81799999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85499999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.872,"15min":0.872},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83699999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.91600000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":0.873},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73499999999999999,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.81100000000000005,"5min":1.05,"15min":1.05},"last":0.502}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.83099999999999996,"5min":1.079,"15min":1.079},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.877,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.755},{"interface":"front","average":{"1min":0.752,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.74199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:11:52.973 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:11:53.163 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:11:53.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:11:53.173 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.174+0000 7fb581800700 1 Processor -- start 2024-03-29T09:11:53.174 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.174+0000 7fb581800700 1 -- start start 2024-03-29T09:11:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb581800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb57c1608e0 0x7fb57c0d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb581800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c0d1b20 0x7fb57c0ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb581800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c0cddc0 con 0x7fb57c160cc0 2024-03-29T09:11:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb581800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57c0ce0a0 con 0x7fb57c1608e0 2024-03-29T09:11:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb581800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0ce380 con 0x7fb57c0d1b20 2024-03-29T09:11:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb57a000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb57c160cc0 0x7fb57c0d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48246/0 (socket says 172.21.15.99:48246) 2024-03-29T09:11:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb57a000700 1 -- 172.21.15.99:0/1705869454 learned_addr learned my addr 172.21.15.99:0/1705869454 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb57b400700 1 --2- 172.21.15.99:0/1705869454 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c0d1b20 0x7fb57c0ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.176+0000 7fb57aa00700 1 --2- 172.21.15.99:0/1705869454 >> v2:172.21.15.133:3300/0 conn(0x7fb57c1608e0 0x7fb57c0d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb579600700 1 -- 172.21.15.99:0/1705869454 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593590828 0 0) 0x7fb57c0cddc0 con 0x7fb57c160cc0 2024-03-29T09:11:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb579600700 1 -- 172.21.15.99:0/1705869454 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5600035b0 con 0x7fb57c160cc0 2024-03-29T09:11:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb57b400700 1 -- 172.21.15.99:0/1705869454 >> v2:172.21.15.133:3300/0 conn(0x7fb57c1608e0 msgr2=0x7fb57c0d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:53.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb57b400700 1 --2- 172.21.15.99:0/1705869454 >> v2:172.21.15.133:3300/0 conn(0x7fb57c1608e0 0x7fb57c0d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb57b400700 1 -- 172.21.15.99:0/1705869454 >> v1:172.21.15.99:6789/0 conn(0x7fb57c160cc0 legacy=0x7fb57c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb57b400700 1 -- 172.21.15.99:0/1705869454 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0ce660 con 0x7fb57c0d1b20 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb57aa00700 1 --2- 172.21.15.99:0/1705869454 >> v2:172.21.15.133:3300/0 conn(0x7fb57c1608e0 0x7fb57c0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.177+0000 7fb57b400700 1 --2- 172.21.15.99:0/1705869454 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c0d1b20 0x7fb57c0ceb80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb574009d90 tx=0x7fb57400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d580e590ae117b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb579600700 1 -- 172.21.15.99:0/1705869454 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400e070 con 0x7fb57c0d1b20 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb579600700 1 -- 172.21.15.99:0/1705869454 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57400bb20 con 0x7fb57c0d1b20 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb579600700 1 -- 172.21.15.99:0/1705869454 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574012430 con 0x7fb57c0d1b20 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 -- 172.21.15.99:0/1705869454 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c0d1b20 msgr2=0x7fb57c0ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 --2- 172.21.15.99:0/1705869454 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c0d1b20 0x7fb57c0ceb80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb574009d90 tx=0x7fb57400b330 comp rx=0 tx=0).stop 2024-03-29T09:11:53.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 -- 172.21.15.99:0/1705869454 shutdown_connections 2024-03-29T09:11:53.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 --2- 172.21.15.99:0/1705869454 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c0d1b20 0x7fb57c0ceb80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 --2- 172.21.15.99:0/1705869454 >> v2:172.21.15.133:3300/0 conn(0x7fb57c1608e0 0x7fb57c0d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 -- 172.21.15.99:0/1705869454 >> 172.21.15.99:0/1705869454 conn(0x7fb57c09c490 msgr2=0x7fb57c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:53.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 -- 172.21.15.99:0/1705869454 shutdown_connections 2024-03-29T09:11:53.183 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.178+0000 7fb581800700 1 -- 172.21.15.99:0/1705869454 wait complete. 2024-03-29T09:11:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.181+0000 7fb581800700 1 Processor -- start 2024-03-29T09:11:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.181+0000 7fb581800700 1 -- start start 2024-03-29T09:11:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb581800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57c0d1b20 0x7fb57c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb581800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c1608e0 0x7fb57c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb581800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57c160cc0 0x7fb57c0b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb581800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0cb0b0 con 0x7fb57c160cc0 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb581800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0ca220 con 0x7fb57c0d1b20 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb581800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c1536e0 con 0x7fb57c1608e0 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb57b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57c160cc0 0x7fb57c0b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb57b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57c160cc0 0x7fb57c0b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57688/0 (socket says 172.21.15.99:57688) 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb57b400700 1 -- 172.21.15.99:0/1790702658 learned_addr learned my addr 172.21.15.99:0/1790702658 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb57a000700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c1608e0 0x7fb57c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.182+0000 7fb57aa00700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57c0d1b20 0x7fb57c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb57b400700 1 -- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c1608e0 msgr2=0x7fb57c156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb57b400700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c1608e0 0x7fb57c156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb57b400700 1 -- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57c0d1b20 msgr2=0x7fb57c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb57b400700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57c0d1b20 0x7fb57c1531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb57b400700 1 -- 172.21.15.99:0/1790702658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0094a0 con 0x7fb57c160cc0 2024-03-29T09:11:53.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb57b400700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57c160cc0 0x7fb57c0b3470 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb574014040 tx=0x7fb574004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:53.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb572a00700 1 -- 172.21.15.99:0/1790702658 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740129f0 con 0x7fb57c160cc0 2024-03-29T09:11:53.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb572a00700 1 -- 172.21.15.99:0/1790702658 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb574020070 con 0x7fb57c160cc0 2024-03-29T09:11:53.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb574009a70 con 0x7fb57c160cc0 2024-03-29T09:11:53.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb572a00700 1 -- 172.21.15.99:0/1790702658 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574012db0 con 0x7fb57c160cc0 2024-03-29T09:11:53.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.183+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c0ca430 con 0x7fb57c160cc0 2024-03-29T09:11:53.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.185+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57c0cb1f0 con 0x7fb57c160cc0 2024-03-29T09:11:53.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.185+0000 7fb572a00700 1 -- 172.21.15.99:0/1790702658 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb574004040 con 0x7fb57c160cc0 2024-03-29T09:11:53.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.186+0000 7fb572a00700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55c06ac90 0x7fb55c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.186+0000 7fb572a00700 1 -- 172.21.15.99:0/1790702658 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb574022070 con 0x7fb57c160cc0 2024-03-29T09:11:53.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.187+0000 7fb57aa00700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55c06ac90 0x7fb55c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.187+0000 7fb57aa00700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55c06ac90 0x7fb55c06d150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009d70 tx=0x7fb56c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:53.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.191+0000 7fb572a00700 1 -- 172.21.15.99:0/1790702658 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb57c0cb1f0 con 0x7fb57c160cc0 2024-03-29T09:11:53.396 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.397+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb57c0b22a0 con 0x7fb55c06ac90 2024-03-29T09:11:53.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.405+0000 7fb572a00700 1 -- 172.21.15.99:0/1790702658 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289153 (secure 0 0 0) 0x7fb56c0099c0 con 0x7fb55c06ac90 2024-03-29T09:11:53.409 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:53.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55c06ac90 msgr2=0x7fb55c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55c06ac90 0x7fb55c06d150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009d70 tx=0x7fb56c009210 comp rx=0 tx=0).stop 2024-03-29T09:11:53.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57c160cc0 msgr2=0x7fb57c0b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57c160cc0 0x7fb57c0b3470 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb574014040 tx=0x7fb574004ad0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 shutdown_connections 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb55c06ac90 0x7fb55c06d150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb57c160cc0 0x7fb57c0b3470 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb57c0d1b20 0x7fb57c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 --2- 172.21.15.99:0/1790702658 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb57c1608e0 0x7fb57c156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 >> 172.21.15.99:0/1790702658 conn(0x7fb57c09c490 msgr2=0x7fb57c0b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.409+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 shutdown_connections 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.410+0000 7fb581800700 1 -- 172.21.15.99:0/1790702658 wait complete. 2024-03-29T09:11:53.412 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:11:53.427 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-03-29T09:11:52.487527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222552,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752168,"statfs":{"total":773094113280,"available":772866220032,"internally_reserved":0,"allocated":6529024,"data_stored":3536444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.832011"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996579+0000","last_change":"2024-03-29T09:11:43.690372+0000","last_active":"2024-03-29T09:11:44.996579+0000","last_peered":"2024-03-29T09:11:44.996579+0000","last_clean":"2024-03-29T09:11:44.996579+0000","last_became_active":"2024-03-29T09:11:43.690046+0000","last_became_peered":"2024-03-29T09:11:43.690046+0000","last_unstale":"2024-03-29T09:11:44.996579+0000","last_undegraded":"2024-03-29T09:11:44.996579+0000","last_fullsized":"2024-03-29T09:11:44.996579+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:23.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667247+0000","last_change":"2024-03-29T09:11:43.681209+0000","last_active":"2024-03-29T09:11:44.667247+0000","last_peered":"2024-03-29T09:11:44.667247+0000","last_clean":"2024-03-29T09:11:44.667247+0000","last_became_active":"2024-03-29T09:11:43.680965+0000","last_became_peered":"2024-03-29T09:11:43.680965+0000","last_unstale":"2024-03-29T09:11:44.667247+0000","last_undegraded":"2024-03-29T09:11:44.667247+0000","last_fullsized":"2024-03-29T09:11:44.667247+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:32.530881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497341+0000","last_change":"2024-03-29T09:11:43.672222+0000","last_active":"2024-03-29T09:11:45.497341+0000","last_peered":"2024-03-29T09:11:45.497341+0000","last_clean":"2024-03-29T09:11:45.497341+0000","last_became_active":"2024-03-29T09:11:43.671980+0000","last_became_peered":"2024-03-29T09:11:43.671980+0000","last_unstale":"2024-03-29T09:11:45.497341+0000","last_undegraded":"2024-03-29T09:11:45.497341+0000","last_fullsized":"2024-03-29T09:11:45.497341+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:26:11.492718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497818+0000","last_change":"2024-03-29T09:11:43.689409+0000","last_active":"2024-03-29T09:11:45.497818+0000","last_peered":"2024-03-29T09:11:45.497818+0000","last_clean":"2024-03-29T09:11:45.497818+0000","last_became_active":"2024-03-29T09:11:43.689142+0000","last_became_peered":"2024-03-29T09:11:43.689142+0000","last_unstale":"2024-03-29T09:11:45.497818+0000","last_undegraded":"2024-03-29T09:11:45.497818+0000","last_fullsized":"2024-03-29T09:11:45.497818+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:45:02.972724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996181+0000","last_change":"2024-03-29T09:11:43.685296+0000","last_active":"2024-03-29T09:11:44.996181+0000","last_peered":"2024-03-29T09:11:44.996181+0000","last_clean":"2024-03-29T09:11:44.996181+0000","last_became_active":"2024-03-29T09:11:43.682430+0000","last_became_peered":"2024-03-29T09:11:43.682430+0000","last_unstale":"2024-03-29T09:11:44.996181+0000","last_undegraded":"2024-03-29T09:11:44.996181+0000","last_fullsized":"2024-03-29T09:11:44.996181+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996027+0000","last_change":"2024-03-29T09:11:43.691095+0000","last_active":"2024-03-29T09:11:44.996027+0000","last_peered":"2024-03-29T09:11:44.996027+0000","last_clean":"2024-03-29T09:11:44.996027+0000","last_became_active":"2024-03-29T09:11:43.690828+0000","last_became_peered":"2024-03-29T09:11:43.690828+0000","last_unstale":"2024-03-29T09:11:44.996027+0000","last_undegraded":"2024-03-29T09:11:44.996027+0000","last_fullsized":"2024-03-29T09:11:44.996027+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:34:41.067618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497922+0000","last_change":"2024-03-29T09:11:43.685480+0000","last_active":"2024-03-29T09:11:45.497922+0000","last_peered":"2024-03-29T09:11:45.497922+0000","last_clean":"2024-03-29T09:11:45.497922+0000","last_became_active":"2024-03-29T09:11:43.683700+0000","last_became_peered":"2024-03-29T09:11:43.683700+0000","last_unstale":"2024-03-29T09:11:45.497922+0000","last_undegraded":"2024-03-29T09:11:45.497922+0000","last_fullsized":"2024-03-29T09:11:45.497922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T21:00:03.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496748+0000","last_change":"2024-03-29T09:11:43.677985+0000","last_active":"2024-03-29T09:11:45.496748+0000","last_peered":"2024-03-29T09:11:45.496748+0000","last_clean":"2024-03-29T09:11:45.496748+0000","last_became_active":"2024-03-29T09:11:43.677658+0000","last_became_peered":"2024-03-29T09:11:43.677658+0000","last_unstale":"2024-03-29T09:11:45.496748+0000","last_undegraded":"2024-03-29T09:11:45.496748+0000","last_fullsized":"2024-03-29T09:11:45.496748+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:04.674681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497195+0000","last_change":"2024-03-29T09:11:43.681141+0000","last_active":"2024-03-29T09:11:45.497195+0000","last_peered":"2024-03-29T09:11:45.497195+0000","last_clean":"2024-03-29T09:11:45.497195+0000","last_became_active":"2024-03-29T09:11:43.680651+0000","last_became_peered":"2024-03-29T09:11:43.680651+0000","last_unstale":"2024-03-29T09:11:45.497195+0000","last_undegraded":"2024-03-29T09:11:45.497195+0000","last_fullsized":"2024-03-29T09:11:45.497195+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:20.720413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996423+0000","last_change":"2024-03-29T09:11:43.689627+0000","last_active":"2024-03-29T09:11:44.996423+0000","last_peered":"2024-03-29T09:11:44.996423+0000","last_clean":"2024-03-29T09:11:44.996423+0000","last_became_active":"2024-03-29T09:11:43.689352+0000","last_became_peered":"2024-03-29T09:11:43.689352+0000","last_unstale":"2024-03-29T09:11:44.996423+0000","last_undegraded":"2024-03-29T09:11:44.996423+0000","last_fullsized":"2024-03-29T09:11:44.996423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:32:04.674681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499008+0000","last_change":"2024-03-29T09:11:43.684198+0000","last_active":"2024-03-29T09:11:45.499008+0000","last_peered":"2024-03-29T09:11:45.499008+0000","last_clean":"2024-03-29T09:11:45.499008+0000","last_became_active":"2024-03-29T09:11:43.683881+0000","last_became_peered":"2024-03-29T09:11:43.683881+0000","last_unstale":"2024-03-29T09:11:45.499008+0000","last_undegraded":"2024-03-29T09:11:45.499008+0000","last_fullsized":"2024-03-29T09:11:45.499008+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:17:07.814736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497080+0000","last_change":"2024-03-29T09:11:43.680518+0000","last_active":"2024-03-29T09:11:45.497080+0000","last_peered":"2024-03-29T09:11:45.497080+0000","last_clean":"2024-03-29T09:11:45.497080+0000","last_became_active":"2024-03-29T09:11:43.680221+0000","last_became_peered":"2024-03-29T09:11:43.680221+0000","last_unstale":"2024-03-29T09:11:45.497080+0000","last_undegraded":"2024-03-29T09:11:45.497080+0000","last_fullsized":"2024-03-29T09:11:45.497080+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:04:14.657461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667609+0000","last_change":"2024-03-29T09:11:43.688051+0000","last_active":"2024-03-29T09:11:44.667609+0000","last_peered":"2024-03-29T09:11:44.667609+0000","last_clean":"2024-03-29T09:11:44.667609+0000","last_became_active":"2024-03-29T09:11:43.687664+0000","last_became_peered":"2024-03-29T09:11:43.687664+0000","last_unstale":"2024-03-29T09:11:44.667609+0000","last_undegraded":"2024-03-29T09:11:44.667609+0000","last_fullsized":"2024-03-29T09:11:44.667609+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:37.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497047+0000","last_change":"2024-03-29T09:11:43.665919+0000","last_active":"2024-03-29T09:11:45.497047+0000","last_peered":"2024-03-29T09:11:45.497047+0000","last_clean":"2024-03-29T09:11:45.497047+0000","last_became_active":"2024-03-29T09:11:43.665662+0000","last_became_peered":"2024-03-29T09:11:43.665662+0000","last_unstale":"2024-03-29T09:11:45.497047+0000","last_undegraded":"2024-03-29T09:11:45.497047+0000","last_fullsized":"2024-03-29T09:11:45.497047+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:25.840707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667467+0000","last_change":"2024-03-29T09:11:43.683560+0000","last_active":"2024-03-29T09:11:44.667467+0000","last_peered":"2024-03-29T09:11:44.667467+0000","last_clean":"2024-03-29T09:11:44.667467+0000","last_became_active":"2024-03-29T09:11:43.683227+0000","last_became_peered":"2024-03-29T09:11:43.683227+0000","last_unstale":"2024-03-29T09:11:44.667467+0000","last_undegraded":"2024-03-29T09:11:44.667467+0000","last_fullsized":"2024-03-29T09:11:44.667467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:34.499347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996022+0000","last_change":"2024-03-29T09:11:43.680836+0000","last_active":"2024-03-29T09:11:44.996022+0000","last_peered":"2024-03-29T09:11:44.996022+0000","last_clean":"2024-03-29T09:11:44.996022+0000","last_became_active":"2024-03-29T09:11:43.680568+0000","last_became_peered":"2024-03-29T09:11:43.680568+0000","last_unstale":"2024-03-29T09:11:44.996022+0000","last_undegraded":"2024-03-29T09:11:44.996022+0000","last_fullsized":"2024-03-29T09:11:44.996022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:03.217442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996654+0000","last_change":"2024-03-29T09:11:43.679345+0000","last_active":"2024-03-29T09:11:44.996654+0000","last_peered":"2024-03-29T09:11:44.996654+0000","last_clean":"2024-03-29T09:11:44.996654+0000","last_became_active":"2024-03-29T09:11:43.678923+0000","last_became_peered":"2024-03-29T09:11:43.678923+0000","last_unstale":"2024-03-29T09:11:44.996654+0000","last_undegraded":"2024-03-29T09:11:44.996654+0000","last_fullsized":"2024-03-29T09:11:44.996654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497557+0000","last_change":"2024-03-29T09:11:43.666174+0000","last_active":"2024-03-29T09:11:45.497557+0000","last_peered":"2024-03-29T09:11:45.497557+0000","last_clean":"2024-03-29T09:11:45.497557+0000","last_became_active":"2024-03-29T09:11:43.665473+0000","last_became_peered":"2024-03-29T09:11:43.665473+0000","last_unstale":"2024-03-29T09:11:45.497557+0000","last_undegraded":"2024-03-29T09:11:45.497557+0000","last_fullsized":"2024-03-29T09:11:45.497557+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667309+0000","last_change":"2024-03-29T09:11:43.671528+0000","last_active":"2024-03-29T09:11:44.667309+0000","last_peered":"2024-03-29T09:11:44.667309+0000","last_clean":"2024-03-29T09:11:44.667309+0000","last_became_active":"2024-03-29T09:11:43.670774+0000","last_became_peered":"2024-03-29T09:11:43.670774+0000","last_unstale":"2024-03-29T09:11:44.667309+0000","last_undegraded":"2024-03-29T09:11:44.667309+0000","last_fullsized":"2024-03-29T09:11:44.667309+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:20.720413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498963+0000","last_change":"2024-03-29T09:11:43.686864+0000","last_active":"2024-03-29T09:11:45.498963+0000","last_peered":"2024-03-29T09:11:45.498963+0000","last_clean":"2024-03-29T09:11:45.498963+0000","last_became_active":"2024-03-29T09:11:43.686418+0000","last_became_peered":"2024-03-29T09:11:43.686418+0000","last_unstale":"2024-03-29T09:11:45.498963+0000","last_undegraded":"2024-03-29T09:11:45.498963+0000","last_fullsized":"2024-03-29T09:11:45.498963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:17.190299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667610+0000","last_change":"2024-03-29T09:11:43.677257+0000","last_active":"2024-03-29T09:11:44.667610+0000","last_peered":"2024-03-29T09:11:44.667610+0000","last_clean":"2024-03-29T09:11:44.667610+0000","last_became_active":"2024-03-29T09:11:43.677003+0000","last_became_peered":"2024-03-29T09:11:43.677003+0000","last_unstale":"2024-03-29T09:11:44.667610+0000","last_undegraded":"2024-03-29T09:11:44.667610+0000","last_fullsized":"2024-03-29T09:11:44.667610+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:02.886449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498962+0000","last_change":"2024-03-29T09:11:43.688297+0000","last_active":"2024-03-29T09:11:45.498962+0000","last_peered":"2024-03-29T09:11:45.498962+0000","last_clean":"2024-03-29T09:11:45.498962+0000","last_became_active":"2024-03-29T09:11:43.688005+0000","last_became_peered":"2024-03-29T09:11:43.688005+0000","last_unstale":"2024-03-29T09:11:45.498962+0000","last_undegraded":"2024-03-29T09:11:45.498962+0000","last_fullsized":"2024-03-29T09:11:45.498962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:20.115771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498025+0000","last_change":"2024-03-29T09:11:43.678188+0000","last_active":"2024-03-29T09:11:45.498025+0000","last_peered":"2024-03-29T09:11:45.498025+0000","last_clean":"2024-03-29T09:11:45.498025+0000","last_became_active":"2024-03-29T09:11:43.677934+0000","last_became_peered":"2024-03-29T09:11:43.677934+0000","last_unstale":"2024-03-29T09:11:45.498025+0000","last_undegraded":"2024-03-29T09:11:45.498025+0000","last_fullsized":"2024-03-29T09:11:45.498025+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:37:26.330731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496826+0000","last_change":"2024-03-29T09:11:43.675646+0000","last_active":"2024-03-29T09:11:45.496826+0000","last_peered":"2024-03-29T09:11:45.496826+0000","last_clean":"2024-03-29T09:11:45.496826+0000","last_became_active":"2024-03-29T09:11:43.675298+0000","last_became_peered":"2024-03-29T09:11:43.675298+0000","last_unstale":"2024-03-29T09:11:45.496826+0000","last_undegraded":"2024-03-29T09:11:45.496826+0000","last_fullsized":"2024-03-29T09:11:45.496826+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:05.122450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667348+0000","last_change":"2024-03-29T09:11:43.680611+0000","last_active":"2024-03-29T09:11:44.667348+0000","last_peered":"2024-03-29T09:11:44.667348+0000","last_clean":"2024-03-29T09:11:44.667348+0000","last_became_active":"2024-03-29T09:11:43.680345+0000","last_became_peered":"2024-03-29T09:11:43.680345+0000","last_unstale":"2024-03-29T09:11:44.667348+0000","last_undegraded":"2024-03-29T09:11:44.667348+0000","last_fullsized":"2024-03-29T09:11:44.667348+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:37.920675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667802+0000","last_change":"2024-03-29T09:11:43.682019+0000","last_active":"2024-03-29T09:11:44.667802+0000","last_peered":"2024-03-29T09:11:44.667802+0000","last_clean":"2024-03-29T09:11:44.667802+0000","last_became_active":"2024-03-29T09:11:43.681763+0000","last_became_peered":"2024-03-29T09:11:43.681763+0000","last_unstale":"2024-03-29T09:11:44.667802+0000","last_undegraded":"2024-03-29T09:11:44.667802+0000","last_fullsized":"2024-03-29T09:11:44.667802+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:48.040844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667365+0000","last_change":"2024-03-29T09:11:43.671717+0000","last_active":"2024-03-29T09:11:44.667365+0000","last_peered":"2024-03-29T09:11:44.667365+0000","last_clean":"2024-03-29T09:11:44.667365+0000","last_became_active":"2024-03-29T09:11:43.671231+0000","last_became_peered":"2024-03-29T09:11:43.671231+0000","last_unstale":"2024-03-29T09:11:44.667365+0000","last_undegraded":"2024-03-29T09:11:44.667365+0000","last_fullsized":"2024-03-29T09:11:44.667365+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:05.122450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496922+0000","last_change":"2024-03-29T09:11:43.680460+0000","last_active":"2024-03-29T09:11:45.496922+0000","last_peered":"2024-03-29T09:11:45.496922+0000","last_clean":"2024-03-29T09:11:45.496922+0000","last_became_active":"2024-03-29T09:11:43.680176+0000","last_became_peered":"2024-03-29T09:11:43.680176+0000","last_unstale":"2024-03-29T09:11:45.496922+0000","last_undegraded":"2024-03-29T09:11:45.496922+0000","last_fullsized":"2024-03-29T09:11:45.496922+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:26.946110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496915+0000","last_change":"2024-03-29T09:11:43.679680+0000","last_active":"2024-03-29T09:11:45.496915+0000","last_peered":"2024-03-29T09:11:45.496915+0000","last_clean":"2024-03-29T09:11:45.496915+0000","last_became_active":"2024-03-29T09:11:43.679397+0000","last_became_peered":"2024-03-29T09:11:43.679397+0000","last_unstale":"2024-03-29T09:11:45.496915+0000","last_undegraded":"2024-03-29T09:11:45.496915+0000","last_fullsized":"2024-03-29T09:11:45.496915+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:16.721779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497963+0000","last_change":"2024-03-29T09:11:43.679106+0000","last_active":"2024-03-29T09:11:45.497963+0000","last_peered":"2024-03-29T09:11:45.497963+0000","last_clean":"2024-03-29T09:11:45.497963+0000","last_became_active":"2024-03-29T09:11:43.678804+0000","last_became_peered":"2024-03-29T09:11:43.678804+0000","last_unstale":"2024-03-29T09:11:45.497963+0000","last_undegraded":"2024-03-29T09:11:45.497963+0000","last_fullsized":"2024-03-29T09:11:45.497963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:48:32.315631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.666942+0000","last_change":"2024-03-29T09:11:43.669556+0000","last_active":"2024-03-29T09:11:44.666942+0000","last_peered":"2024-03-29T09:11:44.666942+0000","last_clean":"2024-03-29T09:11:44.666942+0000","last_became_active":"2024-03-29T09:11:43.669284+0000","last_became_peered":"2024-03-29T09:11:43.669284+0000","last_unstale":"2024-03-29T09:11:44.666942+0000","last_undegraded":"2024-03-29T09:11:44.666942+0000","last_fullsized":"2024-03-29T09:11:44.666942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:37.448155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498776+0000","last_change":"2024-03-29T09:11:43.688520+0000","last_active":"2024-03-29T09:11:45.498776+0000","last_peered":"2024-03-29T09:11:45.498776+0000","last_clean":"2024-03-29T09:11:45.498776+0000","last_became_active":"2024-03-29T09:11:43.688113+0000","last_became_peered":"2024-03-29T09:11:43.688113+0000","last_unstale":"2024-03-29T09:11:45.498776+0000","last_undegraded":"2024-03-29T09:11:45.498776+0000","last_fullsized":"2024-03-29T09:11:45.498776+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:25.840707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996437+0000","last_change":"2024-03-29T09:11:43.690479+0000","last_active":"2024-03-29T09:11:44.996437+0000","last_peered":"2024-03-29T09:11:44.996437+0000","last_clean":"2024-03-29T09:11:44.996437+0000","last_became_active":"2024-03-29T09:11:43.689919+0000","last_became_peered":"2024-03-29T09:11:43.689919+0000","last_unstale":"2024-03-29T09:11:44.996437+0000","last_undegraded":"2024-03-29T09:11:44.996437+0000","last_fullsized":"2024-03-29T09:11:44.996437+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:54.749065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"54'4","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499230+0000","last_change":"2024-03-29T09:11:43.168276+0000","last_active":"2024-03-29T09:11:45.499230+0000","last_peered":"2024-03-29T09:11:45.499230+0000","last_clean":"2024-03-29T09:11:45.499230+0000","last_became_active":"2024-03-29T09:11:43.168028+0000","last_became_peered":"2024-03-29T09:11:43.168028+0000","last_unstale":"2024-03-29T09:11:45.499230+0000","last_undegraded":"2024-03-29T09:11:45.499230+0000","last_fullsized":"2024-03-29T09:11:45.499230+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:48.069355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996656+0000","last_change":"2024-03-29T09:11:43.691607+0000","last_active":"2024-03-29T09:11:44.996656+0000","last_peered":"2024-03-29T09:11:44.996656+0000","last_clean":"2024-03-29T09:11:44.996656+0000","last_became_active":"2024-03-29T09:11:43.691054+0000","last_became_peered":"2024-03-29T09:11:43.691054+0000","last_unstale":"2024-03-29T09:11:44.996656+0000","last_undegraded":"2024-03-29T09:11:44.996656+0000","last_fullsized":"2024-03-29T09:11:44.996656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:37.929321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667464+0000","last_change":"2024-03-29T09:11:43.172794+0000","last_active":"2024-03-29T09:11:44.667464+0000","last_peered":"2024-03-29T09:11:44.667464+0000","last_clean":"2024-03-29T09:11:44.667464+0000","last_became_active":"2024-03-29T09:11:43.172062+0000","last_became_peered":"2024-03-29T09:11:43.172062+0000","last_unstale":"2024-03-29T09:11:44.667464+0000","last_undegraded":"2024-03-29T09:11:44.667464+0000","last_fullsized":"2024-03-29T09:11:44.667464+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:40:54.283138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496997+0000","last_change":"2024-03-29T09:11:43.685364+0000","last_active":"2024-03-29T09:11:45.496997+0000","last_peered":"2024-03-29T09:11:45.496997+0000","last_clean":"2024-03-29T09:11:45.496997+0000","last_became_active":"2024-03-29T09:11:43.685091+0000","last_became_peered":"2024-03-29T09:11:43.685091+0000","last_unstale":"2024-03-29T09:11:45.496997+0000","last_undegraded":"2024-03-29T09:11:45.496997+0000","last_fullsized":"2024-03-29T09:11:45.496997+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:54.749065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996552+0000","last_change":"2024-03-29T09:11:43.173098+0000","last_active":"2024-03-29T09:11:44.996552+0000","last_peered":"2024-03-29T09:11:44.996552+0000","last_clean":"2024-03-29T09:11:44.996552+0000","last_became_active":"2024-03-29T09:11:43.172846+0000","last_became_peered":"2024-03-29T09:11:43.172846+0000","last_unstale":"2024-03-29T09:11:44.996552+0000","last_undegraded":"2024-03-29T09:11:44.996552+0000","last_fullsized":"2024-03-29T09:11:44.996552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:47.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996414+0000","last_change":"2024-03-29T09:11:43.692247+0000","last_active":"2024-03-29T09:11:44.996414+0000","last_peered":"2024-03-29T09:11:44.996414+0000","last_clean":"2024-03-29T09:11:44.996414+0000","last_became_active":"2024-03-29T09:11:43.691491+0000","last_became_peered":"2024-03-29T09:11:43.691491+0000","last_unstale":"2024-03-29T09:11:44.996414+0000","last_undegraded":"2024-03-29T09:11:44.996414+0000","last_fullsized":"2024-03-29T09:11:44.996414+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:52:52.953214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497899+0000","last_change":"2024-03-29T09:11:43.177133+0000","last_active":"2024-03-29T09:11:45.497899+0000","last_peered":"2024-03-29T09:11:45.497899+0000","last_clean":"2024-03-29T09:11:45.497899+0000","last_became_active":"2024-03-29T09:11:43.176799+0000","last_became_peered":"2024-03-29T09:11:43.176799+0000","last_unstale":"2024-03-29T09:11:45.497899+0000","last_undegraded":"2024-03-29T09:11:45.497899+0000","last_fullsized":"2024-03-29T09:11:45.497899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:45:07.789102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497028+0000","last_change":"2024-03-29T09:11:43.683820+0000","last_active":"2024-03-29T09:11:45.497028+0000","last_peered":"2024-03-29T09:11:45.497028+0000","last_clean":"2024-03-29T09:11:45.497028+0000","last_became_active":"2024-03-29T09:11:43.683490+0000","last_became_peered":"2024-03-29T09:11:43.683490+0000","last_unstale":"2024-03-29T09:11:45.497028+0000","last_undegraded":"2024-03-29T09:11:45.497028+0000","last_fullsized":"2024-03-29T09:11:45.497028+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:03.217442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497615+0000","last_change":"2024-03-29T09:11:43.180832+0000","last_active":"2024-03-29T09:11:45.497615+0000","last_peered":"2024-03-29T09:11:45.497615+0000","last_clean":"2024-03-29T09:11:45.497615+0000","last_became_active":"2024-03-29T09:11:43.180518+0000","last_became_peered":"2024-03-29T09:11:43.180518+0000","last_unstale":"2024-03-29T09:11:45.497615+0000","last_undegraded":"2024-03-29T09:11:45.497615+0000","last_fullsized":"2024-03-29T09:11:45.497615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:24:19.226534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498945+0000","last_change":"2024-03-29T09:11:43.688382+0000","last_active":"2024-03-29T09:11:45.498945+0000","last_peered":"2024-03-29T09:11:45.498945+0000","last_clean":"2024-03-29T09:11:45.498945+0000","last_became_active":"2024-03-29T09:11:43.688094+0000","last_became_peered":"2024-03-29T09:11:43.688094+0000","last_unstale":"2024-03-29T09:11:45.498945+0000","last_undegraded":"2024-03-29T09:11:45.498945+0000","last_fullsized":"2024-03-29T09:11:45.498945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:21.218363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996277+0000","last_change":"2024-03-29T09:11:43.183057+0000","last_active":"2024-03-29T09:11:44.996277+0000","last_peered":"2024-03-29T09:11:44.996277+0000","last_clean":"2024-03-29T09:11:44.996277+0000","last_became_active":"2024-03-29T09:11:43.182589+0000","last_became_peered":"2024-03-29T09:11:43.182589+0000","last_unstale":"2024-03-29T09:11:44.996277+0000","last_undegraded":"2024-03-29T09:11:44.996277+0000","last_fullsized":"2024-03-29T09:11:44.996277+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:01.878804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496947+0000","last_change":"2024-03-29T09:11:43.676496+0000","last_active":"2024-03-29T09:11:45.496947+0000","last_peered":"2024-03-29T09:11:45.496947+0000","last_clean":"2024-03-29T09:11:45.496947+0000","last_became_active":"2024-03-29T09:11:43.676208+0000","last_became_peered":"2024-03-29T09:11:43.676208+0000","last_unstale":"2024-03-29T09:11:45.496947+0000","last_undegraded":"2024-03-29T09:11:45.496947+0000","last_fullsized":"2024-03-29T09:11:45.496947+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:29:16.327235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497050+0000","last_change":"2024-03-29T09:11:43.173125+0000","last_active":"2024-03-29T09:11:45.497050+0000","last_peered":"2024-03-29T09:11:45.497050+0000","last_clean":"2024-03-29T09:11:45.497050+0000","last_became_active":"2024-03-29T09:11:43.172897+0000","last_became_peered":"2024-03-29T09:11:43.172897+0000","last_unstale":"2024-03-29T09:11:45.497050+0000","last_undegraded":"2024-03-29T09:11:45.497050+0000","last_fullsized":"2024-03-29T09:11:45.497050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:51.992855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996071+0000","last_change":"2024-03-29T09:11:43.681720+0000","last_active":"2024-03-29T09:11:44.996071+0000","last_peered":"2024-03-29T09:11:44.996071+0000","last_clean":"2024-03-29T09:11:44.996071+0000","last_became_active":"2024-03-29T09:11:43.681366+0000","last_became_peered":"2024-03-29T09:11:43.681366+0000","last_unstale":"2024-03-29T09:11:44.996071+0000","last_undegraded":"2024-03-29T09:11:44.996071+0000","last_fullsized":"2024-03-29T09:11:44.996071+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:34.234919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667404+0000","last_change":"2024-03-29T09:11:43.176057+0000","last_active":"2024-03-29T09:11:44.667404+0000","last_peered":"2024-03-29T09:11:44.667404+0000","last_clean":"2024-03-29T09:11:44.667404+0000","last_became_active":"2024-03-29T09:11:43.175783+0000","last_became_peered":"2024-03-29T09:11:43.175783+0000","last_unstale":"2024-03-29T09:11:44.667404+0000","last_undegraded":"2024-03-29T09:11:44.667404+0000","last_fullsized":"2024-03-29T09:11:44.667404+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:31.151934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667835+0000","last_change":"2024-03-29T09:11:43.681530+0000","last_active":"2024-03-29T09:11:44.667835+0000","last_peered":"2024-03-29T09:11:44.667835+0000","last_clean":"2024-03-29T09:11:44.667835+0000","last_became_active":"2024-03-29T09:11:43.679577+0000","last_became_peered":"2024-03-29T09:11:43.679577+0000","last_unstale":"2024-03-29T09:11:44.667835+0000","last_undegraded":"2024-03-29T09:11:44.667835+0000","last_fullsized":"2024-03-29T09:11:44.667835+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:32.530881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497211+0000","last_change":"2024-03-29T09:11:43.165270+0000","last_active":"2024-03-29T09:11:45.497211+0000","last_peered":"2024-03-29T09:11:45.497211+0000","last_clean":"2024-03-29T09:11:45.497211+0000","last_became_active":"2024-03-29T09:11:43.164467+0000","last_became_peered":"2024-03-29T09:11:43.164467+0000","last_unstale":"2024-03-29T09:11:45.497211+0000","last_undegraded":"2024-03-29T09:11:45.497211+0000","last_fullsized":"2024-03-29T09:11:45.497211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:29.110475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497883+0000","last_change":"2024-03-29T09:11:43.666694+0000","last_active":"2024-03-29T09:11:45.497883+0000","last_peered":"2024-03-29T09:11:45.497883+0000","last_clean":"2024-03-29T09:11:45.497883+0000","last_became_active":"2024-03-29T09:11:43.666396+0000","last_became_peered":"2024-03-29T09:11:43.666396+0000","last_unstale":"2024-03-29T09:11:45.497883+0000","last_undegraded":"2024-03-29T09:11:45.497883+0000","last_fullsized":"2024-03-29T09:11:45.497883+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:51:20.115771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497219+0000","last_change":"2024-03-29T09:11:43.173205+0000","last_active":"2024-03-29T09:11:45.497219+0000","last_peered":"2024-03-29T09:11:45.497219+0000","last_clean":"2024-03-29T09:11:45.497219+0000","last_became_active":"2024-03-29T09:11:43.172920+0000","last_became_peered":"2024-03-29T09:11:43.172920+0000","last_unstale":"2024-03-29T09:11:45.497219+0000","last_undegraded":"2024-03-29T09:11:45.497219+0000","last_fullsized":"2024-03-29T09:11:45.497219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:10:33.491702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497202+0000","last_change":"2024-03-29T09:11:43.685373+0000","last_active":"2024-03-29T09:11:45.497202+0000","last_peered":"2024-03-29T09:11:45.497202+0000","last_clean":"2024-03-29T09:11:45.497202+0000","last_became_active":"2024-03-29T09:11:43.685092+0000","last_became_peered":"2024-03-29T09:11:43.685092+0000","last_unstale":"2024-03-29T09:11:45.497202+0000","last_undegraded":"2024-03-29T09:11:45.497202+0000","last_fullsized":"2024-03-29T09:11:45.497202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"54'3","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498123+0000","last_change":"2024-03-29T09:11:43.181613+0000","last_active":"2024-03-29T09:11:45.498123+0000","last_peered":"2024-03-29T09:11:45.498123+0000","last_clean":"2024-03-29T09:11:45.498123+0000","last_became_active":"2024-03-29T09:11:43.181326+0000","last_became_peered":"2024-03-29T09:11:43.181326+0000","last_unstale":"2024-03-29T09:11:45.498123+0000","last_undegraded":"2024-03-29T09:11:45.498123+0000","last_fullsized":"2024-03-29T09:11:45.498123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:12.031715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497714+0000","last_change":"2024-03-29T09:11:43.675831+0000","last_active":"2024-03-29T09:11:45.497714+0000","last_peered":"2024-03-29T09:11:45.497714+0000","last_clean":"2024-03-29T09:11:45.497714+0000","last_became_active":"2024-03-29T09:11:43.675418+0000","last_became_peered":"2024-03-29T09:11:43.675418+0000","last_unstale":"2024-03-29T09:11:45.497714+0000","last_undegraded":"2024-03-29T09:11:45.497714+0000","last_fullsized":"2024-03-29T09:11:45.497714+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:06:28.079801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"54'6","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667829+0000","last_change":"2024-03-29T09:11:43.181637+0000","last_active":"2024-03-29T09:11:44.667829+0000","last_peered":"2024-03-29T09:11:44.667829+0000","last_clean":"2024-03-29T09:11:44.667829+0000","last_became_active":"2024-03-29T09:11:43.181327+0000","last_became_peered":"2024-03-29T09:11:43.181327+0000","last_unstale":"2024-03-29T09:11:44.667829+0000","last_undegraded":"2024-03-29T09:11:44.667829+0000","last_fullsized":"2024-03-29T09:11:44.667829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:22:34.000096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497070+0000","last_change":"2024-03-29T09:11:43.677020+0000","last_active":"2024-03-29T09:11:45.497070+0000","last_peered":"2024-03-29T09:11:45.497070+0000","last_clean":"2024-03-29T09:11:45.497070+0000","last_became_active":"2024-03-29T09:11:43.676671+0000","last_became_peered":"2024-03-29T09:11:43.676671+0000","last_unstale":"2024-03-29T09:11:45.497070+0000","last_undegraded":"2024-03-29T09:11:45.497070+0000","last_fullsized":"2024-03-29T09:11:45.497070+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:00:54.297415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497048+0000","last_change":"2024-03-29T09:11:43.176006+0000","last_active":"2024-03-29T09:11:45.497048+0000","last_peered":"2024-03-29T09:11:45.497048+0000","last_clean":"2024-03-29T09:11:45.497048+0000","last_became_active":"2024-03-29T09:11:43.175716+0000","last_became_peered":"2024-03-29T09:11:43.175716+0000","last_unstale":"2024-03-29T09:11:45.497048+0000","last_undegraded":"2024-03-29T09:11:45.497048+0000","last_fullsized":"2024-03-29T09:11:45.497048+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:47.033199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497524+0000","last_change":"2024-03-29T09:11:43.673170+0000","last_active":"2024-03-29T09:11:45.497524+0000","last_peered":"2024-03-29T09:11:45.497524+0000","last_clean":"2024-03-29T09:11:45.497524+0000","last_became_active":"2024-03-29T09:11:43.672886+0000","last_became_peered":"2024-03-29T09:11:43.672886+0000","last_unstale":"2024-03-29T09:11:45.497524+0000","last_undegraded":"2024-03-29T09:11:45.497524+0000","last_fullsized":"2024-03-29T09:11:45.497524+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:16.721779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498004+0000","last_change":"2024-03-29T09:11:43.177733+0000","last_active":"2024-03-29T09:11:45.498004+0000","last_peered":"2024-03-29T09:11:45.498004+0000","last_clean":"2024-03-29T09:11:45.498004+0000","last_became_active":"2024-03-29T09:11:43.177447+0000","last_became_peered":"2024-03-29T09:11:43.177447+0000","last_unstale":"2024-03-29T09:11:45.498004+0000","last_undegraded":"2024-03-29T09:11:45.498004+0000","last_fullsized":"2024-03-29T09:11:45.498004+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:31:51.585476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496889+0000","last_change":"2024-03-29T09:11:43.681087+0000","last_active":"2024-03-29T09:11:45.496889+0000","last_peered":"2024-03-29T09:11:45.496889+0000","last_clean":"2024-03-29T09:11:45.496889+0000","last_became_active":"2024-03-29T09:11:43.680814+0000","last_became_peered":"2024-03-29T09:11:43.680814+0000","last_unstale":"2024-03-29T09:11:45.496889+0000","last_undegraded":"2024-03-29T09:11:45.496889+0000","last_fullsized":"2024-03-29T09:11:45.496889+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:37.448155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498207+0000","last_change":"2024-03-29T09:11:43.170448+0000","last_active":"2024-03-29T09:11:45.498207+0000","last_peered":"2024-03-29T09:11:45.498207+0000","last_clean":"2024-03-29T09:11:45.498207+0000","last_became_active":"2024-03-29T09:11:43.170170+0000","last_became_peered":"2024-03-29T09:11:43.170170+0000","last_unstale":"2024-03-29T09:11:45.498207+0000","last_undegraded":"2024-03-29T09:11:45.498207+0000","last_fullsized":"2024-03-29T09:11:45.498207+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:49:21.777400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498121+0000","last_change":"2024-03-29T09:11:43.679100+0000","last_active":"2024-03-29T09:11:45.498121+0000","last_peered":"2024-03-29T09:11:45.498121+0000","last_clean":"2024-03-29T09:11:45.498121+0000","last_became_active":"2024-03-29T09:11:43.678319+0000","last_became_peered":"2024-03-29T09:11:43.678319+0000","last_unstale":"2024-03-29T09:11:45.498121+0000","last_undegraded":"2024-03-29T09:11:45.498121+0000","last_fullsized":"2024-03-29T09:11:45.498121+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:58:28.504247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"54'8","reported_seq":19,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497725+0000","last_change":"2024-03-29T09:11:43.173897+0000","last_active":"2024-03-29T09:11:45.497725+0000","last_peered":"2024-03-29T09:11:45.497725+0000","last_clean":"2024-03-29T09:11:45.497725+0000","last_became_active":"2024-03-29T09:11:43.172759+0000","last_became_peered":"2024-03-29T09:11:43.172759+0000","last_unstale":"2024-03-29T09:11:45.497725+0000","last_undegraded":"2024-03-29T09:11:45.497725+0000","last_fullsized":"2024-03-29T09:11:45.497725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:57.415296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497652+0000","last_change":"2024-03-29T09:11:43.681216+0000","last_active":"2024-03-29T09:11:45.497652+0000","last_peered":"2024-03-29T09:11:45.497652+0000","last_clean":"2024-03-29T09:11:45.497652+0000","last_became_active":"2024-03-29T09:11:43.680966+0000","last_became_peered":"2024-03-29T09:11:43.680966+0000","last_unstale":"2024-03-29T09:11:45.497652+0000","last_undegraded":"2024-03-29T09:11:45.497652+0000","last_fullsized":"2024-03-29T09:11:45.497652+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:06.103995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667467+0000","last_change":"2024-03-29T09:11:43.183881+0000","last_active":"2024-03-29T09:11:44.667467+0000","last_peered":"2024-03-29T09:11:44.667467+0000","last_clean":"2024-03-29T09:11:44.667467+0000","last_became_active":"2024-03-29T09:11:43.183567+0000","last_became_peered":"2024-03-29T09:11:43.183567+0000","last_unstale":"2024-03-29T09:11:44.667467+0000","last_undegraded":"2024-03-29T09:11:44.667467+0000","last_fullsized":"2024-03-29T09:11:44.667467+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:44.603438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.498039+0000","last_change":"2024-03-29T09:11:43.665968+0000","last_active":"2024-03-29T09:11:45.498039+0000","last_peered":"2024-03-29T09:11:45.498039+0000","last_clean":"2024-03-29T09:11:45.498039+0000","last_became_active":"2024-03-29T09:11:43.665714+0000","last_became_peered":"2024-03-29T09:11:43.665714+0000","last_unstale":"2024-03-29T09:11:45.498039+0000","last_undegraded":"2024-03-29T09:11:45.498039+0000","last_fullsized":"2024-03-29T09:11:45.498039+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:21.218363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497372+0000","last_change":"2024-03-29T09:11:43.173582+0000","last_active":"2024-03-29T09:11:45.497372+0000","last_peered":"2024-03-29T09:11:45.497372+0000","last_clean":"2024-03-29T09:11:45.497372+0000","last_became_active":"2024-03-29T09:11:43.173300+0000","last_became_peered":"2024-03-29T09:11:43.173300+0000","last_unstale":"2024-03-29T09:11:45.497372+0000","last_undegraded":"2024-03-29T09:11:45.497372+0000","last_fullsized":"2024-03-29T09:11:45.497372+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:50.765164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.666945+0000","last_change":"2024-03-29T09:11:43.685214+0000","last_active":"2024-03-29T09:11:44.666945+0000","last_peered":"2024-03-29T09:11:44.666945+0000","last_clean":"2024-03-29T09:11:44.666945+0000","last_became_active":"2024-03-29T09:11:43.684951+0000","last_became_peered":"2024-03-29T09:11:43.684951+0000","last_unstale":"2024-03-29T09:11:44.666945+0000","last_undegraded":"2024-03-29T09:11:44.666945+0000","last_fullsized":"2024-03-29T09:11:44.666945+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:21.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"54'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497581+0000","last_change":"2024-03-29T09:11:43.171045+0000","last_active":"2024-03-29T09:11:45.497581+0000","last_peered":"2024-03-29T09:11:45.497581+0000","last_clean":"2024-03-29T09:11:45.497581+0000","last_became_active":"2024-03-29T09:11:43.170833+0000","last_became_peered":"2024-03-29T09:11:43.170833+0000","last_unstale":"2024-03-29T09:11:45.497581+0000","last_undegraded":"2024-03-29T09:11:45.497581+0000","last_fullsized":"2024-03-29T09:11:45.497581+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:28.792665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497822+0000","last_change":"2024-03-29T09:11:43.682084+0000","last_active":"2024-03-29T09:11:45.497822+0000","last_peered":"2024-03-29T09:11:45.497822+0000","last_clean":"2024-03-29T09:11:45.497822+0000","last_became_active":"2024-03-29T09:11:43.681789+0000","last_became_peered":"2024-03-29T09:11:43.681789+0000","last_unstale":"2024-03-29T09:11:45.497822+0000","last_undegraded":"2024-03-29T09:11:45.497822+0000","last_fullsized":"2024-03-29T09:11:45.497822+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:41:59.385726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497200+0000","last_change":"2024-03-29T09:11:43.179870+0000","last_active":"2024-03-29T09:11:45.497200+0000","last_peered":"2024-03-29T09:11:45.497200+0000","last_clean":"2024-03-29T09:11:45.497200+0000","last_became_active":"2024-03-29T09:11:43.179590+0000","last_became_peered":"2024-03-29T09:11:43.179590+0000","last_unstale":"2024-03-29T09:11:45.497200+0000","last_undegraded":"2024-03-29T09:11:45.497200+0000","last_fullsized":"2024-03-29T09:11:45.497200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:13:28.738464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497186+0000","last_change":"2024-03-29T09:11:43.684574+0000","last_active":"2024-03-29T09:11:45.497186+0000","last_peered":"2024-03-29T09:11:45.497186+0000","last_clean":"2024-03-29T09:11:45.497186+0000","last_became_active":"2024-03-29T09:11:43.684273+0000","last_became_peered":"2024-03-29T09:11:43.684273+0000","last_unstale":"2024-03-29T09:11:45.497186+0000","last_undegraded":"2024-03-29T09:11:45.497186+0000","last_fullsized":"2024-03-29T09:11:45.497186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:21.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.499015+0000","last_change":"2024-03-29T09:11:43.177261+0000","last_active":"2024-03-29T09:11:45.499015+0000","last_peered":"2024-03-29T09:11:45.499015+0000","last_clean":"2024-03-29T09:11:45.499015+0000","last_became_active":"2024-03-29T09:11:43.176994+0000","last_became_peered":"2024-03-29T09:11:43.176994+0000","last_unstale":"2024-03-29T09:11:45.499015+0000","last_undegraded":"2024-03-29T09:11:45.499015+0000","last_fullsized":"2024-03-29T09:11:45.499015+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:33.227274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497092+0000","last_change":"2024-03-29T09:11:43.677761+0000","last_active":"2024-03-29T09:11:45.497092+0000","last_peered":"2024-03-29T09:11:45.497092+0000","last_clean":"2024-03-29T09:11:45.497092+0000","last_became_active":"2024-03-29T09:11:43.677445+0000","last_became_peered":"2024-03-29T09:11:43.677445+0000","last_unstale":"2024-03-29T09:11:45.497092+0000","last_undegraded":"2024-03-29T09:11:45.497092+0000","last_fullsized":"2024-03-29T09:11:45.497092+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:34.234919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496977+0000","last_change":"2024-03-29T09:11:43.174858+0000","last_active":"2024-03-29T09:11:45.496977+0000","last_peered":"2024-03-29T09:11:45.496977+0000","last_clean":"2024-03-29T09:11:45.496977+0000","last_became_active":"2024-03-29T09:11:43.174559+0000","last_became_peered":"2024-03-29T09:11:43.174559+0000","last_unstale":"2024-03-29T09:11:45.496977+0000","last_undegraded":"2024-03-29T09:11:45.496977+0000","last_fullsized":"2024-03-29T09:11:45.496977+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:01.878804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496861+0000","last_change":"2024-03-29T09:11:43.677801+0000","last_active":"2024-03-29T09:11:45.496861+0000","last_peered":"2024-03-29T09:11:45.496861+0000","last_clean":"2024-03-29T09:11:45.496861+0000","last_became_active":"2024-03-29T09:11:43.677410+0000","last_became_peered":"2024-03-29T09:11:43.677410+0000","last_unstale":"2024-03-29T09:11:45.496861+0000","last_undegraded":"2024-03-29T09:11:45.496861+0000","last_fullsized":"2024-03-29T09:11:45.496861+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:28.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497264+0000","last_change":"2024-03-29T09:11:43.178139+0000","last_active":"2024-03-29T09:11:45.497264+0000","last_peered":"2024-03-29T09:11:45.497264+0000","last_clean":"2024-03-29T09:11:45.497264+0000","last_became_active":"2024-03-29T09:11:43.176620+0000","last_became_peered":"2024-03-29T09:11:43.176620+0000","last_unstale":"2024-03-29T09:11:45.497264+0000","last_undegraded":"2024-03-29T09:11:45.497264+0000","last_fullsized":"2024-03-29T09:11:45.497264+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:44.603438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497741+0000","last_change":"2024-03-29T09:11:43.666170+0000","last_active":"2024-03-29T09:11:45.497741+0000","last_peered":"2024-03-29T09:11:45.497741+0000","last_clean":"2024-03-29T09:11:45.497741+0000","last_became_active":"2024-03-29T09:11:43.665889+0000","last_became_peered":"2024-03-29T09:11:43.665889+0000","last_unstale":"2024-03-29T09:11:45.497741+0000","last_undegraded":"2024-03-29T09:11:45.497741+0000","last_fullsized":"2024-03-29T09:11:45.497741+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:32:49.077000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"54'2","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667507+0000","last_change":"2024-03-29T09:11:43.176436+0000","last_active":"2024-03-29T09:11:44.667507+0000","last_peered":"2024-03-29T09:11:44.667507+0000","last_clean":"2024-03-29T09:11:44.667507+0000","last_became_active":"2024-03-29T09:11:43.176164+0000","last_became_peered":"2024-03-29T09:11:43.176164+0000","last_unstale":"2024-03-29T09:11:44.667507+0000","last_undegraded":"2024-03-29T09:11:44.667507+0000","last_fullsized":"2024-03-29T09:11:44.667507+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:54:51.708243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497972+0000","last_change":"2024-03-29T09:11:43.677579+0000","last_active":"2024-03-29T09:11:45.497972+0000","last_peered":"2024-03-29T09:11:45.497972+0000","last_clean":"2024-03-29T09:11:45.497972+0000","last_became_active":"2024-03-29T09:11:43.676104+0000","last_became_peered":"2024-03-29T09:11:43.676104+0000","last_unstale":"2024-03-29T09:11:45.497972+0000","last_undegraded":"2024-03-29T09:11:45.497972+0000","last_fullsized":"2024-03-29T09:11:45.497972+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:26.946110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"55'1","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:46.546869+0000","last_change":"2024-03-29T09:11:43.182966+0000","last_active":"2024-03-29T09:11:46.546869+0000","last_peered":"2024-03-29T09:11:46.546869+0000","last_clean":"2024-03-29T09:11:46.546869+0000","last_became_active":"2024-03-29T09:11:43.182644+0000","last_became_peered":"2024-03-29T09:11:43.182644+0000","last_unstale":"2024-03-29T09:11:46.546869+0000","last_undegraded":"2024-03-29T09:11:46.546869+0000","last_fullsized":"2024-03-29T09:11:46.546869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:36.913030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667050+0000","last_change":"2024-03-29T09:11:43.676032+0000","last_active":"2024-03-29T09:11:44.667050+0000","last_peered":"2024-03-29T09:11:44.667050+0000","last_clean":"2024-03-29T09:11:44.667050+0000","last_became_active":"2024-03-29T09:11:43.675680+0000","last_became_peered":"2024-03-29T09:11:43.675680+0000","last_unstale":"2024-03-29T09:11:44.667050+0000","last_undegraded":"2024-03-29T09:11:44.667050+0000","last_fullsized":"2024-03-29T09:11:44.667050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:04:14.657461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497378+0000","last_change":"2024-03-29T09:11:43.171894+0000","last_active":"2024-03-29T09:11:45.497378+0000","last_peered":"2024-03-29T09:11:45.497378+0000","last_clean":"2024-03-29T09:11:45.497378+0000","last_became_active":"2024-03-29T09:11:43.171636+0000","last_became_peered":"2024-03-29T09:11:43.171636+0000","last_unstale":"2024-03-29T09:11:45.497378+0000","last_undegraded":"2024-03-29T09:11:45.497378+0000","last_fullsized":"2024-03-29T09:11:45.497378+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:27.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497284+0000","last_change":"2024-03-29T09:11:43.678609+0000","last_active":"2024-03-29T09:11:45.497284+0000","last_peered":"2024-03-29T09:11:45.497284+0000","last_clean":"2024-03-29T09:11:45.497284+0000","last_became_active":"2024-03-29T09:11:43.678256+0000","last_became_peered":"2024-03-29T09:11:43.678256+0000","last_unstale":"2024-03-29T09:11:45.497284+0000","last_undegraded":"2024-03-29T09:11:45.497284+0000","last_fullsized":"2024-03-29T09:11:45.497284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:23.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497444+0000","last_change":"2024-03-29T09:11:43.175598+0000","last_active":"2024-03-29T09:11:45.497444+0000","last_peered":"2024-03-29T09:11:45.497444+0000","last_clean":"2024-03-29T09:11:45.497444+0000","last_became_active":"2024-03-29T09:11:43.175354+0000","last_became_peered":"2024-03-29T09:11:43.175354+0000","last_unstale":"2024-03-29T09:11:45.497444+0000","last_undegraded":"2024-03-29T09:11:45.497444+0000","last_fullsized":"2024-03-29T09:11:45.497444+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:23:19.175144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996580+0000","last_change":"2024-03-29T09:11:43.691405+0000","last_active":"2024-03-29T09:11:44.996580+0000","last_peered":"2024-03-29T09:11:44.996580+0000","last_clean":"2024-03-29T09:11:44.996580+0000","last_became_active":"2024-03-29T09:11:43.691072+0000","last_became_peered":"2024-03-29T09:11:43.691072+0000","last_unstale":"2024-03-29T09:11:44.996580+0000","last_undegraded":"2024-03-29T09:11:44.996580+0000","last_fullsized":"2024-03-29T09:11:44.996580+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:32:46.071316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667321+0000","last_change":"2024-03-29T09:11:43.179767+0000","last_active":"2024-03-29T09:11:44.667321+0000","last_peered":"2024-03-29T09:11:44.667321+0000","last_clean":"2024-03-29T09:11:44.667321+0000","last_became_active":"2024-03-29T09:11:43.179487+0000","last_became_peered":"2024-03-29T09:11:43.179487+0000","last_unstale":"2024-03-29T09:11:44.667321+0000","last_undegraded":"2024-03-29T09:11:44.667321+0000","last_fullsized":"2024-03-29T09:11:44.667321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:27.773396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667726+0000","last_change":"2024-03-29T09:11:43.687900+0000","last_active":"2024-03-29T09:11:44.667726+0000","last_peered":"2024-03-29T09:11:44.667726+0000","last_clean":"2024-03-29T09:11:44.667726+0000","last_became_active":"2024-03-29T09:11:43.687446+0000","last_became_peered":"2024-03-29T09:11:43.687446+0000","last_unstale":"2024-03-29T09:11:44.667726+0000","last_undegraded":"2024-03-29T09:11:44.667726+0000","last_fullsized":"2024-03-29T09:11:44.667726+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:51.772809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497118+0000","last_change":"2024-03-29T09:11:43.175422+0000","last_active":"2024-03-29T09:11:45.497118+0000","last_peered":"2024-03-29T09:11:45.497118+0000","last_clean":"2024-03-29T09:11:45.497118+0000","last_became_active":"2024-03-29T09:11:43.175145+0000","last_became_peered":"2024-03-29T09:11:43.175145+0000","last_unstale":"2024-03-29T09:11:45.497118+0000","last_undegraded":"2024-03-29T09:11:45.497118+0000","last_fullsized":"2024-03-29T09:11:45.497118+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:04.086596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496735+0000","last_change":"2024-03-29T09:11:43.683642+0000","last_active":"2024-03-29T09:11:45.496735+0000","last_peered":"2024-03-29T09:11:45.496735+0000","last_clean":"2024-03-29T09:11:45.496735+0000","last_became_active":"2024-03-29T09:11:43.680296+0000","last_became_peered":"2024-03-29T09:11:43.680296+0000","last_unstale":"2024-03-29T09:11:45.496735+0000","last_undegraded":"2024-03-29T09:11:45.496735+0000","last_fullsized":"2024-03-29T09:11:45.496735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:06.103995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.996134+0000","last_change":"2024-03-29T09:11:43.177125+0000","last_active":"2024-03-29T09:11:44.996134+0000","last_peered":"2024-03-29T09:11:44.996134+0000","last_clean":"2024-03-29T09:11:44.996134+0000","last_became_active":"2024-03-29T09:11:43.176802+0000","last_became_peered":"2024-03-29T09:11:43.176802+0000","last_unstale":"2024-03-29T09:11:44.996134+0000","last_undegraded":"2024-03-29T09:11:44.996134+0000","last_fullsized":"2024-03-29T09:11:44.996134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:42:31.523236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667170+0000","last_change":"2024-03-29T09:11:43.675090+0000","last_active":"2024-03-29T09:11:44.667170+0000","last_peered":"2024-03-29T09:11:44.667170+0000","last_clean":"2024-03-29T09:11:44.667170+0000","last_became_active":"2024-03-29T09:11:43.674631+0000","last_became_peered":"2024-03-29T09:11:43.674631+0000","last_unstale":"2024-03-29T09:11:44.667170+0000","last_undegraded":"2024-03-29T09:11:44.667170+0000","last_fullsized":"2024-03-29T09:11:44.667170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:00:54.297415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.496892+0000","last_change":"2024-03-29T09:11:43.174863+0000","last_active":"2024-03-29T09:11:45.496892+0000","last_peered":"2024-03-29T09:11:45.496892+0000","last_clean":"2024-03-29T09:11:45.496892+0000","last_became_active":"2024-03-29T09:11:43.174568+0000","last_became_peered":"2024-03-29T09:11:43.174568+0000","last_unstale":"2024-03-29T09:11:45.496892+0000","last_undegraded":"2024-03-29T09:11:45.496892+0000","last_fullsized":"2024-03-29T09:11:45.496892+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:48:29.840684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:45.497110+0000","last_change":"2024-03-29T09:11:43.667721+0000","last_active":"2024-03-29T09:11:45.497110+0000","last_peered":"2024-03-29T09:11:45.497110+0000","last_clean":"2024-03-29T09:11:45.497110+0000","last_became_active":"2024-03-29T09:11:43.667442+0000","last_became_peered":"2024-03-29T09:11:43.667442+0000","last_unstale":"2024-03-29T09:11:45.497110+0000","last_undegraded":"2024-03-29T09:11:45.497110+0000","last_fullsized":"2024-03-29T09:11:45.497110+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:43.149736+0000","last_clean_scrub_stamp":"2024-03-29T09:11:43.149736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:37.929321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-03-29T09:11:44.667517+0000","last_change":"2024-03-29T09:11:43.181790+0000","last_active":"2024-03-29T09:11:44.667517+0000","last_peered":"2024-03-29T09:11:44.667517+0000","last_clean":"2024-03-29T09:11:44.667517+0000","last_became_active":"2024-03-29T09:11:43.181311+0000","last_became_peered":"2024-03-29T09:11:43.181311+0000","last_unstale":"2024-03-29T09:11:44.667517+0000","last_undegraded":"2024-03-29T09:11:44.667517+0000","last_fullsized":"2024-03-29T09:11:44.667517+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:11:42.142091+0000","last_clean_scrub_stamp":"2024-03-29T09:11:42.142091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:06:19.992268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":196,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-03-29T09:11:43.663437+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:11:43.663437+0000","last_peered":"2024-03-29T09:11:43.663437+0000","last_clean":"2024-03-29T09:11:43.663437+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:11:43.663437+0000","last_undegraded":"2024-03-29T09:11:43.663437+0000","last_fullsized":"2024-03-29T09:11:43.663437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":293965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.71399999999999997},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":1.639,"15min":1.639},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":0.86399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77600000000000002,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.71699999999999997,"15min":0.76400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72499999999999998,"5min":1.992,"15min":1.992},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.75900000000000001,"15min":0.81000000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.81899999999999995,"15min":0.86899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85099999999999998,"5min":2.883,"15min":2.883},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.87,"5min":1.387,"15min":1.387},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68500000000000005,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.623},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89200000000000002,"5min":1.367,"15min":1.367},"last":0.89200000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.458,"15min":1.458},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.88700000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.92200000000000004,"5min":1.413,"15min":1.413},"last":0.92200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":293895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.67100000000000004},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.88800000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56399999999999995}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.79500000000000004,"15min":0.82399999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.85799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.81200000000000006,"15min":0.83799999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.96099999999999997,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.75600000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66500000000000004,"15min":0.66600000000000004},"min":{"1min":0.45300000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86899999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.48999999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.871,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.45800000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.74299999999999999,"15min":0.74199999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.79500000000000004,"15min":0.80200000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.006,"5min":1.026,"15min":1.026},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.72099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":884823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.60399999999999998,"15min":0.63300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.175,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.373,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":2.5259999999999998,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.64900000000000002,"15min":0.68300000000000005},"min":{"1min":0.34499999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.57199999999999995,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.372}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.40400000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.57299999999999995,"15min":0.59299999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.66700000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.008,"15min":0.94899999999999995},"min":{"1min":0.5,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":12.356,"5min":12.356,"15min":12.356},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.73899999999999999,"15min":0.71499999999999997},"min":{"1min":0.48499999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.51400000000000001,"5min":0.438,"15min":0.438},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.018},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.91700000000000004,"15min":0.93799999999999994},"min":{"1min":0.56399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.6279999999999999,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":1.034}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83399999999999996,"15min":0.84299999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.903,"5min":1.99,"15min":1.99},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.0609999999999999,"15min":0.99399999999999999},"min":{"1min":0.52700000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":12.669,"5min":12.669,"15min":12.669},"last":0.99199999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.874,"15min":0.88},"min":{"1min":0.57499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.97499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":295437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.51300000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.47599999999999998,"15min":0.48499999999999999},"min":{"1min":0.34399999999999997,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.51700000000000002,"5min":0.751,"15min":0.751},"last":0.32200000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.78300000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.375}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.57099999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.499},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.67800000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.95899999999999996,"5min":1.028,"15min":1.028},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72499999999999998,"15min":0.73299999999999998},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.99099999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.069,"15min":1.069},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.78900000000000003,"15min":0.80200000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.02,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.72799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":295597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.47399999999999998,"15min":0.48799999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.56899999999999995,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.51400000000000001,"15min":0.53000000000000003},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.54400000000000004,"5min":1.252,"15min":1.252},"last":0.38100000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.54900000000000004,"15min":0.56599999999999995},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59599999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.61899999999999999,"15min":0.64100000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.66900000000000004,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.746,"5min":2.415,"15min":2.415},"last":0.504},{"interface":"front","average":{"1min":0.5,"5min":0.66100000000000003,"15min":0.68799999999999994},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70799999999999996,"5min":2.056,"15min":2.056},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.78500000000000003,"15min":0.80500000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.91400000000000003,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.80700000000000005,"15min":0.82699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":2.073,"15min":2.073},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.92500000000000004,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95199999999999996,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.878,"15min":0.90300000000000002},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96799999999999997,"5min":2.387,"15min":2.387},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.91700000000000004,"15min":0.94699999999999995},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.91500000000000004,"5min":2.996,"15min":2.996},"last":0.90200000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:10:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.872},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95499999999999996,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.84999999999999998,"15min":0.874},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88800000000000001,"5min":2.165,"15min":2.165},"last":0.871}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":293963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.53200000000000003,"15min":0.55100000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60599999999999998,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.54300000000000004,"15min":0.55700000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.61599999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.44}]},{"osd":1,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.496,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.53900000000000003,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.63900000000000001,"15min":0.66300000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76300000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.99299999999999999,"5min":2.278,"15min":2.278},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.88,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.0129999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96399999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.59999999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.88400000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81499999999999995,"15min":0.82499999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1619999999999999,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.878,"15min":0.90000000000000002},"min":{"1min":0.61599999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.90000000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Mar 29 09:10:50 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.88400000000000001,"15min":0.90800000000000003},"min":{"1min":0.53700000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88900000000000001,"5min":2.649,"15min":2.649},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.60199999999999998,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.034,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.65400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":667648,"data_stored":294489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1319999999999999,"5min":1.742,"15min":1.742},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.45300000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.77700000000000002,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.66700000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.69599999999999995,"15min":0.71299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.751,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.73099999999999998,"15min":0.74099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.2130000000000001,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72899999999999998,"15min":0.74099999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84199999999999997,"5min":0.999,"15min":0.999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.002,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.48099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.79200000000000004,"15min":0.81799999999999995},"min":{"1min":0.45300000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.91300000000000003,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.82599999999999996,"15min":0.84599999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88800000000000001,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.79700000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.871,"15min":0.90100000000000002},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0269999999999999,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.94399999999999995,"15min":0.97599999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.073,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.86499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1257472,"data_stored":884275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.39800000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81799999999999995,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85499999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85699999999999998,"5min":0.872,"15min":0.872},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.68300000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77100000000000002,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.66500000000000004}]},{"osd":2,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.83699999999999997,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.77200000000000002,"15min":0.78600000000000003},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.91600000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.81599999999999995,"5min":0.873,"15min":0.873},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73499999999999999,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.81100000000000005,"5min":1.05,"15min":1.05},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68700000000000006,"15min":0.68899999999999995},"min":{"1min":0.47499999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.55300000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.83099999999999996,"5min":1.079,"15min":1.079},"last":0.78100000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:10:49 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.877,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.89400000000000002,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:11:53.432 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T09:11:53.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T09:11:53.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.664+0000 7f32a0000700 1 Processor -- start 2024-03-29T09:11:53.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.664+0000 7f32a0000700 1 -- start start 2024-03-29T09:11:53.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f32a0000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f329815d420 0x7f32980d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f32a0000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f32980d3560 0x7f32980d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f32a0000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32980afdd0 con 0x7f329815d800 2024-03-29T09:11:53.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f32a0000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32980d12b0 con 0x7f329815d420 2024-03-29T09:11:53.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f32a0000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32980d3b00 con 0x7f32980d3560 2024-03-29T09:11:53.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f329d600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f329815d420 0x7f32980d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.665 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f329cc00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f329815d800 0x7f32980d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48252/0 (socket says 172.21.15.99:48252) 2024-03-29T09:11:53.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f329cc00700 1 -- 172.21.15.99:0/2718123647 learned_addr learned my addr 172.21.15.99:0/2718123647 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:53.666 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.665+0000 7f329e000700 1 --2- 172.21.15.99:0/2718123647 >> v2:172.21.15.133:3300/0 conn(0x7f32980d3560 0x7f32980d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f329d600700 1 -- 172.21.15.99:0/2718123647 >> v2:172.21.15.133:3300/0 conn(0x7f32980d3560 msgr2=0x7f32980d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:53.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f329d600700 1 --2- 172.21.15.99:0/2718123647 >> v2:172.21.15.133:3300/0 conn(0x7f32980d3560 0x7f32980d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f329d600700 1 -- 172.21.15.99:0/2718123647 >> v1:172.21.15.99:6789/0 conn(0x7f329815d800 legacy=0x7f32980d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f329d600700 1 -- 172.21.15.99:0/2718123647 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980b0070 con 0x7f329815d420 2024-03-29T09:11:53.667 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f329d600700 1 --2- 172.21.15.99:0/2718123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f32980d1780 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32980cdc00 tx=0x7f328c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f746b8aaab1a649 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f3297e00700 1 -- 172.21.15.99:0/2718123647 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c011830 con 0x7f329815d420 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f3297e00700 1 -- 172.21.15.99:0/2718123647 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328c011990 con 0x7f329815d420 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.666+0000 7f3297e00700 1 -- 172.21.15.99:0/2718123647 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c011d20 con 0x7f329815d420 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 -- 172.21.15.99:0/2718123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 msgr2=0x7f32980d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 --2- 172.21.15.99:0/2718123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f32980d1780 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f32980cdc00 tx=0x7f328c009b20 comp rx=0 tx=0).stop 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 -- 172.21.15.99:0/2718123647 shutdown_connections 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 --2- 172.21.15.99:0/2718123647 >> v2:172.21.15.133:3300/0 conn(0x7f32980d3560 0x7f32980d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 --2- 172.21.15.99:0/2718123647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f32980d1780 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 -- 172.21.15.99:0/2718123647 >> 172.21.15.99:0/2718123647 conn(0x7f329808afd0 msgr2=0x7f329809c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 -- 172.21.15.99:0/2718123647 shutdown_connections 2024-03-29T09:11:53.668 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.667+0000 7f32a0000700 1 -- 172.21.15.99:0/2718123647 wait complete. 2024-03-29T09:11:53.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.669+0000 7f32a0000700 1 Processor -- start 2024-03-29T09:11:53.669 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.669+0000 7f32a0000700 1 -- start start 2024-03-29T09:11:53.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f32a0000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32980d3560 0x7f32980b8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f32a0000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f3298151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f32a0000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32980cfa70 0x7f32980cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f32a0000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32980b8da0 con 0x7f329815d420 2024-03-29T09:11:53.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f32a0000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329815cc70 con 0x7f32980cfa70 2024-03-29T09:11:53.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f32a0000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32980b7950 con 0x7f32980d3560 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f329cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f3298151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f329cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f3298151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57704/0 (socket says 172.21.15.99:57704) 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f329cc00700 1 -- 172.21.15.99:0/1553261884 learned_addr learned my addr 172.21.15.99:0/1553261884 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f329cc00700 1 -- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32980d3560 msgr2=0x7f32980b8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f329cc00700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32980d3560 0x7f32980b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f329e000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32980cfa70 0x7f32980cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.670+0000 7f329cc00700 1 -- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32980cfa70 msgr2=0x7f32980cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f329cc00700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32980cfa70 0x7f32980cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f329cc00700 1 -- 172.21.15.99:0/1553261884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328c0094a0 con 0x7f329815d420 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f329e000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32980cfa70 0x7f32980cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f329cc00700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f3298151720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f328400c480 tx=0x7f328400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f3295600700 1 -- 172.21.15.99:0/1553261884 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284005d70 con 0x7f329815d420 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f3295600700 1 -- 172.21.15.99:0/1553261884 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32840070d0 con 0x7f329815d420 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32981522d0 con 0x7f329815d420 2024-03-29T09:11:53.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f3295600700 1 -- 172.21.15.99:0/1553261884 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284007440 con 0x7f329815d420 2024-03-29T09:11:53.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.671+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329815c990 con 0x7f329815d420 2024-03-29T09:11:53.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.672+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32980678b0 con 0x7f329815d420 2024-03-29T09:11:53.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.673+0000 7f3295600700 1 -- 172.21.15.99:0/1553261884 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f32840075a0 con 0x7f329815d420 2024-03-29T09:11:53.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.674+0000 7f3295600700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f327806ac90 0x7f327806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:53.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.674+0000 7f3295600700 1 -- 172.21.15.99:0/1553261884 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3284088810 con 0x7f329815d420 2024-03-29T09:11:53.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.674+0000 7f329d600700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f327806ac90 0x7f327806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:53.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.675+0000 7f329d600700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f327806ac90 0x7f327806d150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f328c0090b0 tx=0x7f328c010000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:53.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.678+0000 7f3295600700 1 -- 172.21.15.99:0/1553261884 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f32840539f0 con 0x7f329815d420 2024-03-29T09:11:53.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.962+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f329806d1a0 con 0x7f329815d420 2024-03-29T09:11:53.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.966+0000 7f3295600700 1 -- 172.21.15.99:0/1553261884 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f32840170e0 con 0x7f329815d420 2024-03-29T09:11:53.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f327806ac90 msgr2=0x7f327806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f327806ac90 0x7f327806d150 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f328c0090b0 tx=0x7f328c010000 comp rx=0 tx=0).stop 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 msgr2=0x7f3298151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f3298151720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f328400c480 tx=0x7f328400c840 comp rx=0 tx=0).stop 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 shutdown_connections 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f327806ac90 0x7f327806d150 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32980cfa70 0x7f32980cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32980d3560 0x7f32980b8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 --2- 172.21.15.99:0/1553261884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f329815d420 0x7f3298151720 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.968+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 >> 172.21.15.99:0/1553261884 conn(0x7f329808afd0 msgr2=0x7f329806c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.969+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 shutdown_connections 2024-03-29T09:11:53.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:53.969+0000 7f32a0000700 1 -- 172.21.15.99:0/1553261884 wait complete. 2024-03-29T09:11:53.970 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:11:53.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:11:54.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.176+0000 7fdc6c800700 1 Processor -- start 2024-03-29T09:11:54.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.177+0000 7fdc6c800700 1 -- start start 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc6c800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdc64143940 0x7fdc64070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc6c800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fdc640ce2c0 0x7fdc640ce710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc6c800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc640cec50 con 0x7fdc64070ca0 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc6c800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc64143d20 con 0x7fdc640ce2c0 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc6c800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6415eaf0 con 0x7fdc64143940 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc69400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fdc64070ca0 0x7fdc6415db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48262/0 (socket says 172.21.15.99:48262) 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc69400700 1 -- 172.21.15.99:0/3952070761 learned_addr learned my addr 172.21.15.99:0/3952070761 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc6a800700 1 --2- 172.21.15.99:0/3952070761 >> v2:172.21.15.99:3300/0 conn(0x7fdc640ce2c0 0x7fdc640ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:54.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc69e00700 1 --2- 172.21.15.99:0/3952070761 >> v2:172.21.15.133:3300/0 conn(0x7fdc64143940 0x7fdc64070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.178+0000 7fdc68a00700 1 -- 172.21.15.99:0/3952070761 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424147440 0 0) 0x7fdc640cec50 con 0x7fdc64070ca0 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc68a00700 1 -- 172.21.15.99:0/3952070761 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc4c0035b0 con 0x7fdc64070ca0 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc6a800700 1 -- 172.21.15.99:0/3952070761 >> v2:172.21.15.133:3300/0 conn(0x7fdc64143940 msgr2=0x7fdc64070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc6a800700 1 --2- 172.21.15.99:0/3952070761 >> v2:172.21.15.133:3300/0 conn(0x7fdc64143940 0x7fdc64070760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc6a800700 1 -- 172.21.15.99:0/3952070761 >> v1:172.21.15.99:6789/0 conn(0x7fdc64070ca0 legacy=0x7fdc6415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc6a800700 1 -- 172.21.15.99:0/3952070761 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640cd820 con 0x7fdc640ce2c0 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc6a800700 1 --2- 172.21.15.99:0/3952070761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc640ce2c0 0x7fdc640ce710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdc6406c160 tx=0x7fdc6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5043f04df79cb97a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc68a00700 1 -- 172.21.15.99:0/3952070761 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000e070 con 0x7fdc640ce2c0 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc68a00700 1 -- 172.21.15.99:0/3952070761 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000bb20 con 0x7fdc640ce2c0 2024-03-29T09:11:54.180 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.179+0000 7fdc68a00700 1 -- 172.21.15.99:0/3952070761 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60012430 con 0x7fdc640ce2c0 2024-03-29T09:11:54.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 -- 172.21.15.99:0/3952070761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc640ce2c0 msgr2=0x7fdc640ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:54.181 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 --2- 172.21.15.99:0/3952070761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc640ce2c0 0x7fdc640ce710 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdc6406c160 tx=0x7fdc6000b330 comp rx=0 tx=0).stop 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 -- 172.21.15.99:0/3952070761 shutdown_connections 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 --2- 172.21.15.99:0/3952070761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc640ce2c0 0x7fdc640ce710 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 --2- 172.21.15.99:0/3952070761 >> v2:172.21.15.133:3300/0 conn(0x7fdc64143940 0x7fdc64070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 -- 172.21.15.99:0/3952070761 >> 172.21.15.99:0/3952070761 conn(0x7fdc64066540 msgr2=0x7fdc64066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 -- 172.21.15.99:0/3952070761 shutdown_connections 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.180+0000 7fdc6c800700 1 -- 172.21.15.99:0/3952070761 wait complete. 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.182+0000 7fdc6c800700 1 Processor -- start 2024-03-29T09:11:54.182 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.182+0000 7fdc6c800700 1 -- start start 2024-03-29T09:11:54.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6c800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc64070ca0 0x7fdc640ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:54.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6c800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc640ce2c0 0x7fdc640b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:54.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6c800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64143940 0x7fdc64155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:54.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6c800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640b4030 con 0x7fdc64070ca0 2024-03-29T09:11:54.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6c800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640b1d00 con 0x7fdc64143940 2024-03-29T09:11:54.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6c800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640b1fe0 con 0x7fdc640ce2c0 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc64070ca0 0x7fdc640ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc64070ca0 0x7fdc640ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57720/0 (socket says 172.21.15.99:57720) 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 -- 172.21.15.99:0/2152148927 learned_addr learned my addr 172.21.15.99:0/2152148927 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6a800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64143940 0x7fdc64155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69400700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc640ce2c0 0x7fdc640b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 -- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc640ce2c0 msgr2=0x7fdc640b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc640ce2c0 0x7fdc640b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 -- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64143940 msgr2=0x7fdc64155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64143940 0x7fdc64155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc69e00700 1 -- 172.21.15.99:0/2152148927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60009a70 con 0x7fdc64070ca0 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.183+0000 7fdc6a800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64143940 0x7fdc64155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:54.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.184+0000 7fdc69e00700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc64070ca0 0x7fdc640ad840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800e870 tx=0x7fdc5800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:54.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.184+0000 7fdc5e000700 1 -- 172.21.15.99:0/2152148927 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58009590 con 0x7fdc64070ca0 2024-03-29T09:11:54.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.184+0000 7fdc5e000700 1 -- 172.21.15.99:0/2152148927 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5800ee70 con 0x7fdc64070ca0 2024-03-29T09:11:54.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.184+0000 7fdc5e000700 1 -- 172.21.15.99:0/2152148927 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800ca50 con 0x7fdc64070ca0 2024-03-29T09:11:54.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.184+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc640b22c0 con 0x7fdc64070ca0 2024-03-29T09:11:54.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.184+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc64156bd0 con 0x7fdc64070ca0 2024-03-29T09:11:54.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.185+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc640b2910 con 0x7fdc64070ca0 2024-03-29T09:11:54.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.186+0000 7fdc5e000700 1 -- 172.21.15.99:0/2152148927 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fdc5800cbb0 con 0x7fdc64070ca0 2024-03-29T09:11:54.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.187+0000 7fdc5e000700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdc4806ac90 0x7fdc4806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:54.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.187+0000 7fdc5e000700 1 -- 172.21.15.99:0/2152148927 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdc58089340 con 0x7fdc64070ca0 2024-03-29T09:11:54.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.187+0000 7fdc69400700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdc4806ac90 0x7fdc4806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:54.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.188+0000 7fdc69400700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdc4806ac90 0x7fdc4806d150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdc64159b90 tx=0x7fdc54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:54.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.192+0000 7fdc5e000700 1 -- 172.21.15.99:0/2152148927 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fdc58054470 con 0x7fdc64070ca0 2024-03-29T09:11:54.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.438+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc640cd1d0 con 0x7fdc64070ca0 2024-03-29T09:11:54.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.438+0000 7fdc5e000700 1 -- 172.21.15.99:0/2152148927 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fdc5800ce60 con 0x7fdc64070ca0 2024-03-29T09:11:54.439 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:54.439 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/790352630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":790352630},{"type":"v1","addr":"172.21.15.133:6833","nonce":790352630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1265449168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1265449168},{"type":"v1","addr":"172.21.15.133:6835","nonce":1265449168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6102,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1161439746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1161439746},{"type":"v1","addr":"172.21.15.99:6836","nonce":1161439746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:11:43.652592+0000","modified":"2024-03-29T09:11:46.662472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6105":{"gid":6105,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2396189132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2396189132},{"type":"v1","addr":"172.21.15.99:6837","nonce":2396189132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T09:11:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdc4806ac90 msgr2=0x7fdc4806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdc4806ac90 0x7fdc4806d150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdc64159b90 tx=0x7fdc54009040 comp rx=0 tx=0).stop 2024-03-29T09:11:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc64070ca0 msgr2=0x7fdc640ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc64070ca0 0x7fdc640ad840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800e870 tx=0x7fdc5800ec30 comp rx=0 tx=0).stop 2024-03-29T09:11:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 shutdown_connections 2024-03-29T09:11:54.441 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fdc4806ac90 0x7fdc4806d150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc64143940 0x7fdc64155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdc64070ca0 0x7fdc640ad840 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 --2- 172.21.15.99:0/2152148927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc640ce2c0 0x7fdc640b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:54.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 >> 172.21.15.99:0/2152148927 conn(0x7fdc64066540 msgr2=0x7fdc6415a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:54.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 shutdown_connections 2024-03-29T09:11:54.442 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:54.441+0000 7fdc6c800700 1 -- 172.21.15.99:0/2152148927 wait complete. 2024-03-29T09:11:54.442 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-03-29T09:11:54.863 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T09:11:43.652592+0000', 'modified': '2024-03-29T09:11:46.662472+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6105}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6105': {'gid': 6105, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.99:6837/2396189132', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6835', 'nonce': 2396189132}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 2396189132}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T09:11:54.863 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T09:11:54.863 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T09:11:55.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.055+0000 7fc9a2200700 1 Processor -- start 2024-03-29T09:11:55.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.055+0000 7fc9a2200700 1 -- start start 2024-03-29T09:11:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.056+0000 7fc9a2200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 0x7fc99c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.056+0000 7fc9a2200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c15c220 con 0x7fc99c15ae70 2024-03-29T09:11:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.056+0000 7fc9a2200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c15b8f0 con 0x7fc99c0b36f0 2024-03-29T09:11:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.056+0000 7fc9a2200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c15bbd0 con 0x7fc99c0b1260 2024-03-29T09:11:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.056+0000 7fc99b400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fc99c15ae70 0x7fc99c0b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48268/0 (socket says 172.21.15.99:48268) 2024-03-29T09:11:55.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.056+0000 7fc99b400700 1 -- 172.21.15.99:0/1724395362 learned_addr learned my addr 172.21.15.99:0/1724395362 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:55.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 0x7fc99c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4282637502 0 0) 0x7fc99c15c220 con 0x7fc99c15ae70 2024-03-29T09:11:55.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9800035b0 con 0x7fc99c15ae70 2024-03-29T09:11:55.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898665979 0 0) 0x7fc9800035b0 con 0x7fc99c15ae70 2024-03-29T09:11:55.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 msgr2=0x7fc99c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 0x7fc99c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 >> v1:172.21.15.133:6789/0 conn(0x7fc99c0b36f0 legacy=0x7fc99c0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c15beb0 con 0x7fc99c15ae70 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fc98c001670 con 0x7fc99c15ae70 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 >> v1:172.21.15.99:6789/0 conn(0x7fc99c15ae70 legacy=0x7fc99c0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc980003a30 0x7fc980003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 0x7fc980004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc980004d90 0x7fc980005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c15bbd0 con 0x7fc980003a30 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98000bac0 con 0x7fc980004d90 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.057+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98000bc30 con 0x7fc99c0b1260 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99b400700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc980003a30 0x7fc980003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 0x7fc980004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99be00700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc980004d90 0x7fc980005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99b400700 1 -- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 msgr2=0x7fc980004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99b400700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc99c0b1260 0x7fc980004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99b400700 1 -- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc980004d90 msgr2=0x7fc980005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99b400700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc980004d90 0x7fc980005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99b400700 1 -- 172.21.15.99:0/1724395362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9800035b0 con 0x7fc980003a30 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99be00700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc980004d90 0x7fc980005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99b400700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc980003a30 0x7fc980003e10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00c210 tx=0x7fc980005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6686567a61171eac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c007b70 con 0x7fc980003a30 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c00cdb0 con 0x7fc980003a30 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.058+0000 7fc99a000700 1 -- 172.21.15.99:0/1724395362 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c005ba0 con 0x7fc980003a30 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.059+0000 7fc9a2200700 1 -- 172.21.15.99:0/1724395362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc980003a30 msgr2=0x7fc980003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.059+0000 7fc9a2200700 1 --2- 172.21.15.99:0/1724395362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc980003a30 0x7fc980003e10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00c210 tx=0x7fc980005880 comp rx=0 tx=0).stop 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.059+0000 7fc99be00700 1 -- 172.21.15.99:0/1724395362 reap_dead start 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.059+0000 7fc9a2200700 1 -- 172.21.15.99:0/1724395362 shutdown_connections 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.059+0000 7fc9a2200700 1 -- 172.21.15.99:0/1724395362 >> 172.21.15.99:0/1724395362 conn(0x7fc99c066540 msgr2=0x7fc99c066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.059+0000 7fc9a2200700 1 -- 172.21.15.99:0/1724395362 shutdown_connections 2024-03-29T09:11:55.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.059+0000 7fc9a2200700 1 -- 172.21.15.99:0/1724395362 wait complete. 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.061+0000 7fc9a2200700 1 Processor -- start 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.061+0000 7fc9a2200700 1 -- start start 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc9a2200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc980003a30 0x7fc99c14ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc9a2200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c14da80 0x7fc99c1427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc9a2200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c142f50 0x7fc99c0af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc9a2200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c142ce0 con 0x7fc99c14da80 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc9a2200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c153150 con 0x7fc99c142f50 2024-03-29T09:11:55.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc9a2200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c1533a0 con 0x7fc980003a30 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc99aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c14da80 0x7fc99c1427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc99aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c14da80 0x7fc99c1427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57740/0 (socket says 172.21.15.99:57740) 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc99aa00700 1 -- 172.21.15.99:0/1158734363 learned_addr learned my addr 172.21.15.99:0/1158734363 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc99b400700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc980003a30 0x7fc99c14ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.062+0000 7fc99be00700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c142f50 0x7fc99c0af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc99aa00700 1 -- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc980003a30 msgr2=0x7fc99c14ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc980003a30 0x7fc99c14ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc99aa00700 1 -- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c142f50 msgr2=0x7fc99c0af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c142f50 0x7fc99c0af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc99aa00700 1 -- 172.21.15.99:0/1158734363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c153680 con 0x7fc99c14da80 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc99b400700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc980003a30 0x7fc99c14ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c14da80 0x7fc99c1427a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc988011aa0 tx=0x7fc988011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc993400700 1 -- 172.21.15.99:0/1158734363 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800f980 con 0x7fc99c14da80 2024-03-29T09:11:55.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc993400700 1 -- 172.21.15.99:0/1158734363 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98800fae0 con 0x7fc99c14da80 2024-03-29T09:11:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c153960 con 0x7fc99c14da80 2024-03-29T09:11:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc993400700 1 -- 172.21.15.99:0/1158734363 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800fe90 con 0x7fc99c14da80 2024-03-29T09:11:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.063+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c153f20 con 0x7fc99c14da80 2024-03-29T09:11:55.066 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.065+0000 7fc993400700 1 -- 172.21.15.99:0/1158734363 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc98801d400 con 0x7fc99c14da80 2024-03-29T09:11:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.066+0000 7fc993400700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc97c06ac90 0x7fc97c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.066+0000 7fc993400700 1 -- 172.21.15.99:0/1158734363 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc988089b30 con 0x7fc99c14da80 2024-03-29T09:11:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.066+0000 7fc99b400700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc97c06ac90 0x7fc97c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.067 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.067+0000 7fc99b400700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc97c06ac90 0x7fc97c06d150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc98c015040 tx=0x7fc98c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.068+0000 7fc9b5d4db80 1 Processor -- start 2024-03-29T09:11:55.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.068+0000 7fc9b5d4db80 1 -- start start 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ed09a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ed0afc0 0x55b59ece15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ece0980 0x55b59ece1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ec3a940 con 0x55b59ed0afc0 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ec38300 con 0x55b59ece0980 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b59ec33bf0 con 0x55b59ed05c20 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc99aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ed0afc0 0x55b59ece15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc99aa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ed0afc0 0x55b59ece15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57754/0 (socket says 172.21.15.99:57754) 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc99b400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ed09a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc99aa00700 1 -- 172.21.15.99:0/3063712936 learned_addr learned my addr 172.21.15.99:0/3063712936 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:55.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.069+0000 7fc99be00700 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ece0980 0x55b59ece1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc99aa00700 1 -- 172.21.15.99:0/3063712936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 msgr2=0x55b59ed09a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc99aa00700 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ed09a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc99aa00700 1 -- 172.21.15.99:0/3063712936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ece0980 msgr2=0x55b59ece1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc99aa00700 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ece0980 0x55b59ece1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc99aa00700 1 -- 172.21.15.99:0/3063712936 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b59ea61b10 con 0x55b59ed0afc0 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc99aa00700 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ed0afc0 0x55b59ece15f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc988060800 tx=0x7fc9880583d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5a0cec06a85534 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc987400700 1 -- 172.21.15.99:0/3063712936 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98805c770 con 0x55b59ed0afc0 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc987400700 1 -- 172.21.15.99:0/3063712936 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc988060d30 con 0x55b59ed0afc0 2024-03-29T09:11:55.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.070+0000 7fc987400700 1 -- 172.21.15.99:0/3063712936 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988058ba0 con 0x55b59ed0afc0 2024-03-29T09:11:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/3063712936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ed0afc0 msgr2=0x55b59ece15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ed0afc0 0x55b59ece15f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc988060800 tx=0x7fc9880583d0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/3063712936 shutdown_connections 2024-03-29T09:11:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ece0980 0x55b59ece1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ed09a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.075 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/3063712936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ed0afc0 0x55b59ece15f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/3063712936 >> 172.21.15.99:0/3063712936 conn(0x55b59ec41650 msgr2=0x55b59ed283e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/3063712936 shutdown_connections 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.071+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/3063712936 wait complete. 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.072+0000 7fc9b5d4db80 1 -- start start 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.072+0000 7fc9b5d4db80 1 Processor -- start 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed06490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 0x55b59ed104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 0x55b59ed10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ebcfd30 con 0x55b59ece0980 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ebcf9d0 con 0x55b59ed05c20 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b59eab7650 con 0x55b59ed0afc0 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc9b5d4db80 10 client.0 _ll_register_callbacks cb 0x55b59ebd9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed06490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed06490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57766/0 (socket says 172.21.15.99:57766) 2024-03-29T09:11:55.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 -- 172.21.15.99:0/1209716180 learned_addr learned my addr 172.21.15.99:0/1209716180 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 0x55b59ed104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99be00700 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 0x55b59ed10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 -- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 msgr2=0x55b59ed10a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 0x55b59ed10a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 -- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 msgr2=0x55b59ed104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 0x55b59ed104c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.073+0000 7fc99b400700 1 -- 172.21.15.99:0/1209716180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c153f20 con 0x55b59ece0980 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc99b400700 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed06490 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00c800 tx=0x7fc98c009050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdc090d6e404e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc97be00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc97be00700 1 -- 172.21.15.99:0/1209716180 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0074e0 con 0x55b59ece0980 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc97be00700 1 -- 172.21.15.99:0/1209716180 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc98c007b10 con 0x55b59ece0980 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b59ebf4330 con 0x55b59ece0980 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc97be00700 1 -- 172.21.15.99:0/1209716180 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc98c007640 con 0x55b59ece0980 2024-03-29T09:11:55.077 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc9b5d4db80 10 client.6270 fetch_fsmap learned FSMap version 25 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55b59ebf9e50 con 0x55b59ece0980 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.074+0000 7fc97be00700 1 -- 172.21.15.99:0/1209716180 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b59ebf4330 con 0x55b59ece0980 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc97be00700 1 -- 172.21.15.99:0/1209716180 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc98c01f410 con 0x55b59ece0980 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc9b5d4db80 10 client.6270 fetch_fsmap finished waiting for FSMap version 25 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc9b5d4db80 10 client.6270 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc9b5d4db80 20 client.6270 populate_metadata read hostname 'smithi099' 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ed04800 0x55b59ed452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc9b5d4db80 4 client.6270 mds_command: new command op to 6105 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 --> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b59ec2c6b0 con 0x55b59ed04800 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ed04800 0x55b59ed452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc99aa00700 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ed04800 0x55b59ed452a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c449fa07185734db server_cookie=576e863e54a629d9 in_seq=0 out_seq=0 2024-03-29T09:11:55.078 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.075+0000 7fc97be00700 10 client.6270 ms_handle_connect on v2:172.21.15.99:6835/2396189132 2024-03-29T09:11:55.079 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc97be00700 1 -- 172.21.15.99:0/1209716180 <== mds.0 v2:172.21.15.99:6835/2396189132 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b59ec2c6b0 con 0x55b59ed04800 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc97be00700 10 client.6270 handle_command_reply: tid=0 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 client.6270 shutdown 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 msgr2=0x55b59ed06490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed06490 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00c800 tx=0x7fc98c009050 comp rx=0 tx=0).stop 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 shutdown_connections 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ed04800 0x55b59ed452a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 0x55b59ed10a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed06490 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc97be00700 0 client.6270 ms_handle_reset on v2:172.21.15.99:6835/2396189132 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/1209716180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 0x55b59ed104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 >> 172.21.15.99:0/1209716180 conn(0x55b59ec41650 msgr2=0x55b59ed09ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.076+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 shutdown_connections 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.077+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/1209716180 wait complete. 2024-03-29T09:11:55.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.077+0000 7fc9b5d4db80 20 client.6270 trim_cache size 0 max 16384 2024-03-29T09:11:55.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.094+0000 7fc9b5d4db80 1 Processor -- start 2024-03-29T09:11:55.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.094+0000 7fc9b5d4db80 1 -- start start 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ecdf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 0x55b59ecf2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 0x55b59ed494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ecde5c0 con 0x55b59ece0980 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ece0080 con 0x55b59ed05c20 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b59ece3030 con 0x55b59ed0afc0 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc99b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ecdf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc99b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ecdf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57770/0 (socket says 172.21.15.99:57770) 2024-03-29T09:11:55.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.095+0000 7fc99b400700 1 -- 172.21.15.99:0/901541295 learned_addr learned my addr 172.21.15.99:0/901541295 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc99b400700 1 -- 172.21.15.99:0/901541295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 msgr2=0x55b59ed494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc99b400700 1 --2- 172.21.15.99:0/901541295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 0x55b59ed494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc99b400700 1 -- 172.21.15.99:0/901541295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 msgr2=0x55b59ecf2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc99b400700 1 --2- 172.21.15.99:0/901541295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 0x55b59ecf2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc99b400700 1 -- 172.21.15.99:0/901541295 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b59eab9c80 con 0x55b59ece0980 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc99b400700 1 --2- 172.21.15.99:0/901541295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ecdf700 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00cf20 tx=0x7fc98c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82983f1a284edd04 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc987e00700 1 -- 172.21.15.99:0/901541295 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c01f970 con 0x55b59ece0980 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc987e00700 1 -- 172.21.15.99:0/901541295 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c023070 con 0x55b59ece0980 2024-03-29T09:11:55.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.096+0000 7fc987e00700 1 -- 172.21.15.99:0/901541295 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c01fbb0 con 0x55b59ece0980 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/901541295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 msgr2=0x55b59ecdf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/901541295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ecdf700 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00cf20 tx=0x7fc98c02e000 comp rx=0 tx=0).stop 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/901541295 shutdown_connections 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/901541295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed0afc0 0x55b59ed494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/901541295 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ecdf700 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/901541295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed05c20 0x55b59ecf2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/901541295 >> 172.21.15.99:0/901541295 conn(0x55b59ec41650 msgr2=0x55b59ed41920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/901541295 shutdown_connections 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.097+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/901541295 wait complete. 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.098+0000 7fc9b5d4db80 1 -- start start 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.098+0000 7fc9b5d4db80 1 Processor -- start 2024-03-29T09:11:55.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed0a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ece2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc9b5d4db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed0afc0 0x55b59ece1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ed892b0 con 0x55b59ece0980 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59ed85030 con 0x55b59ed0afc0 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc9b5d4db80 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b59ece1580 con 0x55b59ed05c20 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc9b5d4db80 10 client.0 _ll_register_callbacks cb 0x55b59ebd9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed0a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed0a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57778/0 (socket says 172.21.15.99:57778) 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 -- 172.21.15.99:0/4171525905 learned_addr learned my addr 172.21.15.99:0/4171525905 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99aa00700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ece2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99be00700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed0afc0 0x55b59ece1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 -- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 msgr2=0x55b59ece2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ece2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 -- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed0afc0 msgr2=0x55b59ece1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed0afc0 0x55b59ece1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99b400700 1 -- 172.21.15.99:0/4171525905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b59ebf9e50 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.099+0000 7fc99be00700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed0afc0 0x55b59ece1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc99b400700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed0a810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc98c02c040 tx=0x7fc98c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df9ba313478f046 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc97be00700 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc97be00700 1 -- 172.21.15.99:0/4171525905 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c01f970 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc988060d30 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc97be00700 1 -- 172.21.15.99:0/4171525905 <== mon.0 v2:172.21.15.99:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc98c000c30 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc97be00700 1 -- 172.21.15.99:0/4171525905 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc98c01f410 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 10 client.6276 fetch_fsmap learned FSMap version 25 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55b59ea5ffe0 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc97be00700 1 -- 172.21.15.99:0/4171525905 <== mon.0 v2:172.21.15.99:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc988060d30 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc97be00700 1 -- 172.21.15.99:0/4171525905 <== mon.0 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc98c034440 con 0x55b59ece0980 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 10 client.6276 fetch_fsmap finished waiting for FSMap version 25 2024-03-29T09:11:55.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 10 client.6276 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 20 client.6276 populate_metadata read hostname 'smithi099' 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ece1f50 0x55b59ecdc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.100+0000 7fc9b5d4db80 4 client.6276 mds_command: new command op to 6105 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 --> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b59ea82ca0 con 0x55b59ece1f50 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc99aa00700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ece1f50 0x55b59ecdc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc99aa00700 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ece1f50 0x55b59ecdc0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ffc4309f53a3369 server_cookie=c90efd6bca06a24c in_seq=0 out_seq=0 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc97be00700 10 client.6276 ms_handle_connect on v2:172.21.15.99:6835/2396189132 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc97be00700 1 -- 172.21.15.99:0/4171525905 <== mds.0 v2:172.21.15.99:6835/2396189132 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b59ea82ca0 con 0x55b59ece1f50 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc97be00700 10 client.6276 handle_command_reply: tid=0 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc9b5d4db80 1 client.6276 shutdown 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 msgr2=0x55b59ed0a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.101+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed0a810 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc98c02c040 tx=0x7fc98c001dc0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 shutdown_connections 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:6835/2396189132,v1:172.21.15.99:6837/2396189132] conn(0x55b59ece1f50 0x55b59ecdc0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b59ed0afc0 0x55b59ece1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b59ece0980 0x55b59ed0a810 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc97be00700 0 client.6276 ms_handle_reset on v2:172.21.15.99:6835/2396189132 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 --2- 172.21.15.99:0/4171525905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b59ed05c20 0x55b59ece2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 >> 172.21.15.99:0/4171525905 conn(0x55b59ec41650 msgr2=0x55b59ecf2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 shutdown_connections 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 1 -- 172.21.15.99:0/4171525905 wait complete. 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9b5d4db80 20 client.6276 trim_cache size 0 max 16384 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stdout: "cluster_fsid": "83e3f490-14b9-4a33-8e2d-bbf792b299cc", 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stdout: "whoami": 0, 2024-03-29T09:11:55.106 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6105, 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "want_state": "up:active", 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "state": "up:active", 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs", 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "rank_uptime": 11.447593424000001, 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "mdsmap_epoch": 25, 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch": 55, 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_epoch_barrier": 55, 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout: "uptime": 16.474811125999999 2024-03-29T09:11:55.107 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:11:55.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc97c06ac90 msgr2=0x7fc97c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9a2200700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc97c06ac90 0x7fc97c06d150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc98c015040 tx=0x7fc98c007000 comp rx=0 tx=0).stop 2024-03-29T09:11:55.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c14da80 msgr2=0x7fc99c1427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.102+0000 7fc9a2200700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c14da80 0x7fc99c1427a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc988011aa0 tx=0x7fc988011e60 comp rx=0 tx=0).stop 2024-03-29T09:11:55.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 shutdown_connections 2024-03-29T09:11:55.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc97c06ac90 0x7fc97c06d150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c142f50 0x7fc99c0af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc980003a30 0x7fc99c14ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 --2- 172.21.15.99:0/1158734363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c14da80 0x7fc99c1427a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 >> 172.21.15.99:0/1158734363 conn(0x7fc99c066540 msgr2=0x7fc99c155530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 shutdown_connections 2024-03-29T09:11:55.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.103+0000 7fc9a2200700 1 -- 172.21.15.99:0/1158734363 wait complete. 2024-03-29T09:11:55.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:11:55.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.309+0000 7f6dc1c00700 1 Processor -- start 2024-03-29T09:11:55.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.310+0000 7f6dc1c00700 1 -- start start 2024-03-29T09:11:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6dc1c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 0x7f6dbc161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6dc1c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc0d1be0 con 0x7f6dbc0cdcb0 2024-03-29T09:11:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6dc1c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc0b4980 con 0x7f6dbc0cfd70 2024-03-29T09:11:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6dc1c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b46a0 con 0x7f6dbc0d0150 2024-03-29T09:11:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6dbb400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6dbc0cdcb0 0x7f6dbc0ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:48278/0 (socket says 172.21.15.99:48278) 2024-03-29T09:11:55.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6dbb400700 1 -- 172.21.15.99:0/4165136595 learned_addr learned my addr 172.21.15.99:0/4165136595 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6dba000700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 0x7f6dbc161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912926114 0 0) 0x7f6dbc0d1be0 con 0x7f6dbc0cdcb0 2024-03-29T09:11:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.311+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da00035b0 con 0x7f6dbc0cdcb0 2024-03-29T09:11:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2740232019 0 0) 0x7f6da00035b0 con 0x7f6dbc0cdcb0 2024-03-29T09:11:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 msgr2=0x7f6dbc161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 0x7f6dbc161cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 >> v1:172.21.15.133:6789/0 conn(0x7f6dbc0cfd70 legacy=0x7f6dbc1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0d1590 con 0x7f6dbc0cdcb0 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f6db40019f0 con 0x7f6dbc0cdcb0 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 >> v1:172.21.15.99:6789/0 conn(0x7f6dbc0cdcb0 legacy=0x7f6dbc0ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 0x7f6da0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da0003f70 0x7f6da0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0004d90 0x7f6da0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b46a0 con 0x7f6da0003f70 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da000ba70 con 0x7f6da0004d90 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da000bc30 con 0x7f6dbc0d0150 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6dba000700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da0003f70 0x7f6da0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6dbb400700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0004d90 0x7f6da0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.312+0000 7f6dbaa00700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 0x7f6da0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6dba000700 1 -- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 msgr2=0x7f6da0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6dba000700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0d0150 0x7f6da0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6dba000700 1 -- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0004d90 msgr2=0x7f6da0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6dba000700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0004d90 0x7f6da0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6dba000700 1 -- 172.21.15.99:0/4165136595 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da00035b0 con 0x7f6da0003f70 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6dba000700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da0003f70 0x7f6da0004850 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009b60 tx=0x7f6da0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a84780cd3ec2c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800e6c0 con 0x7f6da0003f70 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800e820 con 0x7f6da0003f70 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.313+0000 7f6db9600700 1 -- 172.21.15.99:0/4165136595 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800eb90 con 0x7f6da0003f70 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.314+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4165136595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da0003f70 msgr2=0x7f6da0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.314+0000 7f6dc1c00700 1 --2- 172.21.15.99:0/4165136595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6da0003f70 0x7f6da0004850 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009b60 tx=0x7f6da0005880 comp rx=0 tx=0).stop 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.314+0000 7f6dbb400700 1 -- 172.21.15.99:0/4165136595 reap_dead start 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.314+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4165136595 shutdown_connections 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.314+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4165136595 >> 172.21.15.99:0/4165136595 conn(0x7f6dbc09c4a0 msgr2=0x7f6dbc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.314+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4165136595 shutdown_connections 2024-03-29T09:11:55.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.314+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4165136595 wait complete. 2024-03-29T09:11:55.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.315+0000 7f6dc1c00700 1 Processor -- start 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.316+0000 7f6dc1c00700 1 -- start start 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.316+0000 7f6dc1c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0003f70 0x7f6dbc0b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.316+0000 7f6dc1c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0b7140 0x7f6dbc1e7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.316+0000 7f6dc1c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc159650 0x7f6dbc0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dc1c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b83a0 con 0x7f6dbc159650 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dc1c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b7790 con 0x7f6da0003f70 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dc1c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0b7a70 con 0x7f6dbc0b7140 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc159650 0x7f6dbc0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc159650 0x7f6dbc0cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57802/0 (socket says 172.21.15.99:57802) 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 -- 172.21.15.99:0/4052338778 learned_addr learned my addr 172.21.15.99:0/4052338778 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbaa00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0003f70 0x7f6dbc0b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dba000700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0b7140 0x7f6dbc1e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 -- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0b7140 msgr2=0x7f6dbc1e7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0b7140 0x7f6dbc1e7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 -- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0003f70 msgr2=0x7f6dbc0b8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0003f70 0x7f6dbc0b8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 -- 172.21.15.99:0/4052338778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0b7d50 con 0x7f6dbc159650 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dba000700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0b7140 0x7f6dbc1e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbaa00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0003f70 0x7f6dbc0b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.317+0000 7f6dbb400700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc159650 0x7f6dbc0cb850 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc08d8b0 tx=0x7f6db400a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.318+0000 7f6db2a00700 1 -- 172.21.15.99:0/4052338778 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db400ad60 con 0x7f6dbc159650 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.318+0000 7f6db2a00700 1 -- 172.21.15.99:0/4052338778 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db40133f0 con 0x7f6dbc159650 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.318+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc0b8030 con 0x7f6dbc159650 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.318+0000 7f6db2a00700 1 -- 172.21.15.99:0/4052338778 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db40137a0 con 0x7f6dbc159650 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.318+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc159ad0 con 0x7f6dbc159650 2024-03-29T09:11:55.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.319+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc0b7520 con 0x7f6dbc159650 2024-03-29T09:11:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.320+0000 7f6db2a00700 1 -- 172.21.15.99:0/4052338778 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6db4013900 con 0x7f6dbc159650 2024-03-29T09:11:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.320+0000 7f6db2a00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6d9c06acc0 0x7f6d9c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.321+0000 7f6db2a00700 1 -- 172.21.15.99:0/4052338778 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6db4089d40 con 0x7f6dbc159650 2024-03-29T09:11:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.321+0000 7f6dbaa00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6d9c06acc0 0x7f6d9c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:55.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.321+0000 7f6dbaa00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6d9c06acc0 0x7f6d9c06d180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6dac000c00 tx=0x7f6dac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:55.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.325+0000 7f6db2a00700 1 -- 172.21.15.99:0/4052338778 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6db4054d40 con 0x7f6dbc159650 2024-03-29T09:11:55.573 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.574+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6dbc0b1e20 con 0x7f6dbc159650 2024-03-29T09:11:55.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.574+0000 7f6db2a00700 1 -- 172.21.15.99:0/4052338778 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6db4058360 con 0x7f6dbc159650 2024-03-29T09:11:55.574 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6d9c06acc0 msgr2=0x7f6d9c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6d9c06acc0 0x7f6d9c06d180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6dac000c00 tx=0x7f6dac009040 comp rx=0 tx=0).stop 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc159650 msgr2=0x7f6dbc0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc159650 0x7f6dbc0cb850 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc08d8b0 tx=0x7f6db400a5b0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 shutdown_connections 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6d9c06acc0 0x7f6d9c06d180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6dbc159650 0x7f6dbc0cb850 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6da0003f70 0x7f6dbc0b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 --2- 172.21.15.99:0/4052338778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6dbc0b7140 0x7f6dbc1e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.576+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 >> 172.21.15.99:0/4052338778 conn(0x7f6dbc09c4a0 msgr2=0x7f6dbc0b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.577+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 shutdown_connections 2024-03-29T09:11:55.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:55.577+0000 7f6dc1c00700 1 -- 172.21.15.99:0/4052338778 wait complete. 2024-03-29T09:11:55.594 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:11:55.594 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:11:55.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:11:55.595 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T09:11:55.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:55.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:11:55.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:11:55.595 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:11:55.595 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:55.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:55.595 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:11:55.637 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:11:55.637 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:55.638 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42072sec preferred_lft 42072sec 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::3caa:acff:feac:fccd/64 scope link 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5832:eeff:fe0b:ee92/64 scope link 2024-03-29T09:11:55.639 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:55.640 INFO:teuthology.orchestra.run.smithi099.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:11:55.640 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether fe:d0:b6:f6:cb:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:11:55.640 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::fcd0:b6ff:fef6:cba2/64 scope link 2024-03-29T09:11:55.640 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:11:55.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:55.640 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:11:55.740 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:11:55.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:55.743 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:11:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:11:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:11:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:11:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:11:55.787 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:11:55.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:55.788 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:11:55.788 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:11:55.788 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:11:55.788 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:11:55.788 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:11:55.788 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:11:55.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:55.988 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:11:56.032 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:11:56.032 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T09:11:56.033 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:56.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:56.033 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:11:56.139 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T09:11:56.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:11:56.139 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:56.181 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:11:56.181 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:11:56.181 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:56.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:11:56.366 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:11:56.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:11:56.429 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:11:56.432 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:11:56.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:56.432 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:11:56.473 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:11:56.473 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T09:11:56.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:11:56.582 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:11:56.582 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:11:56.646 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:11:56.648 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:11:56.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:56.649 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:11:56.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-29T09:11:56.679+0000 7fc50dfe95c0 -1 init, newargv = 0x55ceb4730620 newargc=15 2024-03-29T09:11:56.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[66366]: starting ceph client 2024-03-29T09:11:56.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[66366]: starting fuse 2024-03-29T09:11:56.695 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:11:56.695 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:11:56.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:11:56.695 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:11:56.695 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:11:56.696 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-29T09:11:56.697 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:11:56.874 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.66366.asok 2024-03-29T09:11:57.053 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.66366.asok 2024-03-29T09:11:57.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:57.053 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.66366.asok status 2024-03-29T09:11:57.272 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:11:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:11:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:11:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:11:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-29T09:11:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:11:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "66366", 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6291, 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6291 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:11:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3177016507 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 3177016507 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6291 192.168.0.1:0/3177016507", 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/3177016507", 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 25, 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 55, 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:11:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:11:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:11:57.275 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:11:57.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:57.287 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:11:57.287 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:57.329 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:11:57.329 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:57.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:57.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:11:57.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:11:57.434 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2024-03-29T09:11:57.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2024-03-29T09:11:57.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.677+0000 7f6444200700 1 Processor -- start 2024-03-29T09:11:57.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.677+0000 7f6444200700 1 -- start start 2024-03-29T09:11:57.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.678+0000 7f6444200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f643c15f290 0x7f643c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:57.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.678+0000 7f6444200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f643c099f80 0x7f643c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:57.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.678+0000 7f6444200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c09a3b0 con 0x7f643c0b2470 2024-03-29T09:11:57.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.678+0000 7f6444200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c0b2850 con 0x7f643c099f80 2024-03-29T09:11:57.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.678+0000 7f6444200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c0cf130 con 0x7f643c15f290 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.678+0000 7f6442200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f643c099f80 0x7f643c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6442200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f643c099f80 0x7f643c06c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57806/0 (socket says 172.21.15.99:57806) 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6442200700 1 -- 172.21.15.99:0/692901932 learned_addr learned my addr 172.21.15.99:0/692901932 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6440e00700 1 --2- 172.21.15.99:0/692901932 >> v2:172.21.15.133:3300/0 conn(0x7f643c15f290 0x7f643c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6442200700 1 -- 172.21.15.99:0/692901932 >> v2:172.21.15.133:3300/0 conn(0x7f643c15f290 msgr2=0x7f643c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6442200700 1 --2- 172.21.15.99:0/692901932 >> v2:172.21.15.133:3300/0 conn(0x7f643c15f290 0x7f643c099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6442200700 1 -- 172.21.15.99:0/692901932 >> v1:172.21.15.99:6789/0 conn(0x7f643c0b2470 legacy=0x7f643c15e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6442200700 1 -- 172.21.15.99:0/692901932 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c15ef80 con 0x7f643c099f80 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f643be00700 1 -- 172.21.15.99:0/692901932 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181170179 0 0) 0x7f643c09a3b0 con 0x7f643c0b2470 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f6442200700 1 --2- 172.21.15.99:0/692901932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c099f80 0x7f643c06c4e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f643c071c50 tx=0x7f643400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a8951f9f0aefd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.679+0000 7f643be00700 1 -- 172.21.15.99:0/692901932 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400e070 con 0x7f643c099f80 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f643be00700 1 -- 172.21.15.99:0/692901932 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400b9c0 con 0x7f643c099f80 2024-03-29T09:11:57.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f643be00700 1 -- 172.21.15.99:0/692901932 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400bd70 con 0x7f643c099f80 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 -- 172.21.15.99:0/692901932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c099f80 msgr2=0x7f643c06c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 --2- 172.21.15.99:0/692901932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c099f80 0x7f643c06c4e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f643c071c50 tx=0x7f643400b330 comp rx=0 tx=0).stop 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 -- 172.21.15.99:0/692901932 shutdown_connections 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 --2- 172.21.15.99:0/692901932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c099f80 0x7f643c06c4e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 --2- 172.21.15.99:0/692901932 >> v2:172.21.15.133:3300/0 conn(0x7f643c15f290 0x7f643c099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 -- 172.21.15.99:0/692901932 >> 172.21.15.99:0/692901932 conn(0x7f643c0b2b20 msgr2=0x7f643c0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 -- 172.21.15.99:0/692901932 shutdown_connections 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.680+0000 7f6444200700 1 -- 172.21.15.99:0/692901932 wait complete. 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.682+0000 7f6444200700 1 Processor -- start 2024-03-29T09:11:57.682 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.682+0000 7f6444200700 1 -- start start 2024-03-29T09:11:57.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.683+0000 7f6444200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f643c099f80 0x7f643c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:57.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.683+0000 7f6444200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0b2470 0x7f643c09d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:57.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.683+0000 7f6444200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f643c15f290 0x7f643c0c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:57.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.683+0000 7f6444200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c15aee0 con 0x7f643c0b2470 2024-03-29T09:11:57.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.683+0000 7f6444200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c09db20 con 0x7f643c099f80 2024-03-29T09:11:57.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.683+0000 7f6444200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c09de00 con 0x7f643c15f290 2024-03-29T09:11:57.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0b2470 0x7f643c09d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0b2470 0x7f643c09d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:57814/0 (socket says 172.21.15.99:57814) 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 -- 172.21.15.99:0/3770018829 learned_addr learned my addr 172.21.15.99:0/3770018829 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6441800700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f643c099f80 0x7f643c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6442200700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f643c15f290 0x7f643c0c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 -- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f643c15f290 msgr2=0x7f643c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f643c15f290 0x7f643c0c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 -- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f643c099f80 msgr2=0x7f643c15a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f643c099f80 0x7f643c15a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 -- 172.21.15.99:0/3770018829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434009b30 con 0x7f643c0b2470 2024-03-29T09:11:57.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.684+0000 7f6440e00700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0b2470 0x7f643c09d060 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f642800d800 tx=0x7f642800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:57.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.685+0000 7f6439600700 1 -- 172.21.15.99:0/3770018829 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280125f0 con 0x7f643c0b2470 2024-03-29T09:11:57.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.685+0000 7f6439600700 1 -- 172.21.15.99:0/3770018829 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6428012750 con 0x7f643c0b2470 2024-03-29T09:11:57.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.685+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c09e0e0 con 0x7f643c0b2470 2024-03-29T09:11:57.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.685+0000 7f6439600700 1 -- 172.21.15.99:0/3770018829 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428012b00 con 0x7f643c0b2470 2024-03-29T09:11:57.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.685+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c09e6a0 con 0x7f643c0b2470 2024-03-29T09:11:57.686 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.686+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c09e8c0 con 0x7f643c0b2470 2024-03-29T09:11:57.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.687+0000 7f6439600700 1 -- 172.21.15.99:0/3770018829 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6428012c60 con 0x7f643c0b2470 2024-03-29T09:11:57.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.688+0000 7f6439600700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f641c06acc0 0x7f641c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:57.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.688+0000 7f6439600700 1 -- 172.21.15.99:0/3770018829 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6428089510 con 0x7f643c0b2470 2024-03-29T09:11:57.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.688+0000 7f6441800700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f641c06acc0 0x7f641c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:57.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.688+0000 7f6441800700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f641c06acc0 0x7f641c06d180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6430000f60 tx=0x7f6430009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:57.692 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.692+0000 7f6439600700 1 -- 172.21.15.99:0/3770018829 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6428054480 con 0x7f643c0b2470 2024-03-29T09:11:57.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.980+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7f643c0678b0 con 0x7f643c0b2470 2024-03-29T09:11:57.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.985+0000 7f6439600700 1 -- 172.21.15.99:0/3770018829 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v20) v1 ==== 183+0+0 (secure 0 0 0) 0x7f6428057aa0 con 0x7f643c0b2470 2024-03-29T09:11:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f641c06acc0 msgr2=0x7f641c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f641c06acc0 0x7f641c06d180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6430000f60 tx=0x7f6430009040 comp rx=0 tx=0).stop 2024-03-29T09:11:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0b2470 msgr2=0x7f643c09d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0b2470 0x7f643c09d060 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f642800d800 tx=0x7f642800db10 comp rx=0 tx=0).stop 2024-03-29T09:11:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 shutdown_connections 2024-03-29T09:11:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f641c06acc0 0x7f641c06d180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f643c15f290 0x7f643c0c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f643c099f80 0x7f643c15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 --2- 172.21.15.99:0/3770018829 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f643c0b2470 0x7f643c09d060 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:57.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 >> 172.21.15.99:0/3770018829 conn(0x7f643c0b2b20 msgr2=0x7f643c0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:57.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 shutdown_connections 2024-03-29T09:11:57.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:57.988+0000 7f6444200700 1 -- 172.21.15.99:0/3770018829 wait complete. 2024-03-29T09:11:57.989 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:11:58.006 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:11:58.006 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:11:58.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:11:58.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.277+0000 7fd7d8800700 1 Processor -- start 2024-03-29T09:11:58.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.278+0000 7fd7d8800700 1 -- start start 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d8800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7d01608e0 0x7fd7d00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d8800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d00d1b20 0x7fd7d00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d8800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d00cddc0 con 0x7fd7d0160cc0 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d8800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d00ce0a0 con 0x7fd7d01608e0 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d8800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00ce380 con 0x7fd7d00d1b20 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d5e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7d01608e0 0x7fd7d00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d5e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7d01608e0 0x7fd7d00d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38270/0 (socket says 172.21.15.99:38270) 2024-03-29T09:11:58.279 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d5e00700 1 -- 172.21.15.99:0/3887235102 learned_addr learned my addr 172.21.15.99:0/3887235102 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d6800700 1 --2- 172.21.15.99:0/3887235102 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d00d1b20 0x7fd7d00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d5e00700 1 -- 172.21.15.99:0/3887235102 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d00d1b20 msgr2=0x7fd7d00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d5e00700 1 --2- 172.21.15.99:0/3887235102 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d00d1b20 0x7fd7d00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.279+0000 7fd7d5e00700 1 -- 172.21.15.99:0/3887235102 >> v1:172.21.15.133:6789/0 conn(0x7fd7d0160cc0 legacy=0x7fd7d00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d5e00700 1 -- 172.21.15.99:0/3887235102 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d00ce660 con 0x7fd7d01608e0 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d6800700 1 --2- 172.21.15.99:0/3887235102 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d00d1b20 0x7fd7d00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d5e00700 1 --2- 172.21.15.99:0/3887235102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d01608e0 0x7fd7d00d0150 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40098f0 tx=0x7fd7c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78d0d439c931fa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d4a00700 1 -- 172.21.15.99:0/3887235102 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4004230 con 0x7fd7d01608e0 2024-03-29T09:11:58.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d4a00700 1 -- 172.21.15.99:0/3887235102 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c4004390 con 0x7fd7d01608e0 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d4a00700 1 -- 172.21.15.99:0/3887235102 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40036a0 con 0x7fd7d01608e0 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d8800700 1 -- 172.21.15.99:0/3887235102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d01608e0 msgr2=0x7fd7d00d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.280+0000 7fd7d8800700 1 --2- 172.21.15.99:0/3887235102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d01608e0 0x7fd7d00d0150 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40098f0 tx=0x7fd7c4009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.281+0000 7fd7d8800700 1 -- 172.21.15.99:0/3887235102 shutdown_connections 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.281+0000 7fd7d8800700 1 --2- 172.21.15.99:0/3887235102 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d00d1b20 0x7fd7d00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.281+0000 7fd7d8800700 1 --2- 172.21.15.99:0/3887235102 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d01608e0 0x7fd7d00d0150 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.281+0000 7fd7d8800700 1 -- 172.21.15.99:0/3887235102 >> 172.21.15.99:0/3887235102 conn(0x7fd7d009c490 msgr2=0x7fd7d00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.281+0000 7fd7d8800700 1 -- 172.21.15.99:0/3887235102 shutdown_connections 2024-03-29T09:11:58.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.281+0000 7fd7d8800700 1 -- 172.21.15.99:0/3887235102 wait complete. 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.282+0000 7fd7d8800700 1 Processor -- start 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.282+0000 7fd7d8800700 1 -- start start 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d8800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d00d1b20 0x7fd7d0158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d8800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7d01608e0 0x7fd7d01f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d8800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d0160cc0 0x7fd7d00a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d8800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00cb290 con 0x7fd7d00d1b20 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d8800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01f2cd0 con 0x7fd7d01608e0 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d8800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00a0e80 con 0x7fd7d0160cc0 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d00d1b20 0x7fd7d0158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.283+0000 7fd7d5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d00d1b20 0x7fd7d0158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38286/0 (socket says 172.21.15.99:38286) 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d5e00700 1 -- 172.21.15.99:0/1443391539 learned_addr learned my addr 172.21.15.99:0/1443391539 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d5e00700 1 -- 172.21.15.99:0/1443391539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d0160cc0 msgr2=0x7fd7d00a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d5e00700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d0160cc0 0x7fd7d00a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d5e00700 1 -- 172.21.15.99:0/1443391539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7d01608e0 msgr2=0x7fd7d01f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d5e00700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7d01608e0 0x7fd7d01f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d5e00700 1 -- 172.21.15.99:0/1443391539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4009550 con 0x7fd7d00d1b20 2024-03-29T09:11:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d5e00700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d00d1b20 0x7fd7d0158fc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009130 tx=0x7fd7c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7ca000700 1 -- 172.21.15.99:0/1443391539 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400b660 con 0x7fd7d00d1b20 2024-03-29T09:11:58.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7ca000700 1 -- 172.21.15.99:0/1443391539 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c400b7e0 con 0x7fd7d00d1b20 2024-03-29T09:11:58.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7ca000700 1 -- 172.21.15.99:0/1443391539 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400bb90 con 0x7fd7d00d1b20 2024-03-29T09:11:58.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.284+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d00cacd0 con 0x7fd7d00d1b20 2024-03-29T09:11:58.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.285+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d015b6b0 con 0x7fd7d00d1b20 2024-03-29T09:11:58.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.286+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d015b510 con 0x7fd7d00d1b20 2024-03-29T09:11:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.287+0000 7fd7ca000700 1 -- 172.21.15.99:0/1443391539 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd7c4021070 con 0x7fd7d00d1b20 2024-03-29T09:11:58.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.288+0000 7fd7ca000700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd7b406ac90 0x7fd7b406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.288+0000 7fd7ca000700 1 -- 172.21.15.99:0/1443391539 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd7c408ad60 con 0x7fd7d00d1b20 2024-03-29T09:11:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.288+0000 7fd7d5400700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd7b406ac90 0x7fd7b406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.289+0000 7fd7d5400700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd7b406ac90 0x7fd7b406d150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd7d01f6300 tx=0x7fd7c0006c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:58.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.292+0000 7fd7ca000700 1 -- 172.21.15.99:0/1443391539 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd7c4055de0 con 0x7fd7d00d1b20 2024-03-29T09:11:58.497 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.497+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7d015b510 con 0x7fd7d00d1b20 2024-03-29T09:11:58.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.498+0000 7fd7ca000700 1 -- 172.21.15.99:0/1443391539 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7fd7d015b510 con 0x7fd7d00d1b20 2024-03-29T09:11:58.499 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:11:58.499 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:11:58.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd7b406ac90 msgr2=0x7fd7b406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd7b406ac90 0x7fd7b406d150 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd7d01f6300 tx=0x7fd7c0006c50 comp rx=0 tx=0).stop 2024-03-29T09:11:58.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d00d1b20 msgr2=0x7fd7d0158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d00d1b20 0x7fd7d0158fc0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009130 tx=0x7fd7c4004750 comp rx=0 tx=0).stop 2024-03-29T09:11:58.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 shutdown_connections 2024-03-29T09:11:58.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd7b406ac90 0x7fd7b406d150 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7d0160cc0 0x7fd7d00a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7d00d1b20 0x7fd7d0158fc0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 --2- 172.21.15.99:0/1443391539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7d01608e0 0x7fd7d01f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 >> 172.21.15.99:0/1443391539 conn(0x7fd7d009c490 msgr2=0x7fd7d015faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:58.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 shutdown_connections 2024-03-29T09:11:58.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.501+0000 7fd7d8800700 1 -- 172.21.15.99:0/1443391539 wait complete. 2024-03-29T09:11:58.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:11:41.952804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:11:42.633561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:11:58.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:11:58.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.714+0000 7f1513000700 1 Processor -- start 2024-03-29T09:11:58.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.714+0000 7f1513000700 1 -- start start 2024-03-29T09:11:58.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f1513000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c160770 0x7f150c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f1513000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f150c0d0d60 0x7f150c0d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f1513000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150c0d3810 con 0x7f150c0d3380 2024-03-29T09:11:58.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f1513000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150c0d1520 con 0x7f150c0d0d60 2024-03-29T09:11:58.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f1513000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c0d1800 con 0x7f150c160770 2024-03-29T09:11:58.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f150b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f150c0d0d60 0x7f150c0d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f150b400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f150c0d0d60 0x7f150c0d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38294/0 (socket says 172.21.15.99:38294) 2024-03-29T09:11:58.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.715+0000 7f150b400700 1 -- 172.21.15.99:0/267176302 learned_addr learned my addr 172.21.15.99:0/267176302 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:58.716 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150be00700 1 --2- 172.21.15.99:0/267176302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c160770 0x7f150c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150b400700 1 -- 172.21.15.99:0/267176302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c160770 msgr2=0x7f150c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:58.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150b400700 1 --2- 172.21.15.99:0/267176302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c160770 0x7f150c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150b400700 1 -- 172.21.15.99:0/267176302 >> v1:172.21.15.133:6789/0 conn(0x7f150c0d3380 legacy=0x7f150c0cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150b400700 1 -- 172.21.15.99:0/267176302 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c0d1ae0 con 0x7f150c0d0d60 2024-03-29T09:11:58.717 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150b400700 1 --2- 172.21.15.99:0/267176302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0d0d60 0x7f150c0d2e40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f14f80098f0 tx=0x7f14f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b9a64c5fc701b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150aa00700 1 -- 172.21.15.99:0/267176302 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800b8a0 con 0x7f150c0d0d60 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150aa00700 1 -- 172.21.15.99:0/267176302 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f800ba00 con 0x7f150c0d0d60 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.716+0000 7f150aa00700 1 -- 172.21.15.99:0/267176302 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800bd90 con 0x7f150c0d0d60 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 -- 172.21.15.99:0/267176302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0d0d60 msgr2=0x7f150c0d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 --2- 172.21.15.99:0/267176302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0d0d60 0x7f150c0d2e40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f14f80098f0 tx=0x7f14f8009c00 comp rx=0 tx=0).stop 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 -- 172.21.15.99:0/267176302 shutdown_connections 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 --2- 172.21.15.99:0/267176302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c160770 0x7f150c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 --2- 172.21.15.99:0/267176302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c0d0d60 0x7f150c0d2e40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 -- 172.21.15.99:0/267176302 >> 172.21.15.99:0/267176302 conn(0x7f150c08afd0 msgr2=0x7f150c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 -- 172.21.15.99:0/267176302 shutdown_connections 2024-03-29T09:11:58.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.717+0000 7f1513000700 1 -- 172.21.15.99:0/267176302 wait complete. 2024-03-29T09:11:58.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.719+0000 7f1513000700 1 Processor -- start 2024-03-29T09:11:58.719 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.719+0000 7f1513000700 1 -- start start 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1513000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150c0d0d60 0x7f150c0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1513000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c0d3380 0x7f150c0b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1513000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c160770 0x7f150c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1513000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c151e30 con 0x7f150c160770 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1513000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150c152040 con 0x7f150c0d0d60 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1513000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150c152320 con 0x7f150c0d3380 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c160770 0x7f150c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c160770 0x7f150c0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38300/0 (socket says 172.21.15.99:38300) 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 -- 172.21.15.99:0/1846402369 learned_addr learned my addr 172.21.15.99:0/1846402369 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f150b400700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c0d3380 0x7f150c0b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f150be00700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150c0d0d60 0x7f150c0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 -- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c0d3380 msgr2=0x7f150c0b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c0d3380 0x7f150c0b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 -- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150c0d0d60 msgr2=0x7f150c0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150c0d0d60 0x7f150c0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f1511000700 1 -- 172.21.15.99:0/1846402369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f8009550 con 0x7f150c160770 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.720+0000 7f150be00700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150c0d0d60 0x7f150c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.721+0000 7f1511000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c160770 0x7f150c0cc430 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f150400cc50 tx=0x7f150400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.721+0000 7f1503e00700 1 -- 172.21.15.99:0/1846402369 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1504003c00 con 0x7f150c160770 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.721+0000 7f1503e00700 1 -- 172.21.15.99:0/1846402369 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1504003d60 con 0x7f150c160770 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.721+0000 7f1503e00700 1 -- 172.21.15.99:0/1846402369 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150400ab60 con 0x7f150c160770 2024-03-29T09:11:58.722 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.721+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150c152600 con 0x7f150c160770 2024-03-29T09:11:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.721+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150c152bc0 con 0x7f150c160770 2024-03-29T09:11:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.722+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150c152de0 con 0x7f150c160770 2024-03-29T09:11:58.723 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.723+0000 7f1503e00700 1 -- 172.21.15.99:0/1846402369 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f150400acc0 con 0x7f150c160770 2024-03-29T09:11:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.724+0000 7f1503e00700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f14ec06ac90 0x7f14ec06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.724+0000 7f1503e00700 1 -- 172.21.15.99:0/1846402369 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1504089550 con 0x7f150c160770 2024-03-29T09:11:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.724+0000 7f150be00700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f14ec06ac90 0x7f14ec06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:58.724 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.725+0000 7f150be00700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f14ec06ac90 0x7f14ec06d150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f150c0ccff0 tx=0x7f14fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:58.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.729+0000 7f1503e00700 1 -- 172.21.15.99:0/1846402369 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1504054550 con 0x7f150c160770 2024-03-29T09:11:58.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.928+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f150c0678b0 con 0x7f150c160770 2024-03-29T09:11:58.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.929+0000 7f1503e00700 1 -- 172.21.15.99:0/1846402369 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1504057b70 con 0x7f150c160770 2024-03-29T09:11:58.929 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:11:58.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f14ec06ac90 msgr2=0x7f14ec06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.933 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f14ec06ac90 0x7f14ec06d150 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f150c0ccff0 tx=0x7f14fc008040 comp rx=0 tx=0).stop 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c160770 msgr2=0x7f150c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c160770 0x7f150c0cc430 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f150400cc50 tx=0x7f150400cf60 comp rx=0 tx=0).stop 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 shutdown_connections 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f14ec06ac90 0x7f14ec06d150 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f150c160770 0x7f150c0cc430 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150c0d0d60 0x7f150c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 --2- 172.21.15.99:0/1846402369 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f150c0d3380 0x7f150c0b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 >> 172.21.15.99:0/1846402369 conn(0x7f150c08afd0 msgr2=0x7f150c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 shutdown_connections 2024-03-29T09:11:58.934 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:58.931+0000 7f1513000700 1 -- 172.21.15.99:0/1846402369 wait complete. 2024-03-29T09:11:58.950 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:11:59.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.144+0000 7ff9b2800700 1 Processor -- start 2024-03-29T09:11:59.144 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.144+0000 7ff9b2800700 1 -- start start 2024-03-29T09:11:59.148 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9b2800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff9ac15d420 0x7ff9ac0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9b2800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9ac15d800 0x7ff9ac0d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9b2800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac0afdd0 con 0x7ff9ac0d2e50 2024-03-29T09:11:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9b2800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac0d12b0 con 0x7ff9ac15d800 2024-03-29T09:11:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9b2800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac0d3b00 con 0x7ff9ac15d420 2024-03-29T09:11:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9aaa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9ac15d800 0x7ff9ac0d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9aaa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9ac15d800 0x7ff9ac0d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38302/0 (socket says 172.21.15.99:38302) 2024-03-29T09:11:59.149 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.145+0000 7ff9aaa00700 1 -- 172.21.15.99:0/3502346720 learned_addr learned my addr 172.21.15.99:0/3502346720 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:59.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9ab400700 1 --2- 172.21.15.99:0/3502346720 >> v2:172.21.15.133:3300/0 conn(0x7ff9ac15d420 0x7ff9ac0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aaa00700 1 -- 172.21.15.99:0/3502346720 >> v2:172.21.15.133:3300/0 conn(0x7ff9ac15d420 msgr2=0x7ff9ac0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:59.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aaa00700 1 --2- 172.21.15.99:0/3502346720 >> v2:172.21.15.133:3300/0 conn(0x7ff9ac15d420 0x7ff9ac0d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aaa00700 1 -- 172.21.15.99:0/3502346720 >> v1:172.21.15.133:6789/0 conn(0x7ff9ac0d2e50 legacy=0x7ff9ac0d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:11:59.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aaa00700 1 -- 172.21.15.99:0/3502346720 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0b0070 con 0x7ff9ac15d800 2024-03-29T09:11:59.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9ab400700 1 --2- 172.21.15.99:0/3502346720 >> v2:172.21.15.133:3300/0 conn(0x7ff9ac15d420 0x7ff9ac0d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:59.150 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aaa00700 1 --2- 172.21.15.99:0/3502346720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac15d800 0x7ff9ac0d1cc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0d0c30 tx=0x7ff998009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b405e82d14359 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aa000700 1 -- 172.21.15.99:0/3502346720 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998004230 con 0x7ff9ac15d800 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aa000700 1 -- 172.21.15.99:0/3502346720 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff998004390 con 0x7ff9ac15d800 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.146+0000 7ff9aa000700 1 -- 172.21.15.99:0/3502346720 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980036a0 con 0x7ff9ac15d800 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 -- 172.21.15.99:0/3502346720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac15d800 msgr2=0x7ff9ac0d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3502346720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac15d800 0x7ff9ac0d1cc0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0d0c30 tx=0x7ff998009bd0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 -- 172.21.15.99:0/3502346720 shutdown_connections 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3502346720 >> v2:172.21.15.133:3300/0 conn(0x7ff9ac15d420 0x7ff9ac0d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.151 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3502346720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac15d800 0x7ff9ac0d1cc0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 -- 172.21.15.99:0/3502346720 >> 172.21.15.99:0/3502346720 conn(0x7ff9ac08afd0 msgr2=0x7ff9ac09c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 -- 172.21.15.99:0/3502346720 shutdown_connections 2024-03-29T09:11:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.147+0000 7ff9b2800700 1 -- 172.21.15.99:0/3502346720 wait complete. 2024-03-29T09:11:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.149+0000 7ff9b2800700 1 Processor -- start 2024-03-29T09:11:59.152 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.149+0000 7ff9b2800700 1 -- start start 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9b2800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9ac0d2e50 0x7ff9ac15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9b2800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9ac15d420 0x7ff9ac0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9b2800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0cc600 0x7ff9ac0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9b2800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0ccab0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9b2800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0cafa0 con 0x7ff9ac15d420 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9b2800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0cb280 con 0x7ff9ac0d2e50 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0cc600 0x7ff9ac0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0cc600 0x7ff9ac0ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38310/0 (socket says 172.21.15.99:38310) 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 -- 172.21.15.99:0/3301760101 learned_addr learned my addr 172.21.15.99:0/3301760101 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9ab400700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9ac0d2e50 0x7ff9ac15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9aaa00700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9ac15d420 0x7ff9ac0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 -- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9ac0d2e50 msgr2=0x7ff9ac15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9ac0d2e50 0x7ff9ac15b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 -- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9ac15d420 msgr2=0x7ff9ac0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9ac15d420 0x7ff9ac0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9abe00700 1 -- 172.21.15.99:0/3301760101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff998009550 con 0x7ff9ac0cc600 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9ab400700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9ac0d2e50 0x7ff9ac15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.150+0000 7ff9aaa00700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9ac15d420 0x7ff9ac0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.151+0000 7ff9abe00700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0cc600 0x7ff9ac0ca270 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400cc10 tx=0x7ff9a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.151+0000 7ff9a3400700 1 -- 172.21.15.99:0/3301760101 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400abb0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.151+0000 7ff9a3400700 1 -- 172.21.15.99:0/3301760101 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400ad10 con 0x7ff9ac0cc600 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.151+0000 7ff9a3400700 1 -- 172.21.15.99:0/3301760101 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4012660 con 0x7ff9ac0cc600 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.151+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac0cb560 con 0x7ff9ac0cc600 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.151+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac0cbb20 con 0x7ff9ac0cc600 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.152+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac0678b0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.153+0000 7ff9a3400700 1 -- 172.21.15.99:0/3301760101 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff9a40127c0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.154+0000 7ff9a3400700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff98c06ac90 0x7ff98c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.154+0000 7ff9a3400700 1 -- 172.21.15.99:0/3301760101 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9a4089cd0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.154+0000 7ff9ab400700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff98c06ac90 0x7ff98c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.155+0000 7ff9ab400700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff98c06ac90 0x7ff98c06d150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006fd0 tx=0x7ff99c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:59.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.158+0000 7ff9a3400700 1 -- 172.21.15.99:0/3301760101 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff9a4054cd0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.363+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9ac0b2ba0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.363+0000 7ff9a3400700 1 -- 172.21.15.99:0/3301760101 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9a40160a0 con 0x7ff9ac0cc600 2024-03-29T09:11:59.364 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:11:59.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff98c06ac90 msgr2=0x7ff98c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff98c06ac90 0x7ff98c06d150 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006fd0 tx=0x7ff99c008040 comp rx=0 tx=0).stop 2024-03-29T09:11:59.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0cc600 msgr2=0x7ff9ac0ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0cc600 0x7ff9ac0ca270 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400cc10 tx=0x7ff9a400a060 comp rx=0 tx=0).stop 2024-03-29T09:11:59.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 shutdown_connections 2024-03-29T09:11:59.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff98c06ac90 0x7ff98c06d150 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9ac0cc600 0x7ff9ac0ca270 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9ac0d2e50 0x7ff9ac15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 --2- 172.21.15.99:0/3301760101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9ac15d420 0x7ff9ac0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 >> 172.21.15.99:0/3301760101 conn(0x7ff9ac08afd0 msgr2=0x7ff9ac06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:59.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 shutdown_connections 2024-03-29T09:11:59.368 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.366+0000 7ff9b2800700 1 -- 172.21.15.99:0/3301760101 wait complete. 2024-03-29T09:11:59.384 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:11:59.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.578+0000 7facad600700 1 Processor -- start 2024-03-29T09:11:59.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.578+0000 7facad600700 1 -- start start 2024-03-29T09:11:59.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7facad600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faca8160770 0x7faca80b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7facad600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca80d1520 con 0x7faca80d3550 2024-03-29T09:11:59.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7facad600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca80d1800 con 0x7faca80d0d60 2024-03-29T09:11:59.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7facad600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca80d1ae0 con 0x7faca8160770 2024-03-29T09:11:59.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7faca6a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faca8160770 0x7faca80b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7faca7400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7faca80d3550 0x7faca80ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36160/0 (socket says 172.21.15.99:36160) 2024-03-29T09:11:59.580 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7faca6a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7faca8160770 0x7faca80b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38314/0 (socket says 172.21.15.99:38314) 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.579+0000 7faca7400700 1 -- 172.21.15.99:0/1909076665 learned_addr learned my addr 172.21.15.99:0/1909076665 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca6a00700 1 -- 172.21.15.99:0/1909076665 >> v1:172.21.15.133:6789/0 conn(0x7faca80d0d60 legacy=0x7faca80d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca6a00700 1 -- 172.21.15.99:0/1909076665 >> v1:172.21.15.99:6789/0 conn(0x7faca80d3550 legacy=0x7faca80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca6a00700 1 -- 172.21.15.99:0/1909076665 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca80d1dc0 con 0x7faca8160770 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca5600700 1 -- 172.21.15.99:0/1909076665 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734972669 0 0) 0x7faca80d1520 con 0x7faca80d3550 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca6a00700 1 --2- 172.21.15.99:0/1909076665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca8160770 0x7faca80b3cc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fac98009840 tx=0x7fac98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3757e499fb0ccc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca5600700 1 -- 172.21.15.99:0/1909076665 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98011830 con 0x7faca8160770 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca5600700 1 -- 172.21.15.99:0/1909076665 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98011990 con 0x7faca8160770 2024-03-29T09:11:59.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.580+0000 7faca5600700 1 -- 172.21.15.99:0/1909076665 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98011d20 con 0x7faca8160770 2024-03-29T09:11:59.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.581+0000 7facad600700 1 -- 172.21.15.99:0/1909076665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca8160770 msgr2=0x7faca80b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.581+0000 7facad600700 1 --2- 172.21.15.99:0/1909076665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca8160770 0x7faca80b3cc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fac98009840 tx=0x7fac98009b50 comp rx=0 tx=0).stop 2024-03-29T09:11:59.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.581+0000 7facad600700 1 -- 172.21.15.99:0/1909076665 shutdown_connections 2024-03-29T09:11:59.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.581+0000 7facad600700 1 --2- 172.21.15.99:0/1909076665 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca8160770 0x7faca80b3cc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.581+0000 7facad600700 1 -- 172.21.15.99:0/1909076665 >> 172.21.15.99:0/1909076665 conn(0x7faca808afd0 msgr2=0x7faca809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:59.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.581+0000 7facad600700 1 -- 172.21.15.99:0/1909076665 shutdown_connections 2024-03-29T09:11:59.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.581+0000 7facad600700 1 -- 172.21.15.99:0/1909076665 wait complete. 2024-03-29T09:11:59.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.582+0000 7facad600700 1 Processor -- start 2024-03-29T09:11:59.583 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.583+0000 7facad600700 1 -- start start 2024-03-29T09:11:59.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.583+0000 7facad600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca80d0d60 0x7faca8153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.583+0000 7facad600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca80d3550 0x7faca815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7facad600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faca8160770 0x7faca80b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7facad600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca8154500 con 0x7faca80d0d60 2024-03-29T09:11:59.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7facad600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca815afe0 con 0x7faca80d3550 2024-03-29T09:11:59.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7facad600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca80b64c0 con 0x7faca8160770 2024-03-29T09:11:59.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca80d0d60 0x7faca8153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca80d0d60 0x7faca8153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38324/0 (socket says 172.21.15.99:38324) 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 -- 172.21.15.99:0/468719837 learned_addr learned my addr 172.21.15.99:0/468719837 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca7400700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faca8160770 0x7faca80b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6000700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca80d3550 0x7faca815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 -- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faca8160770 msgr2=0x7faca80b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faca8160770 0x7faca80b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 -- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca80d3550 msgr2=0x7faca815a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca80d3550 0x7faca815a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 -- 172.21.15.99:0/468719837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac980094a0 con 0x7faca80d0d60 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca7400700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faca8160770 0x7faca80b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6000700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca80d3550 0x7faca815a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.584+0000 7faca6a00700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca80d0d60 0x7faca8153fc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fac980090b0 tx=0x7fac98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:59.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.585+0000 7fac9ea00700 1 -- 172.21.15.99:0/468719837 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980036a0 con 0x7faca80d0d60 2024-03-29T09:11:59.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.585+0000 7fac9ea00700 1 -- 172.21.15.99:0/468719837 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9801e070 con 0x7faca80d0d60 2024-03-29T09:11:59.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.585+0000 7facad600700 1 -- 172.21.15.99:0/468719837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca815ad00 con 0x7faca80d0d60 2024-03-29T09:11:59.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.585+0000 7fac9ea00700 1 -- 172.21.15.99:0/468719837 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98003a20 con 0x7faca80d0d60 2024-03-29T09:11:59.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.585+0000 7facad600700 1 -- 172.21.15.99:0/468719837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca80b6150 con 0x7faca80d0d60 2024-03-29T09:11:59.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.586+0000 7facad600700 1 -- 172.21.15.99:0/468719837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca80678b0 con 0x7faca80d0d60 2024-03-29T09:11:59.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.587+0000 7fac9ea00700 1 -- 172.21.15.99:0/468719837 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fac98003b80 con 0x7faca80d0d60 2024-03-29T09:11:59.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.587+0000 7fac9ea00700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fac8806ac90 0x7fac8806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:11:59.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.587+0000 7fac9ea00700 1 -- 172.21.15.99:0/468719837 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac9808a690 con 0x7faca80d0d60 2024-03-29T09:11:59.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.588+0000 7faca6000700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fac8806ac90 0x7fac8806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:11:59.588 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.588+0000 7faca6000700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fac8806ac90 0x7fac8806d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faca80b1eb0 tx=0x7fac94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:11:59.592 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.592+0000 7fac9ea00700 1 -- 172.21.15.99:0/468719837 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fac98055720 con 0x7faca80d0d60 2024-03-29T09:11:59.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.801+0000 7facad600700 1 -- 172.21.15.99:0/468719837 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faca80d0760 con 0x7faca80d0d60 2024-03-29T09:11:59.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.801+0000 7fac9ea00700 1 -- 172.21.15.99:0/468719837 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac98058d40 con 0x7faca80d0d60 2024-03-29T09:11:59.801 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:11:59.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 -- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fac8806ac90 msgr2=0x7fac8806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fac8806ac90 0x7fac8806d150 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7faca80b1eb0 tx=0x7fac94009040 comp rx=0 tx=0).stop 2024-03-29T09:11:59.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 -- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca80d0d60 msgr2=0x7faca8153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:11:59.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca80d0d60 0x7faca8153fc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fac980090b0 tx=0x7fac98004750 comp rx=0 tx=0).stop 2024-03-29T09:11:59.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 -- 172.21.15.99:0/468719837 shutdown_connections 2024-03-29T09:11:59.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fac8806ac90 0x7fac8806d150 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.804 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faca8160770 0x7faca80b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca80d0d60 0x7faca8153fc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 --2- 172.21.15.99:0/468719837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faca80d3550 0x7faca815a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:11:59.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 -- 172.21.15.99:0/468719837 >> 172.21.15.99:0/468719837 conn(0x7faca808afd0 msgr2=0x7faca815fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:11:59.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 -- 172.21.15.99:0/468719837 shutdown_connections 2024-03-29T09:11:59.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:11:59.804+0000 7facad600700 1 -- 172.21.15.99:0/468719837 wait complete. 2024-03-29T09:11:59.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:12:00.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.013+0000 7f8fc9000700 1 Processor -- start 2024-03-29T09:12:00.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.013+0000 7f8fc9000700 1 -- start start 2024-03-29T09:12:00.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.014+0000 7f8fc9000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8fc41608e0 0x7f8fc40d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.014+0000 7f8fc9000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40cddc0 con 0x7f8fc40d1b20 2024-03-29T09:12:00.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc9000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40ce0a0 con 0x7f8fc4160cc0 2024-03-29T09:12:00.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc9000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40ce380 con 0x7f8fc41608e0 2024-03-29T09:12:00.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc1e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8fc41608e0 0x7f8fc40d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc2800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8fc40d1b20 0x7f8fc40afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36162/0 (socket says 172.21.15.99:36162) 2024-03-29T09:12:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc2800700 1 -- 172.21.15.99:0/945963879 learned_addr learned my addr 172.21.15.99:0/945963879 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc1e00700 1 -- 172.21.15.99:0/945963879 >> v1:172.21.15.133:6789/0 conn(0x7f8fc4160cc0 legacy=0x7f8fc40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc1e00700 1 -- 172.21.15.99:0/945963879 >> v1:172.21.15.99:6789/0 conn(0x7f8fc40d1b20 legacy=0x7f8fc40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc1e00700 1 -- 172.21.15.99:0/945963879 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40ce660 con 0x7f8fc41608e0 2024-03-29T09:12:00.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.015+0000 7f8fc1e00700 1 --2- 172.21.15.99:0/945963879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc40d0150 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009840 tx=0x7f8fb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0164136c9d0c0b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc0a00700 1 -- 172.21.15.99:0/945963879 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4011830 con 0x7f8fc41608e0 2024-03-29T09:12:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc0a00700 1 -- 172.21.15.99:0/945963879 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4011990 con 0x7f8fc41608e0 2024-03-29T09:12:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc0a00700 1 -- 172.21.15.99:0/945963879 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4011d20 con 0x7f8fc41608e0 2024-03-29T09:12:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc9000700 1 -- 172.21.15.99:0/945963879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 msgr2=0x7f8fc40d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc9000700 1 --2- 172.21.15.99:0/945963879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc40d0150 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009840 tx=0x7f8fb4009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:00.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc9000700 1 -- 172.21.15.99:0/945963879 shutdown_connections 2024-03-29T09:12:00.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc9000700 1 --2- 172.21.15.99:0/945963879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc40d0150 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc9000700 1 -- 172.21.15.99:0/945963879 >> 172.21.15.99:0/945963879 conn(0x7f8fc409c490 msgr2=0x7f8fc40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:00.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc9000700 1 -- 172.21.15.99:0/945963879 shutdown_connections 2024-03-29T09:12:00.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.016+0000 7f8fc9000700 1 -- 172.21.15.99:0/945963879 wait complete. 2024-03-29T09:12:00.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.018+0000 7f8fc9000700 1 Processor -- start 2024-03-29T09:12:00.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.018+0000 7f8fc9000700 1 -- start start 2024-03-29T09:12:00.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc9000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fc40d1b20 0x7f8fc414ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc9000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc41f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc9000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc4160cc0 0x7f8fc41f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc9000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b4f60 con 0x7f8fc41608e0 2024-03-29T09:12:00.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc9000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b65e0 con 0x7f8fc4160cc0 2024-03-29T09:12:00.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc9000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc409cf00 con 0x7f8fc40d1b20 2024-03-29T09:12:00.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc41f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc1400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc41f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38334/0 (socket says 172.21.15.99:38334) 2024-03-29T09:12:00.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc1400700 1 -- 172.21.15.99:0/1666899419 learned_addr learned my addr 172.21.15.99:0/1666899419 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:00.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc1e00700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fc40d1b20 0x7f8fc414ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.019+0000 7f8fc2800700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc4160cc0 0x7f8fc41f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fc1400700 1 -- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fc40d1b20 msgr2=0x7f8fc414ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fc1400700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fc40d1b20 0x7f8fc414ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fc1400700 1 -- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc4160cc0 msgr2=0x7f8fc41f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fc1400700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc4160cc0 0x7f8fc41f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fc1400700 1 -- 172.21.15.99:0/1666899419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb40094a0 con 0x7f8fc41608e0 2024-03-29T09:12:00.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fc1400700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc41f37c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c450 tx=0x7f8fb000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fba000700 1 -- 172.21.15.99:0/1666899419 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00052d0 con 0x7f8fc41608e0 2024-03-29T09:12:00.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fba000700 1 -- 172.21.15.99:0/1666899419 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0005430 con 0x7f8fc41608e0 2024-03-29T09:12:00.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc414f4d0 con 0x7f8fc41608e0 2024-03-29T09:12:00.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.020+0000 7f8fba000700 1 -- 172.21.15.99:0/1666899419 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00038c0 con 0x7f8fc41608e0 2024-03-29T09:12:00.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.021+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc414fa90 con 0x7f8fc41608e0 2024-03-29T09:12:00.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.021+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc409d0c0 con 0x7f8fc41608e0 2024-03-29T09:12:00.026 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.022+0000 7f8fba000700 1 -- 172.21.15.99:0/1666899419 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8fb0003a20 con 0x7f8fc41608e0 2024-03-29T09:12:00.027 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.023+0000 7f8fba000700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8fa406ac90 0x7f8fa406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.024+0000 7f8fba000700 1 -- 172.21.15.99:0/1666899419 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8fb0088750 con 0x7f8fc41608e0 2024-03-29T09:12:00.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.024+0000 7f8fc1e00700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8fa406ac90 0x7f8fa406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.024+0000 7f8fc1e00700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8fa406ac90 0x7f8fa406d150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40090b0 tx=0x7f8fb4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.028+0000 7f8fba000700 1 -- 172.21.15.99:0/1666899419 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8fb00537f0 con 0x7f8fc41608e0 2024-03-29T09:12:00.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.246+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8fc40677a0 con 0x7f8fc41608e0 2024-03-29T09:12:00.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.247+0000 7f8fba000700 1 -- 172.21.15.99:0/1666899419 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8fb0056e10 con 0x7f8fc41608e0 2024-03-29T09:12:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8fa406ac90 msgr2=0x7f8fa406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8fa406ac90 0x7f8fa406d150 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40090b0 tx=0x7f8fb4008030 comp rx=0 tx=0).stop 2024-03-29T09:12:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 msgr2=0x7f8fc41f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc41f37c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c450 tx=0x7f8fb000c760 comp rx=0 tx=0).stop 2024-03-29T09:12:00.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 shutdown_connections 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8fa406ac90 0x7f8fa406d150 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8fc4160cc0 0x7f8fc41f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8fc40d1b20 0x7f8fc414ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 --2- 172.21.15.99:0/1666899419 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fc41608e0 0x7f8fc41f37c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 >> 172.21.15.99:0/1666899419 conn(0x7f8fc409c490 msgr2=0x7f8fc40b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 shutdown_connections 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.249+0000 7f8fc9000700 1 -- 172.21.15.99:0/1666899419 wait complete. 2024-03-29T09:12:00.250 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:12:00.267 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:12:00.267 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:00.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:00.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.539+0000 7f6fdfa00700 1 Processor -- start 2024-03-29T09:12:00.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.539+0000 7f6fdfa00700 1 -- start start 2024-03-29T09:12:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.540+0000 7f6fdfa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b1260 0x7f6fd815b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.540+0000 7f6fdfa00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6fd80b36f0 0x7f6fd80b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.540+0000 7f6fdfa00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd815c220 con 0x7f6fd815ae70 2024-03-29T09:12:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.540+0000 7f6fdfa00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd80b40c0 con 0x7f6fd80b36f0 2024-03-29T09:12:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.540+0000 7f6fdfa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd815b8f0 con 0x7f6fd80b1260 2024-03-29T09:12:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fdd000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6fd815ae70 0x7f6fd80b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36170/0 (socket says 172.21.15.99:36170) 2024-03-29T09:12:00.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fdd000700 1 -- 172.21.15.99:0/4081866000 learned_addr learned my addr 172.21.15.99:0/4081866000 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fdda00700 1 --2- 172.21.15.99:0/4081866000 >> v2:172.21.15.133:3300/0 conn(0x7f6fd80b36f0 0x7f6fd80b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fd7e00700 1 --2- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b1260 0x7f6fd815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fd7400700 1 -- 172.21.15.99:0/4081866000 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915895602 0 0) 0x7f6fd815c220 con 0x7f6fd815ae70 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fd7400700 1 -- 172.21.15.99:0/4081866000 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc00035b0 con 0x7f6fd815ae70 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fdda00700 1 -- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b1260 msgr2=0x7f6fd815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fdda00700 1 --2- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b1260 0x7f6fd815b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fdda00700 1 -- 172.21.15.99:0/4081866000 >> v1:172.21.15.99:6789/0 conn(0x7f6fd815ae70 legacy=0x7f6fd80b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.541+0000 7f6fdda00700 1 -- 172.21.15.99:0/4081866000 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd815bbd0 con 0x7f6fd80b36f0 2024-03-29T09:12:00.542 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdda00700 1 --2- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd80b36f0 0x7f6fd80b3b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009d90 tx=0x7f6fd000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd73963e694e3916 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fd7400700 1 -- 172.21.15.99:0/4081866000 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000e070 con 0x7f6fd80b36f0 2024-03-29T09:12:00.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fd7400700 1 -- 172.21.15.99:0/4081866000 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd000bb20 con 0x7f6fd80b36f0 2024-03-29T09:12:00.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fd7400700 1 -- 172.21.15.99:0/4081866000 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0012430 con 0x7f6fd80b36f0 2024-03-29T09:12:00.543 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 -- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd80b36f0 msgr2=0x7f6fd80b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd80b36f0 0x7f6fd80b3b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009d90 tx=0x7f6fd000b330 comp rx=0 tx=0).stop 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 -- 172.21.15.99:0/4081866000 shutdown_connections 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd80b36f0 0x7f6fd80b3b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/4081866000 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b1260 0x7f6fd815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 -- 172.21.15.99:0/4081866000 >> 172.21.15.99:0/4081866000 conn(0x7f6fd8066540 msgr2=0x7f6fd8066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 -- 172.21.15.99:0/4081866000 shutdown_connections 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.542+0000 7f6fdfa00700 1 -- 172.21.15.99:0/4081866000 wait complete. 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.544+0000 7f6fdfa00700 1 Processor -- start 2024-03-29T09:12:00.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.544+0000 7f6fdfa00700 1 -- start start 2024-03-29T09:12:00.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdfa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fd80b1260 0x7f6fd814b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdfa00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b36f0 0x7f6fd815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdfa00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd815ae70 0x7f6fd815eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.546 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdfa00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8072460 con 0x7f6fd80b1260 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdfa00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8071850 con 0x7f6fd815ae70 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdfa00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8071b30 con 0x7f6fd80b36f0 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdd000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fd80b1260 0x7f6fd814b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.545+0000 7f6fdd000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fd80b1260 0x7f6fd814b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38344/0 (socket says 172.21.15.99:38344) 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdd000700 1 -- 172.21.15.99:0/3556760708 learned_addr learned my addr 172.21.15.99:0/3556760708 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fd7e00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b36f0 0x7f6fd815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdda00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd815ae70 0x7f6fd815eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdd000700 1 -- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b36f0 msgr2=0x7f6fd815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdd000700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b36f0 0x7f6fd815e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdd000700 1 -- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd815ae70 msgr2=0x7f6fd815eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdd000700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd815ae70 0x7f6fd815eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdd000700 1 -- 172.21.15.99:0/3556760708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd0009a70 con 0x7f6fd80b1260 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdd000700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fd80b1260 0x7f6fd814b750 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc001eb0 tx=0x7f6fcc00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fd4c00700 1 -- 172.21.15.99:0/3556760708 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc005d70 con 0x7f6fd80b1260 2024-03-29T09:12:00.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fd4c00700 1 -- 172.21.15.99:0/3556760708 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0070d0 con 0x7f6fd80b1260 2024-03-29T09:12:00.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fd4c00700 1 -- 172.21.15.99:0/3556760708 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc007460 con 0x7f6fd80b1260 2024-03-29T09:12:00.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.546+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd8071e10 con 0x7f6fd80b1260 2024-03-29T09:12:00.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.547+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd80ae2e0 con 0x7f6fd80b1260 2024-03-29T09:12:00.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.548+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd80ae470 con 0x7f6fd80b1260 2024-03-29T09:12:00.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.548+0000 7f6fd4c00700 1 -- 172.21.15.99:0/3556760708 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6fcc0075c0 con 0x7f6fd80b1260 2024-03-29T09:12:00.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.549+0000 7f6fd4c00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6fb806ac90 0x7f6fb806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.549+0000 7f6fd4c00700 1 -- 172.21.15.99:0/3556760708 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6fcc0888f0 con 0x7f6fd80b1260 2024-03-29T09:12:00.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.549+0000 7f6fd7e00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6fb806ac90 0x7f6fb806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.550+0000 7f6fd7e00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6fb806ac90 0x7f6fb806d150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8155ad0 tx=0x7f6fc4009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.553+0000 7f6fd4c00700 1 -- 172.21.15.99:0/3556760708 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f6fcc0538f0 con 0x7f6fd80b1260 2024-03-29T09:12:00.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.756+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fd80677a0 con 0x7f6fd80b1260 2024-03-29T09:12:00.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.757+0000 7f6fd4c00700 1 -- 172.21.15.99:0/3556760708 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f6fcc017090 con 0x7f6fd80b1260 2024-03-29T09:12:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:00.757 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6fb806ac90 msgr2=0x7f6fb806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6fb806ac90 0x7f6fb806d150 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8155ad0 tx=0x7f6fc4009210 comp rx=0 tx=0).stop 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fd80b1260 msgr2=0x7f6fd814b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fd80b1260 0x7f6fd814b750 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc001eb0 tx=0x7f6fcc00c7b0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 shutdown_connections 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f6fb806ac90 0x7f6fb806d150 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fd815ae70 0x7f6fd815eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6fd80b1260 0x7f6fd814b750 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 --2- 172.21.15.99:0/3556760708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6fd80b36f0 0x7f6fd815e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 >> 172.21.15.99:0/3556760708 conn(0x7f6fd8066540 msgr2=0x7f6fd80ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:00.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 shutdown_connections 2024-03-29T09:12:00.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.760+0000 7f6fdfa00700 1 -- 172.21.15.99:0/3556760708 wait complete. 2024-03-29T09:12:00.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:11:41.952804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:11:42.633561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:12:00.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:12:00.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.970+0000 7f486ee00700 1 Processor -- start 2024-03-29T09:12:00.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.970+0000 7f486ee00700 1 -- start start 2024-03-29T09:12:00.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.971+0000 7f486ee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868160770 0x7f48680b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.971+0000 7f486ee00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680d1520 con 0x7f48680d3550 2024-03-29T09:12:00.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.971+0000 7f486ee00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48680d1800 con 0x7f48680d0d60 2024-03-29T09:12:00.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.971+0000 7f486ee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48680d1ae0 con 0x7f4868160770 2024-03-29T09:12:00.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f486ce00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f48680d3550 0x7f48680ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36184/0 (socket says 172.21.15.99:36184) 2024-03-29T09:12:00.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f486ce00700 1 -- 172.21.15.99:0/1304702733 learned_addr learned my addr 172.21.15.99:0/1304702733 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:00.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4867e00700 1 --2- 172.21.15.99:0/1304702733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868160770 0x7f48680b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4866a00700 1 -- 172.21.15.99:0/1304702733 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480594199 0 0) 0x7f48680d1520 con 0x7f48680d3550 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4866a00700 1 -- 172.21.15.99:0/1304702733 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f484c0035b0 con 0x7f48680d3550 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4866a00700 1 -- 172.21.15.99:0/1304702733 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731180070 0 0) 0x7f48680d1800 con 0x7f48680d0d60 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4866a00700 1 -- 172.21.15.99:0/1304702733 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48680d1520 con 0x7f48680d0d60 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4867e00700 1 -- 172.21.15.99:0/1304702733 >> v1:172.21.15.133:6789/0 conn(0x7f48680d0d60 legacy=0x7f48680d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4867e00700 1 -- 172.21.15.99:0/1304702733 >> v1:172.21.15.99:6789/0 conn(0x7f48680d3550 legacy=0x7f48680ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4867e00700 1 -- 172.21.15.99:0/1304702733 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680d1dc0 con 0x7f4868160770 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.972+0000 7f4867e00700 1 --2- 172.21.15.99:0/1304702733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868160770 0x7f48680b3cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4858009870 tx=0x7f4858009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1408648015dae333 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f4866a00700 1 -- 172.21.15.99:0/1304702733 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858011830 con 0x7f4868160770 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f4866a00700 1 -- 172.21.15.99:0/1304702733 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4858011990 con 0x7f4868160770 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f4866a00700 1 -- 172.21.15.99:0/1304702733 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858011d20 con 0x7f4868160770 2024-03-29T09:12:00.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f486ee00700 1 -- 172.21.15.99:0/1304702733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868160770 msgr2=0x7f48680b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f486ee00700 1 --2- 172.21.15.99:0/1304702733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868160770 0x7f48680b3cc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4858009870 tx=0x7f4858009b80 comp rx=0 tx=0).stop 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f486ee00700 1 -- 172.21.15.99:0/1304702733 shutdown_connections 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f486ee00700 1 --2- 172.21.15.99:0/1304702733 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4868160770 0x7f48680b3cc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f486ee00700 1 -- 172.21.15.99:0/1304702733 >> 172.21.15.99:0/1304702733 conn(0x7f486808afd0 msgr2=0x7f486809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f486ee00700 1 -- 172.21.15.99:0/1304702733 shutdown_connections 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.973+0000 7f486ee00700 1 -- 172.21.15.99:0/1304702733 wait complete. 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.975+0000 7f486ee00700 1 Processor -- start 2024-03-29T09:12:00.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.975+0000 7f486ee00700 1 -- start start 2024-03-29T09:12:00.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ee00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48680d0d60 0x7f48680c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ee00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48680d3550 0x7f4868149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ee00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868160770 0x7f48680cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ee00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486815b9f0 con 0x7f4868160770 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ee00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48680cd1f0 con 0x7f48680d0d60 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ee00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4868149d60 con 0x7f48680d3550 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868160770 0x7f48680cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868160770 0x7f48680cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38352/0 (socket says 172.21.15.99:38352) 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f486ce00700 1 -- 172.21.15.99:0/1295202318 learned_addr learned my addr 172.21.15.99:0/1295202318 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f4867400700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48680d3550 0x7f4868149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.976+0000 7f4867e00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48680d0d60 0x7f48680c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ce00700 1 -- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48680d3550 msgr2=0x7f4868149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ce00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48680d3550 0x7f4868149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ce00700 1 -- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48680d0d60 msgr2=0x7f48680c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ce00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48680d0d60 0x7f48680c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ce00700 1 -- 172.21.15.99:0/1295202318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48580094a0 con 0x7f4868160770 2024-03-29T09:12:00.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ce00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868160770 0x7f48680cf7f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f486000cb10 tx=0x7f486000ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f485fe00700 1 -- 172.21.15.99:0/1295202318 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486000a980 con 0x7f4868160770 2024-03-29T09:12:00.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f485fe00700 1 -- 172.21.15.99:0/1295202318 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486000aae0 con 0x7f4868160770 2024-03-29T09:12:00.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486815b370 con 0x7f4868160770 2024-03-29T09:12:00.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f485fe00700 1 -- 172.21.15.99:0/1295202318 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486000ae50 con 0x7f4868160770 2024-03-29T09:12:00.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.977+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48680ccc30 con 0x7f4868160770 2024-03-29T09:12:00.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.978+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486815bb30 con 0x7f4868160770 2024-03-29T09:12:00.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.979+0000 7f485fe00700 1 -- 172.21.15.99:0/1295202318 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f486001b3b0 con 0x7f4868160770 2024-03-29T09:12:00.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.980+0000 7f485fe00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f484806ac90 0x7f484806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:00.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.980+0000 7f485fe00700 1 -- 172.21.15.99:0/1295202318 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4860089060 con 0x7f4868160770 2024-03-29T09:12:00.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.980+0000 7f4867e00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f484806ac90 0x7f484806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:00.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.981+0000 7f4867e00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f484806ac90 0x7f484806d150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f48580097c0 tx=0x7f485801c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:00.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:00.985+0000 7f485fe00700 1 -- 172.21.15.99:0/1295202318 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f48600bc580 con 0x7f4868160770 2024-03-29T09:12:01.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.189+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48680d29d0 con 0x7f4868160770 2024-03-29T09:12:01.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.189+0000 7f485fe00700 1 -- 172.21.15.99:0/1295202318 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4860017020 con 0x7f4868160770 2024-03-29T09:12:01.189 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f484806ac90 msgr2=0x7f484806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f484806ac90 0x7f484806d150 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f48580097c0 tx=0x7f485801c040 comp rx=0 tx=0).stop 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868160770 msgr2=0x7f48680cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868160770 0x7f48680cf7f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f486000cb10 tx=0x7f486000ce20 comp rx=0 tx=0).stop 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 shutdown_connections 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f484806ac90 0x7f484806d150 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4868160770 0x7f48680cf7f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48680d0d60 0x7f48680c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 --2- 172.21.15.99:0/1295202318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48680d3550 0x7f4868149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.191+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 >> 172.21.15.99:0/1295202318 conn(0x7f486808afd0 msgr2=0x7f48680b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.192+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 shutdown_connections 2024-03-29T09:12:01.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.192+0000 7f486ee00700 1 -- 172.21.15.99:0/1295202318 wait complete. 2024-03-29T09:12:01.210 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:12:01.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.403+0000 7f05d1e00700 1 Processor -- start 2024-03-29T09:12:01.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.403+0000 7f05d1e00700 1 -- start start 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.404+0000 7f05d1e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05cc0adc40 0x7f05cc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.404+0000 7f05d1e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc06c580 con 0x7f05cc0ae020 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.404+0000 7f05d1e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05cc0abf10 con 0x7f05cc072120 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.404+0000 7f05d1e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc15c7a0 con 0x7f05cc0adc40 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05cb400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f05cc0ae020 0x7f05cc155070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36200/0 (socket says 172.21.15.99:36200) 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05cb400700 1 -- 172.21.15.99:0/2463228968 learned_addr learned my addr 172.21.15.99:0/2463228968 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05caa00700 1 --2- 172.21.15.99:0/2463228968 >> v2:172.21.15.133:3300/0 conn(0x7f05cc0adc40 0x7f05cc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960014219 0 0) 0x7f05cc06c580 con 0x7f05cc0ae020 2024-03-29T09:12:01.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b00035b0 con 0x7f05cc0ae020 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579353299 0 0) 0x7f05cc0abf10 con 0x7f05cc072120 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05cc06c580 con 0x7f05cc072120 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784592034 0 0) 0x7f05b00035b0 con 0x7f05cc0ae020 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 >> v2:172.21.15.133:3300/0 conn(0x7f05cc0adc40 msgr2=0x7f05cc15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 --2- 172.21.15.99:0/2463228968 >> v2:172.21.15.133:3300/0 conn(0x7f05cc0adc40 0x7f05cc15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 >> v1:172.21.15.133:6789/0 conn(0x7f05cc072120 legacy=0x7f05cc06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc0ac1f0 con 0x7f05cc0ae020 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f05c40019f0 con 0x7f05cc0ae020 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.405+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 >> v1:172.21.15.99:6789/0 conn(0x7f05cc0ae020 legacy=0x7f05cc155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05c9600700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05b0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05c9600700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b0004850 0x7f05b0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05c9600700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05b0005210 0x7f05b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc15c7a0 con 0x7f05b0003b90 2024-03-29T09:12:01.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b000c140 con 0x7f05b0004850 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b000c300 con 0x7f05b0005210 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05caa00700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05b0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05ca000700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b0004850 0x7f05b0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05cb400700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05b0005210 0x7f05b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05caa00700 1 -- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05b0005210 msgr2=0x7f05b000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05caa00700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05b0005210 0x7f05b000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05caa00700 1 -- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b0004850 msgr2=0x7f05b0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05caa00700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05b0004850 0x7f05b0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05caa00700 1 -- 172.21.15.99:0/2463228968 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00035b0 con 0x7f05b0003b90 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05cb400700 1 -- 172.21.15.99:0/2463228968 reap_dead start 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.406+0000 7f05caa00700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05b0003f90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009b60 tx=0x7f05b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82933ef497c9a1a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00e6c0 con 0x7f05b0003b90 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc00e820 con 0x7f05b0003b90 2024-03-29T09:12:01.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05c9600700 1 -- 172.21.15.99:0/2463228968 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00eb90 con 0x7f05b0003b90 2024-03-29T09:12:01.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05d1e00700 1 -- 172.21.15.99:0/2463228968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 msgr2=0x7f05b0003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05d1e00700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05b0003f90 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009b60 tx=0x7f05b0005880 comp rx=0 tx=0).stop 2024-03-29T09:12:01.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05d1e00700 1 -- 172.21.15.99:0/2463228968 shutdown_connections 2024-03-29T09:12:01.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05d1e00700 1 --2- 172.21.15.99:0/2463228968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05b0003f90 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05d1e00700 1 -- 172.21.15.99:0/2463228968 >> 172.21.15.99:0/2463228968 conn(0x7f05cc066540 msgr2=0x7f05cc066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:01.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05d1e00700 1 -- 172.21.15.99:0/2463228968 shutdown_connections 2024-03-29T09:12:01.411 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.407+0000 7f05d1e00700 1 -- 172.21.15.99:0/2463228968 wait complete. 2024-03-29T09:12:01.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.409+0000 7f05d1e00700 1 Processor -- start 2024-03-29T09:12:01.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.409+0000 7f05d1e00700 1 -- start start 2024-03-29T09:12:01.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05d1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05cc1f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05d1e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05cc14c960 0x7f05cc0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05d1e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05cc0cbd30 0x7f05cc1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05d1e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc06c8d0 con 0x7f05b0003b90 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05d1e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc152f90 con 0x7f05cc0cbd30 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05d1e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc0cc640 con 0x7f05cc14c960 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05caa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05cc1f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05caa00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05cc1f4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38366/0 (socket says 172.21.15.99:38366) 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.410+0000 7f05caa00700 1 -- 172.21.15.99:0/1490612744 learned_addr learned my addr 172.21.15.99:0/1490612744 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05caa00700 1 -- 172.21.15.99:0/1490612744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05cc14c960 msgr2=0x7f05cc0cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05caa00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05cc14c960 0x7f05cc0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05caa00700 1 -- 172.21.15.99:0/1490612744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05cc0cbd30 msgr2=0x7f05cc1594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05caa00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05cc0cbd30 0x7f05cc1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05caa00700 1 -- 172.21.15.99:0/1490612744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc0cc360 con 0x7f05b0003b90 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05caa00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05cc1f4960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c110 tx=0x7f05bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:01.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05c3400700 1 -- 172.21.15.99:0/1490612744 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00e6c0 con 0x7f05b0003b90 2024-03-29T09:12:01.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05c3400700 1 -- 172.21.15.99:0/1490612744 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc003ad0 con 0x7f05b0003b90 2024-03-29T09:12:01.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc1f3e80 con 0x7f05b0003b90 2024-03-29T09:12:01.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05c3400700 1 -- 172.21.15.99:0/1490612744 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc003e80 con 0x7f05b0003b90 2024-03-29T09:12:01.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.411+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc1f4440 con 0x7f05b0003b90 2024-03-29T09:12:01.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.412+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc0678b0 con 0x7f05b0003b90 2024-03-29T09:12:01.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.413+0000 7f05c3400700 1 -- 172.21.15.99:0/1490612744 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f05bc013470 con 0x7f05b0003b90 2024-03-29T09:12:01.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.414+0000 7f05c3400700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f05ac06ac90 0x7f05ac06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.414+0000 7f05c3400700 1 -- 172.21.15.99:0/1490612744 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f05bc08aee0 con 0x7f05b0003b90 2024-03-29T09:12:01.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.414+0000 7f05ca000700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f05ac06ac90 0x7f05ac06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.415+0000 7f05ca000700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f05ac06ac90 0x7f05ac06d150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f05cc071550 tx=0x7f05b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:01.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.418+0000 7f05c3400700 1 -- 172.21.15.99:0/1490612744 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f05bc056090 con 0x7f05b0003b90 2024-03-29T09:12:01.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.622+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05cc06dd80 con 0x7f05b0003b90 2024-03-29T09:12:01.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.623+0000 7f05c3400700 1 -- 172.21.15.99:0/1490612744 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05bc0170a0 con 0x7f05b0003b90 2024-03-29T09:12:01.623 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:12:01.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f05ac06ac90 msgr2=0x7f05ac06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f05ac06ac90 0x7f05ac06d150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f05cc071550 tx=0x7f05b8009040 comp rx=0 tx=0).stop 2024-03-29T09:12:01.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 msgr2=0x7f05cc1f4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05cc1f4960 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c110 tx=0x7f05bc0047c0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 shutdown_connections 2024-03-29T09:12:01.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f05ac06ac90 0x7f05ac06d150 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05cc0cbd30 0x7f05cc1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f05b0003b90 0x7f05cc1f4960 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 --2- 172.21.15.99:0/1490612744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f05cc14c960 0x7f05cc0cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 >> 172.21.15.99:0/1490612744 conn(0x7f05cc066540 msgr2=0x7f05cc06d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:01.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 shutdown_connections 2024-03-29T09:12:01.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.625+0000 7f05d1e00700 1 -- 172.21.15.99:0/1490612744 wait complete. 2024-03-29T09:12:01.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:12:01.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.836+0000 7fcd0ac00700 1 Processor -- start 2024-03-29T09:12:01.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.836+0000 7fcd0ac00700 1 -- start start 2024-03-29T09:12:01.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd0ac00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd04160770 0x7fcd040b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd0ac00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d0d60 0x7fcd040d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd0ac00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd040d3810 con 0x7fcd040d3380 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd0ac00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd040d1520 con 0x7fcd04160770 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd0ac00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040d1800 con 0x7fcd040d0d60 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd03e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd04160770 0x7fcd040b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd03e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fcd04160770 0x7fcd040b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38378/0 (socket says 172.21.15.99:38378) 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.837+0000 7fcd03e00700 1 -- 172.21.15.99:0/2287204709 learned_addr learned my addr 172.21.15.99:0/2287204709 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd03400700 1 --2- 172.21.15.99:0/2287204709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d0d60 0x7fcd040d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd03e00700 1 -- 172.21.15.99:0/2287204709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d0d60 msgr2=0x7fcd040d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd03e00700 1 --2- 172.21.15.99:0/2287204709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d0d60 0x7fcd040d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd03e00700 1 -- 172.21.15.99:0/2287204709 >> v1:172.21.15.133:6789/0 conn(0x7fcd040d3380 legacy=0x7fcd040cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd03e00700 1 -- 172.21.15.99:0/2287204709 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040d1ae0 con 0x7fcd04160770 2024-03-29T09:12:01.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd03400700 1 --2- 172.21.15.99:0/2287204709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d0d60 0x7fcd040d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:01.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd03e00700 1 --2- 172.21.15.99:0/2287204709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd040b3cc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fccf40098f0 tx=0x7fccf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f325f698b649d5c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd02a00700 1 -- 172.21.15.99:0/2287204709 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4004230 con 0x7fcd04160770 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd02a00700 1 -- 172.21.15.99:0/2287204709 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4004390 con 0x7fcd04160770 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.838+0000 7fcd02a00700 1 -- 172.21.15.99:0/2287204709 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf40036a0 con 0x7fcd04160770 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 -- 172.21.15.99:0/2287204709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 msgr2=0x7fcd040b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/2287204709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd040b3cc0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fccf40098f0 tx=0x7fccf4009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 -- 172.21.15.99:0/2287204709 shutdown_connections 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/2287204709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd040b3cc0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/2287204709 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d0d60 0x7fcd040d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 -- 172.21.15.99:0/2287204709 >> 172.21.15.99:0/2287204709 conn(0x7fcd0408afd0 msgr2=0x7fcd0409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 -- 172.21.15.99:0/2287204709 shutdown_connections 2024-03-29T09:12:01.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.839+0000 7fcd0ac00700 1 -- 172.21.15.99:0/2287204709 wait complete. 2024-03-29T09:12:01.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.841+0000 7fcd0ac00700 1 Processor -- start 2024-03-29T09:12:01.841 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.841+0000 7fcd0ac00700 1 -- start start 2024-03-29T09:12:01.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd0ac00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd040d0d60 0x7fcd040cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd0ac00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d3380 0x7fcd041f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd0ac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd0415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd0ac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040b9020 con 0x7fcd04160770 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd0ac00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040b8d40 con 0x7fcd040d0d60 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd0ac00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040ca390 con 0x7fcd040d3380 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd0415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd0415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38388/0 (socket says 172.21.15.99:38388) 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 -- 172.21.15.99:0/3824034790 learned_addr learned my addr 172.21.15.99:0/3824034790 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd03e00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd040d0d60 0x7fcd040cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd03400700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d3380 0x7fcd041f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 -- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d3380 msgr2=0x7fcd041f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d3380 0x7fcd041f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 -- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd040d0d60 msgr2=0x7fcd040cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd040d0d60 0x7fcd040cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.842+0000 7fcd08c00700 1 -- 172.21.15.99:0/3824034790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4009550 con 0x7fcd04160770 2024-03-29T09:12:01.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.843+0000 7fcd08c00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd0415e050 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00cc10 tx=0x7fccfc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:01.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.843+0000 7fccfbe00700 1 -- 172.21.15.99:0/3824034790 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00aa60 con 0x7fcd04160770 2024-03-29T09:12:01.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.843+0000 7fccfbe00700 1 -- 172.21.15.99:0/3824034790 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc00abc0 con 0x7fcd04160770 2024-03-29T09:12:01.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.843+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd040ca670 con 0x7fcd04160770 2024-03-29T09:12:01.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.843+0000 7fccfbe00700 1 -- 172.21.15.99:0/3824034790 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc012420 con 0x7fcd04160770 2024-03-29T09:12:01.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.843+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd041f28b0 con 0x7fcd04160770 2024-03-29T09:12:01.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.844+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd040cab90 con 0x7fcd04160770 2024-03-29T09:12:01.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.845+0000 7fccfbe00700 1 -- 172.21.15.99:0/3824034790 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fccfc012580 con 0x7fcd04160770 2024-03-29T09:12:01.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.846+0000 7fccfbe00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcce406ac90 0x7fcce406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:01.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.846+0000 7fccfbe00700 1 -- 172.21.15.99:0/3824034790 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fccfc0896e0 con 0x7fcd04160770 2024-03-29T09:12:01.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.847+0000 7fcd03e00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcce406ac90 0x7fcce406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:01.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.847+0000 7fcd03e00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcce406ac90 0x7fcce406d150 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009130 tx=0x7fccf4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:01.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:01.851+0000 7fccfbe00700 1 -- 172.21.15.99:0/3824034790 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcd040cab90 con 0x7fcd04160770 2024-03-29T09:12:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.053+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd040cab90 con 0x7fcd04160770 2024-03-29T09:12:02.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.054+0000 7fccfbe00700 1 -- 172.21.15.99:0/3824034790 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd040cab90 con 0x7fcd04160770 2024-03-29T09:12:02.054 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:12:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcce406ac90 msgr2=0x7fcce406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcce406ac90 0x7fcce406d150 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009130 tx=0x7fccf4008030 comp rx=0 tx=0).stop 2024-03-29T09:12:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 msgr2=0x7fcd0415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd0415e050 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00cc10 tx=0x7fccfc00cf20 comp rx=0 tx=0).stop 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 shutdown_connections 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcce406ac90 0x7fcce406d150 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd04160770 0x7fcd0415e050 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd040d0d60 0x7fcd040cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 --2- 172.21.15.99:0/3824034790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd040d3380 0x7fcd041f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 >> 172.21.15.99:0/3824034790 conn(0x7fcd0408afd0 msgr2=0x7fcd040b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 shutdown_connections 2024-03-29T09:12:02.057 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.056+0000 7fcd0ac00700 1 -- 172.21.15.99:0/3824034790 wait complete. 2024-03-29T09:12:02.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:12:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.265+0000 7f004f200700 1 Processor -- start 2024-03-29T09:12:02.265 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.265+0000 7f004f200700 1 -- start start 2024-03-29T09:12:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.266+0000 7f004f200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 0x7f00480d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.266+0000 7f004f200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00480cddc0 con 0x7f00480d1b20 2024-03-29T09:12:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.266+0000 7f004f200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00480ce0a0 con 0x7f0048160cc0 2024-03-29T09:12:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.266+0000 7f004f200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00480ce380 con 0x7f00481608e0 2024-03-29T09:12:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f004d200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f00480d1b20 0x7f00480afd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36206/0 (socket says 172.21.15.99:36206) 2024-03-29T09:12:02.266 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f004d200700 1 -- 172.21.15.99:0/4477097 learned_addr learned my addr 172.21.15.99:0/4477097 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0047e00700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 0x7f00480d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025803257 0 0) 0x7f00480cddc0 con 0x7f00480d1b20 2024-03-29T09:12:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f002c0035b0 con 0x7f00480d1b20 2024-03-29T09:12:02.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895400500 0 0) 0x7f00480ce0a0 con 0x7f0048160cc0 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00480cddc0 con 0x7f0048160cc0 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542721246 0 0) 0x7f002c0035b0 con 0x7f00480d1b20 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 msgr2=0x7f00480d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 0x7f00480d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 >> v1:172.21.15.133:6789/0 conn(0x7f0048160cc0 legacy=0x7f00480d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480ce660 con 0x7f00480d1b20 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f00400019f0 con 0x7f00480d1b20 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.267+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 >> v1:172.21.15.99:6789/0 conn(0x7f00480d1b20 legacy=0x7f00480afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0046a00700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c003bc0 0x7f002c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0046a00700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 0x7f002c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0046a00700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f002c004d90 0x7f002c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480ce380 con 0x7f002c004d90 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c00bac0 con 0x7f002c003bc0 2024-03-29T09:12:02.270 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c00bc30 con 0x7f00481608e0 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f004d200700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f002c004d90 0x7f002c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0047e00700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c003bc0 0x7f002c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f0047400700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 0x7f002c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f004d200700 1 -- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 msgr2=0x7f002c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f004d200700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00481608e0 0x7f002c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f004d200700 1 -- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c003bc0 msgr2=0x7f002c003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f004d200700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c003bc0 0x7f002c003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f004d200700 1 -- 172.21.15.99:0/4477097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0035b0 con 0x7f002c004d90 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.268+0000 7f004d200700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f002c004d90 0x7f002c005210 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f004808c5e0 tx=0x7f002c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e138c2932622e30 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400072d0 con 0x7f002c004d90 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040007450 con 0x7f002c004d90 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f0046a00700 1 -- 172.21.15.99:0/4477097 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040007800 con 0x7f002c004d90 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f004f200700 1 -- 172.21.15.99:0/4477097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f002c004d90 msgr2=0x7f002c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.271 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f004f200700 1 --2- 172.21.15.99:0/4477097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f002c004d90 0x7f002c005210 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f004808c5e0 tx=0x7f002c005880 comp rx=0 tx=0).stop 2024-03-29T09:12:02.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f004d200700 1 -- 172.21.15.99:0/4477097 reap_dead start 2024-03-29T09:12:02.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f004f200700 1 -- 172.21.15.99:0/4477097 shutdown_connections 2024-03-29T09:12:02.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f004f200700 1 -- 172.21.15.99:0/4477097 >> 172.21.15.99:0/4477097 conn(0x7f004809c490 msgr2=0x7f00480d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:02.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f004f200700 1 -- 172.21.15.99:0/4477097 shutdown_connections 2024-03-29T09:12:02.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.269+0000 7f004f200700 1 -- 172.21.15.99:0/4477097 wait complete. 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.271+0000 7f004f200700 1 Processor -- start 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.271+0000 7f004f200700 1 -- start start 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f004f200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c004d90 0x7f004815c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f004f200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00480ca7e0 0x7f00480cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f004f200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0048145f60 0x7f0048147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f004f200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480cd660 con 0x7f00480ca7e0 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f004f200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00480cd380 con 0x7f002c004d90 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f004f200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0048157500 con 0x7f0048145f60 2024-03-29T09:12:02.274 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f0047400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00480ca7e0 0x7f00480cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f0047400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00480ca7e0 0x7f00480cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38402/0 (socket says 172.21.15.99:38402) 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f0047400700 1 -- 172.21.15.99:0/1118271022 learned_addr learned my addr 172.21.15.99:0/1118271022 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f0047e00700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c004d90 0x7f004815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.272+0000 7f004d200700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0048145f60 0x7f0048147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f0047400700 1 -- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0048145f60 msgr2=0x7f0048147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f0047400700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0048145f60 0x7f0048147fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f0047400700 1 -- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c004d90 msgr2=0x7f004815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f0047400700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c004d90 0x7f004815c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f0047400700 1 -- 172.21.15.99:0/1118271022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00481577e0 con 0x7f00480ca7e0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f0047400700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00480ca7e0 0x7f00480cca10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f003400e7b0 tx=0x7f003400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f003fe00700 1 -- 172.21.15.99:0/1118271022 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00340052e0 con 0x7f00480ca7e0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f003fe00700 1 -- 172.21.15.99:0/1118271022 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0034005440 con 0x7f00480ca7e0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0048157ac0 con 0x7f00480ca7e0 2024-03-29T09:12:02.275 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f003fe00700 1 -- 172.21.15.99:0/1118271022 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00340038c0 con 0x7f00480ca7e0 2024-03-29T09:12:02.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.273+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00480ccf50 con 0x7f00480ca7e0 2024-03-29T09:12:02.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.274+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0048157d00 con 0x7f00480ca7e0 2024-03-29T09:12:02.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.275+0000 7f003fe00700 1 -- 172.21.15.99:0/1118271022 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0034003a20 con 0x7f00480ca7e0 2024-03-29T09:12:02.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.276+0000 7f003fe00700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f002806ac90 0x7f002806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.276+0000 7f003fe00700 1 -- 172.21.15.99:0/1118271022 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0034088d00 con 0x7f00480ca7e0 2024-03-29T09:12:02.276 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.277+0000 7f0047e00700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f002806ac90 0x7f002806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.277 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.277+0000 7f0047e00700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f002806ac90 0x7f002806d150 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0038009c60 tx=0x7f0038009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:02.280 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.281+0000 7f003fe00700 1 -- 172.21.15.99:0/1118271022 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0048157d00 con 0x7f00480ca7e0 2024-03-29T09:12:02.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.498+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0048157d00 con 0x7f00480ca7e0 2024-03-29T09:12:02.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.498+0000 7f003fe00700 1 -- 172.21.15.99:0/1118271022 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0048157d00 con 0x7f00480ca7e0 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f002806ac90 msgr2=0x7f002806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f002806ac90 0x7f002806d150 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0038009c60 tx=0x7f0038009210 comp rx=0 tx=0).stop 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00480ca7e0 msgr2=0x7f00480cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00480ca7e0 0x7f00480cca10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f003400e7b0 tx=0x7f003400eac0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 shutdown_connections 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f002806ac90 0x7f002806d150 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0048145f60 0x7f0048147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f002c004d90 0x7f004815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 --2- 172.21.15.99:0/1118271022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f00480ca7e0 0x7f00480cca10 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 >> 172.21.15.99:0/1118271022 conn(0x7f004809c490 msgr2=0x7f00480b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:02.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 shutdown_connections 2024-03-29T09:12:02.502 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.501+0000 7f004f200700 1 -- 172.21.15.99:0/1118271022 wait complete. 2024-03-29T09:12:02.502 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:12:02.520 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:12:02.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:02.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:02.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[66366]: fuse finished with error 0 and tester_r 0 2024-03-29T09:12:02.585 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:12:02.669 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:12:02.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:02.669 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:12:02.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:12:02.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.940+0000 7f9080c00700 1 Processor -- start 2024-03-29T09:12:02.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.940+0000 7f9080c00700 1 -- start start 2024-03-29T09:12:02.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.941+0000 7f9080c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b1260 0x7f907c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.941+0000 7f9080c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f907c0b36f0 0x7f907c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.941+0000 7f9080c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907c15c220 con 0x7f907c15ae70 2024-03-29T09:12:02.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.941+0000 7f9080c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907c0b40c0 con 0x7f907c0b36f0 2024-03-29T09:12:02.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.941+0000 7f9080c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c15b8f0 con 0x7f907c0b1260 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f9079e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f907c15ae70 0x7f907c0b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36218/0 (socket says 172.21.15.99:36218) 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f907a800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f907c0b36f0 0x7f907c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f9079e00700 1 -- 172.21.15.99:0/3806653688 learned_addr learned my addr 172.21.15.99:0/3806653688 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f9079400700 1 --2- 172.21.15.99:0/3806653688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b1260 0x7f907c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f9078a00700 1 -- 172.21.15.99:0/3806653688 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646724560 0 0) 0x7f907c15c220 con 0x7f907c15ae70 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f9078a00700 1 -- 172.21.15.99:0/3806653688 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90600035b0 con 0x7f907c15ae70 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f907a800700 1 -- 172.21.15.99:0/3806653688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b1260 msgr2=0x7f907c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f907a800700 1 --2- 172.21.15.99:0/3806653688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b1260 0x7f907c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f907a800700 1 -- 172.21.15.99:0/3806653688 >> v1:172.21.15.99:6789/0 conn(0x7f907c15ae70 legacy=0x7f907c0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f907a800700 1 -- 172.21.15.99:0/3806653688 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c15bbd0 con 0x7f907c0b36f0 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f9079400700 1 --2- 172.21.15.99:0/3806653688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b1260 0x7f907c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.942+0000 7f907a800700 1 --2- 172.21.15.99:0/3806653688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c0b36f0 0x7f907c0b3b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9074009d90 tx=0x7f907400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf35659398837b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9078a00700 1 -- 172.21.15.99:0/3806653688 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907400e070 con 0x7f907c0b36f0 2024-03-29T09:12:02.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9078a00700 1 -- 172.21.15.99:0/3806653688 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907400bb20 con 0x7f907c0b36f0 2024-03-29T09:12:02.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9078a00700 1 -- 172.21.15.99:0/3806653688 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074012430 con 0x7f907c0b36f0 2024-03-29T09:12:02.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 -- 172.21.15.99:0/3806653688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c0b36f0 msgr2=0x7f907c0b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 --2- 172.21.15.99:0/3806653688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c0b36f0 0x7f907c0b3b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9074009d90 tx=0x7f907400b330 comp rx=0 tx=0).stop 2024-03-29T09:12:02.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 -- 172.21.15.99:0/3806653688 shutdown_connections 2024-03-29T09:12:02.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 --2- 172.21.15.99:0/3806653688 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c0b36f0 0x7f907c0b3b80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 --2- 172.21.15.99:0/3806653688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b1260 0x7f907c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 -- 172.21.15.99:0/3806653688 >> 172.21.15.99:0/3806653688 conn(0x7f907c066540 msgr2=0x7f907c066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:02.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 -- 172.21.15.99:0/3806653688 shutdown_connections 2024-03-29T09:12:02.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.943+0000 7f9080c00700 1 -- 172.21.15.99:0/3806653688 wait complete. 2024-03-29T09:12:02.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.946+0000 7f9080c00700 1 Processor -- start 2024-03-29T09:12:02.945 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.946+0000 7f9080c00700 1 -- start start 2024-03-29T09:12:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9080c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f907c0b1260 0x7f907c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9080c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b36f0 0x7f907c13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9080c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c15ae70 0x7f907c141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9080c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c14dc80 con 0x7f907c15ae70 2024-03-29T09:12:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9080c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c152390 con 0x7f907c0b1260 2024-03-29T09:12:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9080c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c152670 con 0x7f907c0b36f0 2024-03-29T09:12:02.948 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f907a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c15ae70 0x7f907c141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f907a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c15ae70 0x7f907c141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38426/0 (socket says 172.21.15.99:38426) 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f907a800700 1 -- 172.21.15.99:0/2446704418 learned_addr learned my addr 172.21.15.99:0/2446704418 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9079e00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f907c0b1260 0x7f907c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.947+0000 7f9079400700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b36f0 0x7f907c13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f907a800700 1 -- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b36f0 msgr2=0x7f907c13eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f907a800700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b36f0 0x7f907c13eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f907a800700 1 -- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f907c0b1260 msgr2=0x7f907c071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f907a800700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f907c0b1260 0x7f907c071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f907a800700 1 -- 172.21.15.99:0/2446704418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9074009a70 con 0x7f907c15ae70 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f9079400700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b36f0 0x7f907c13eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f9079e00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f907c0b1260 0x7f907c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f907a800700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c15ae70 0x7f907c141960 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9074014040 tx=0x7f9074004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f9072000700 1 -- 172.21.15.99:0/2446704418 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074012430 con 0x7f907c15ae70 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f9072000700 1 -- 172.21.15.99:0/2446704418 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074020070 con 0x7f907c15ae70 2024-03-29T09:12:02.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f9072000700 1 -- 172.21.15.99:0/2446704418 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074003ea0 con 0x7f907c15ae70 2024-03-29T09:12:02.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c070f50 con 0x7f907c15ae70 2024-03-29T09:12:02.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.948+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c071510 con 0x7f907c15ae70 2024-03-29T09:12:02.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.949+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c0677a0 con 0x7f907c15ae70 2024-03-29T09:12:02.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.950+0000 7f9072000700 1 -- 172.21.15.99:0/2446704418 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9074004280 con 0x7f907c15ae70 2024-03-29T09:12:02.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.951+0000 7f9072000700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f905c06acc0 0x7f905c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:02.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.951+0000 7f9072000700 1 -- 172.21.15.99:0/2446704418 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9074022070 con 0x7f907c15ae70 2024-03-29T09:12:02.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.951+0000 7f9079e00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f905c06acc0 0x7f905c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:02.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.952+0000 7f9079e00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f905c06acc0 0x7f905c06d180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f906c0011e0 tx=0x7f906c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:02.955 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:02.956+0000 7f9072000700 1 -- 172.21.15.99:0/2446704418 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f90740be580 con 0x7f907c15ae70 2024-03-29T09:12:03.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.205+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f907c0b1a90 con 0x7f907c15ae70 2024-03-29T09:12:03.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.205+0000 7f9072000700 1 -- 172.21.15.99:0/2446704418 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f907402c060 con 0x7f907c15ae70 2024-03-29T09:12:03.205 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T09:12:03.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f905c06acc0 msgr2=0x7f905c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:03.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f905c06acc0 0x7f905c06d180 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f906c0011e0 tx=0x7f906c009040 comp rx=0 tx=0).stop 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c15ae70 msgr2=0x7f907c141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c15ae70 0x7f907c141960 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9074014040 tx=0x7f9074004ad0 comp rx=0 tx=0).stop 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 shutdown_connections 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f905c06acc0 0x7f905c06d180 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f907c15ae70 0x7f907c141960 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f907c0b1260 0x7f907c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 --2- 172.21.15.99:0/2446704418 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f907c0b36f0 0x7f907c13eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.207+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 >> 172.21.15.99:0/2446704418 conn(0x7f907c066540 msgr2=0x7f907c15e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.208+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 shutdown_connections 2024-03-29T09:12:03.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:03.208+0000 7f9080c00700 1 -- 172.21.15.99:0/2446704418 wait complete. 2024-03-29T09:12:03.225 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:12:03.226 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:12:03.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:03.226 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:12:03.271 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42065sec preferred_lft 42065sec 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:12:03.272 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::3caa:acff:feac:fccd/64 scope link 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5832:eeff:fe0b:ee92/64 scope link 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether fe:d0:b6:f6:cb:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::fcd0:b6ff:fef6:cba2/64 scope link 2024-03-29T09:12:03.273 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:03.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:03.274 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:12:03.373 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:12:03.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:03.377 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:12:03.421 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:12:03.421 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:12:03.421 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:12:03.421 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:12:03.421 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:12:03.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:03.422 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:12:03.422 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:12:03.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:12:03.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:12:03.422 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:12:03.422 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:12:03.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:03.622 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:12:03.664 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:12:03.664 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T09:12:03.665 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:12:03.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:03.665 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:12:03.770 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T09:12:03.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:12:03.771 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:03.812 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:12:03.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:12:03.812 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:03.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:12:03.999 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:12:03.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:12:04.062 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:12:04.064 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:12:04.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:04.065 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:12:04.106 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:12:04.106 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T09:12:04.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:12:04.215 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:12:04.215 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:12:04.281 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:12:04.284 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:12:04.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:04.284 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:12:04.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-29T09:12:04.315+0000 7ff135dfc5c0 -1 init, newargv = 0x55ae677b1500 newargc=15 2024-03-29T09:12:04.315 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[67469]: starting ceph client 2024-03-29T09:12:04.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[67469]: starting fuse 2024-03-29T09:12:04.329 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:12:04.330 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:12:04.330 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:12:04.331 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:04.332 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-29T09:12:04.332 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:12:04.506 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.67469.asok 2024-03-29T09:12:04.685 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67469.asok 2024-03-29T09:12:04.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:04.685 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67469.asok status 2024-03-29T09:12:04.904 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:12:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:12:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:12:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "67469", 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6381, 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6381 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1035261534 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 1035261534 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6381 192.168.0.1:0/1035261534", 2024-03-29T09:12:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/1035261534", 2024-03-29T09:12:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:12:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 25, 2024-03-29T09:12:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 55, 2024-03-29T09:12:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:12:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:12:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:12:04.907 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:04.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:04.919 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:12:04.919 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:04.960 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:12:04.961 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:04.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:04.961 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> import errno 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> if False: 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> ret = os.read(fd, 1024) 2024-03-29T09:12:05.065 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-03-29T09:12:05.066 DEBUG:teuthology.orchestra.run.smithi099:> os.write(fd, b'"'"'content'"'"') 2024-03-29T09:12:05.066 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2024-03-29T09:12:05.066 DEBUG:teuthology.orchestra.run.smithi099:> if e.errno != errno.EPERM: 2024-03-29T09:12:05.066 DEBUG:teuthology.orchestra.run.smithi099:> raise 2024-03-29T09:12:05.066 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-03-29T09:12:05.066 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("client does not check permission of data pool") 2024-03-29T09:12:05.066 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:12:05.357 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2024-03-29T09:12:05.594 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.595+0000 7f8d91000700 1 Processor -- start 2024-03-29T09:12:05.595 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.595+0000 7f8d91000700 1 -- start start 2024-03-29T09:12:05.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d91000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d8c0d35c0 0x7f8d8c0d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:05.597 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d91000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0cddb0 0x7f8d8c0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d91000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c0b3e00 con 0x7f8d8c15d420 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d91000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c0d12b0 con 0x7f8d8c0d35c0 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d91000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0b31f0 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d83e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d8c0d35c0 0x7f8d8c0d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d83e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f8d8c0d35c0 0x7f8d8c0d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38430/0 (socket says 172.21.15.99:38430) 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d83e00700 1 -- 172.21.15.99:0/2420156714 learned_addr learned my addr 172.21.15.99:0/2420156714 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d89e00700 1 --1- 172.21.15.99:0/2420156714 >> v1:172.21.15.99:6789/0 conn(0x7f8d8c15d420 0x7f8d8c0d2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36224/0 (socket says 172.21.15.99:36224) 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.596+0000 7f8d8a800700 1 --2- 172.21.15.99:0/2420156714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0cddb0 0x7f8d8c0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d83e00700 1 -- 172.21.15.99:0/2420156714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0cddb0 msgr2=0x7f8d8c0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d83e00700 1 --2- 172.21.15.99:0/2420156714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0cddb0 0x7f8d8c0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d83e00700 1 -- 172.21.15.99:0/2420156714 >> v1:172.21.15.99:6789/0 conn(0x7f8d8c15d420 legacy=0x7f8d8c0d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:05.598 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d83e00700 1 -- 172.21.15.99:0/2420156714 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0b34d0 con 0x7f8d8c0d35c0 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d89400700 1 -- 172.21.15.99:0/2420156714 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127471122 0 0) 0x7f8d8c0b3e00 con 0x7f8d8c15d420 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d83e00700 1 --2- 172.21.15.99:0/2420156714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0d35c0 0x7f8d8c0d0d70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d740098f0 tx=0x7f8d74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59178aabccc57bb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d89400700 1 -- 172.21.15.99:0/2420156714 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400b8a0 con 0x7f8d8c0d35c0 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d89400700 1 -- 172.21.15.99:0/2420156714 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7400ba00 con 0x7f8d8c0d35c0 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.597+0000 7f8d89400700 1 -- 172.21.15.99:0/2420156714 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400bd90 con 0x7f8d8c0d35c0 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 -- 172.21.15.99:0/2420156714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0d35c0 msgr2=0x7f8d8c0d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 --2- 172.21.15.99:0/2420156714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0d35c0 0x7f8d8c0d0d70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8d740098f0 tx=0x7f8d74009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 -- 172.21.15.99:0/2420156714 shutdown_connections 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 --2- 172.21.15.99:0/2420156714 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0cddb0 0x7f8d8c0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 --2- 172.21.15.99:0/2420156714 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0d35c0 0x7f8d8c0d0d70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 -- 172.21.15.99:0/2420156714 >> 172.21.15.99:0/2420156714 conn(0x7f8d8c08afd0 msgr2=0x7f8d8c09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 -- 172.21.15.99:0/2420156714 shutdown_connections 2024-03-29T09:12:05.599 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.598+0000 7f8d91000700 1 -- 172.21.15.99:0/2420156714 wait complete. 2024-03-29T09:12:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.599+0000 7f8d91000700 1 Processor -- start 2024-03-29T09:12:05.601 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.600+0000 7f8d91000700 1 -- start start 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.600+0000 7f8d91000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0cddb0 0x7f8d8c1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.600+0000 7f8d91000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0d35c0 0x7f8d8c1e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.600+0000 7f8d91000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d8c15d420 0x7f8d8c1e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.600+0000 7f8d91000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c14c890 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.600+0000 7f8d91000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c150290 con 0x7f8d8c15d420 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.600+0000 7f8d91000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c14bf00 con 0x7f8d8c0d35c0 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0cddb0 0x7f8d8c1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0cddb0 0x7f8d8c1f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38438/0 (socket says 172.21.15.99:38438) 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 -- 172.21.15.99:0/632693687 learned_addr learned my addr 172.21.15.99:0/632693687 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d83e00700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0d35c0 0x7f8d8c1e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 -- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0d35c0 msgr2=0x7f8d8c1e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0d35c0 0x7f8d8c1e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 -- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d8c15d420 msgr2=0x7f8d8c1e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:05.602 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d8a800700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d8c15d420 0x7f8d8c1e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d8c15d420 0x7f8d8c1e76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 -- 172.21.15.99:0/632693687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74009550 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d8a800700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d8c15d420 0x7f8d8c1e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d89e00700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0cddb0 0x7f8d8c1f6a80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c550 tx=0x7f8d7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.601+0000 7f8d82000700 1 -- 172.21.15.99:0/632693687 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c004c80 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.602+0000 7f8d82000700 1 -- 172.21.15.99:0/632693687 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c004e00 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.602+0000 7f8d82000700 1 -- 172.21.15.99:0/632693687 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c007310 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.602+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c0c9980 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.602+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c14c4b0 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.603 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.603+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c0678b0 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.604+0000 7f8d82000700 1 -- 172.21.15.99:0/632693687 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8d7c007470 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.604+0000 7f8d82000700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d6806ac90 0x7f8d6806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:05.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.604+0000 7f8d82000700 1 -- 172.21.15.99:0/632693687 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8d7c089110 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.604 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.605+0000 7f8d83e00700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d6806ac90 0x7f8d6806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:05.605 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.605+0000 7f8d83e00700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d6806ac90 0x7f8d6806d150 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d74008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:05.608 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.609+0000 7f8d82000700 1 -- 172.21.15.99:0/632693687 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8d7c054240 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.892+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7f8d8c0d01a0 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.895 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.896+0000 7f8d82000700 1 -- 172.21.15.99:0/632693687 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v21) v1 ==== 183+0+0 (secure 0 0 0) 0x7f8d7c057860 con 0x7f8d8c0cddb0 2024-03-29T09:12:05.899 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d6806ac90 msgr2=0x7f8d6806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:05.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d6806ac90 0x7f8d6806d150 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d74008030 comp rx=0 tx=0).stop 2024-03-29T09:12:05.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0cddb0 msgr2=0x7f8d8c1f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:05.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0cddb0 0x7f8d8c1f6a80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c550 tx=0x7f8d7c00c910 comp rx=0 tx=0).stop 2024-03-29T09:12:05.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 shutdown_connections 2024-03-29T09:12:05.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8d6806ac90 0x7f8d6806d150 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d8c15d420 0x7f8d8c1e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d8c0cddb0 0x7f8d8c1f6a80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 --2- 172.21.15.99:0/632693687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d8c0d35c0 0x7f8d8c1e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:05.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.898+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 >> 172.21.15.99:0/632693687 conn(0x7f8d8c08afd0 msgr2=0x7f8d8c15c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:05.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.899+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 shutdown_connections 2024-03-29T09:12:05.902 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:05.899+0000 7f8d91000700 1 -- 172.21.15.99:0/632693687 wait complete. 2024-03-29T09:12:05.902 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:12:05.917 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:12:05.917 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:05.959 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.188+0000 7f664de00700 1 Processor -- start 2024-03-29T09:12:06.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.188+0000 7f664de00700 1 -- start start 2024-03-29T09:12:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.189+0000 7f664de00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66481608e0 0x7f66480d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.189+0000 7f664de00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66480d1b20 0x7f66480ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.189+0000 7f664de00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66480cddc0 con 0x7f6648160cc0 2024-03-29T09:12:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f664de00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66480ce0a0 con 0x7f66480d1b20 2024-03-29T09:12:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f664de00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66480ce380 con 0x7f66481608e0 2024-03-29T09:12:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6647400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66480d1b20 0x7f66480ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6647400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f66480d1b20 0x7f66480ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38442/0 (socket says 172.21.15.99:38442) 2024-03-29T09:12:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6646000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f6648160cc0 0x7f66480d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36228/0 (socket says 172.21.15.99:36228) 2024-03-29T09:12:06.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6647400700 1 -- 172.21.15.99:0/1036552718 learned_addr learned my addr 172.21.15.99:0/1036552718 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6646a00700 1 --2- 172.21.15.99:0/1036552718 >> v2:172.21.15.133:3300/0 conn(0x7f66481608e0 0x7f66480d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6647400700 1 -- 172.21.15.99:0/1036552718 >> v2:172.21.15.133:3300/0 conn(0x7f66481608e0 msgr2=0x7f66480d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6647400700 1 --2- 172.21.15.99:0/1036552718 >> v2:172.21.15.133:3300/0 conn(0x7f66481608e0 0x7f66480d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6647400700 1 -- 172.21.15.99:0/1036552718 >> v1:172.21.15.99:6789/0 conn(0x7f6648160cc0 legacy=0x7f66480d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6647400700 1 -- 172.21.15.99:0/1036552718 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480ce660 con 0x7f66480d1b20 2024-03-29T09:12:06.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6646a00700 1 --2- 172.21.15.99:0/1036552718 >> v2:172.21.15.133:3300/0 conn(0x7f66481608e0 0x7f66480d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.190+0000 7f6645600700 1 -- 172.21.15.99:0/1036552718 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995614164 0 0) 0x7f66480cddc0 con 0x7f6648160cc0 2024-03-29T09:12:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f6647400700 1 --2- 172.21.15.99:0/1036552718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ceb80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f664808d790 tx=0x7f664000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4df044f67aeb433 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f6645600700 1 -- 172.21.15.99:0/1036552718 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000e070 con 0x7f66480d1b20 2024-03-29T09:12:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f6645600700 1 -- 172.21.15.99:0/1036552718 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664000b9c0 con 0x7f66480d1b20 2024-03-29T09:12:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f6645600700 1 -- 172.21.15.99:0/1036552718 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000bd70 con 0x7f66480d1b20 2024-03-29T09:12:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 -- 172.21.15.99:0/1036552718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 msgr2=0x7f66480ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 --2- 172.21.15.99:0/1036552718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ceb80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f664808d790 tx=0x7f664000b330 comp rx=0 tx=0).stop 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 -- 172.21.15.99:0/1036552718 shutdown_connections 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 --2- 172.21.15.99:0/1036552718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ceb80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 --2- 172.21.15.99:0/1036552718 >> v2:172.21.15.133:3300/0 conn(0x7f66481608e0 0x7f66480d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 -- 172.21.15.99:0/1036552718 >> 172.21.15.99:0/1036552718 conn(0x7f664809c490 msgr2=0x7f66480d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 -- 172.21.15.99:0/1036552718 shutdown_connections 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.191+0000 7f664de00700 1 -- 172.21.15.99:0/1036552718 wait complete. 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.193+0000 7f664de00700 1 Processor -- start 2024-03-29T09:12:06.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.193+0000 7f664de00700 1 -- start start 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.194+0000 7f664de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.194+0000 7f664de00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66481608e0 0x7f66480b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.194+0000 7f664de00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6648160cc0 0x7f66481547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.194+0000 7f664de00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648158820 con 0x7f66480d1b20 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.194+0000 7f664de00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648158260 con 0x7f6648160cc0 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.194+0000 7f664de00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6648158540 con 0x7f66481608e0 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.194+0000 7f6646a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38454/0 (socket says 172.21.15.99:38454) 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 -- 172.21.15.99:0/407402244 learned_addr learned my addr 172.21.15.99:0/407402244 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 -- 172.21.15.99:0/407402244 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66481608e0 msgr2=0x7f66480b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66481608e0 0x7f66480b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 -- 172.21.15.99:0/407402244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6648160cc0 msgr2=0x7f66481547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6648160cc0 0x7f66481547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 -- 172.21.15.99:0/407402244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6640009b30 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f6646a00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ccf30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f663800e810 tx=0x7f663800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f663f400700 1 -- 172.21.15.99:0/407402244 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638013610 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f663f400700 1 -- 172.21.15.99:0/407402244 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6638013770 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6648153fd0 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f663f400700 1 -- 172.21.15.99:0/407402244 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6638013b20 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.195+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66480cd470 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.196+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66480677a0 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.197+0000 7f663f400700 1 -- 172.21.15.99:0/407402244 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f6638013c80 con 0x7f66480d1b20 2024-03-29T09:12:06.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.198+0000 7f663f400700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f662806ac90 0x7f662806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.198+0000 7f663f400700 1 -- 172.21.15.99:0/407402244 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f66380891c0 con 0x7f66480d1b20 2024-03-29T09:12:06.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.198+0000 7f6646000700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f662806ac90 0x7f662806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.199+0000 7f6646000700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f662806ac90 0x7f662806d150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f66480d2c50 tx=0x7f6634007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:06.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.202+0000 7f663f400700 1 -- 172.21.15.99:0/407402244 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f66380541c0 con 0x7f66480d1b20 2024-03-29T09:12:06.405 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.405+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66480d0700 con 0x7f66480d1b20 2024-03-29T09:12:06.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.406+0000 7f663f400700 1 -- 172.21.15.99:0/407402244 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f66380577e0 con 0x7f66480d1b20 2024-03-29T09:12:06.406 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:06.407 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:06.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f662806ac90 msgr2=0x7f662806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f662806ac90 0x7f662806d150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f66480d2c50 tx=0x7f6634007a10 comp rx=0 tx=0).stop 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 msgr2=0x7f66480ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ccf30 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f663800e810 tx=0x7f663800ebd0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 shutdown_connections 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f662806ac90 0x7f662806d150 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6648160cc0 0x7f66481547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66480d1b20 0x7f66480ccf30 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 --2- 172.21.15.99:0/407402244 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66481608e0 0x7f66480b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 >> 172.21.15.99:0/407402244 conn(0x7f664809c490 msgr2=0x7f664815e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 shutdown_connections 2024-03-29T09:12:06.410 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.409+0000 7f664de00700 1 -- 172.21.15.99:0/407402244 wait complete. 2024-03-29T09:12:06.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:11:41.952804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:11:42.633561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:12:06.430 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:12:06.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.624+0000 7fba2a000700 1 Processor -- start 2024-03-29T09:12:06.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.624+0000 7fba2a000700 1 -- start start 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba2a000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba24160770 0x7fba240b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba2a000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fba240d0d60 0x7fba240d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba2a000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba240d3380 0x7fba240cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba2a000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba240d3810 con 0x7fba240d3380 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba2a000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba240d1520 con 0x7fba240d0d60 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba2a000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba240d1800 con 0x7fba24160770 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba23e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba240d3380 0x7fba240cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba23e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fba240d3380 0x7fba240cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38468/0 (socket says 172.21.15.99:38468) 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba23e00700 1 -- 172.21.15.99:0/981286656 learned_addr learned my addr 172.21.15.99:0/981286656 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:06.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba23400700 1 --2- 172.21.15.99:0/981286656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba24160770 0x7fba240b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.625+0000 7fba22a00700 1 --2- 172.21.15.99:0/981286656 >> v2:172.21.15.133:3300/0 conn(0x7fba240d0d60 0x7fba240d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba23e00700 1 -- 172.21.15.99:0/981286656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba24160770 msgr2=0x7fba240b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba23e00700 1 --2- 172.21.15.99:0/981286656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba24160770 0x7fba240b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba23e00700 1 -- 172.21.15.99:0/981286656 >> v2:172.21.15.133:3300/0 conn(0x7fba240d0d60 msgr2=0x7fba240d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba23e00700 1 --2- 172.21.15.99:0/981286656 >> v2:172.21.15.133:3300/0 conn(0x7fba240d0d60 0x7fba240d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba23e00700 1 -- 172.21.15.99:0/981286656 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba240d1ae0 con 0x7fba240d3380 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba22a00700 1 --2- 172.21.15.99:0/981286656 >> v2:172.21.15.133:3300/0 conn(0x7fba240d0d60 0x7fba240d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba23e00700 1 --2- 172.21.15.99:0/981286656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba240d3380 0x7fba240cfa80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fba240b4be0 tx=0x7fba1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440fd259c537e6c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba22000700 1 -- 172.21.15.99:0/981286656 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00e070 con 0x7fba240d3380 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba22000700 1 -- 172.21.15.99:0/981286656 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c00b910 con 0x7fba240d3380 2024-03-29T09:12:06.627 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.626+0000 7fba22000700 1 -- 172.21.15.99:0/981286656 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00bcc0 con 0x7fba240d3380 2024-03-29T09:12:06.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 -- 172.21.15.99:0/981286656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba240d3380 msgr2=0x7fba240cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 --2- 172.21.15.99:0/981286656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba240d3380 0x7fba240cfa80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fba240b4be0 tx=0x7fba1c00b330 comp rx=0 tx=0).stop 2024-03-29T09:12:06.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 -- 172.21.15.99:0/981286656 shutdown_connections 2024-03-29T09:12:06.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 --2- 172.21.15.99:0/981286656 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba24160770 0x7fba240b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 --2- 172.21.15.99:0/981286656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba240d3380 0x7fba240cfa80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 --2- 172.21.15.99:0/981286656 >> v2:172.21.15.133:3300/0 conn(0x7fba240d0d60 0x7fba240d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 -- 172.21.15.99:0/981286656 >> 172.21.15.99:0/981286656 conn(0x7fba2408afd0 msgr2=0x7fba2409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:06.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 -- 172.21.15.99:0/981286656 shutdown_connections 2024-03-29T09:12:06.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.627+0000 7fba2a000700 1 -- 172.21.15.99:0/981286656 wait complete. 2024-03-29T09:12:06.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.629+0000 7fba2a000700 1 Processor -- start 2024-03-29T09:12:06.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.629+0000 7fba2a000700 1 -- start start 2024-03-29T09:12:06.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba2a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba240d0d60 0x7fba2409ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba2a000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba240d3380 0x7fba240ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba2a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba24160770 0x7fba2409d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba2a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2409f280 con 0x7fba24160770 2024-03-29T09:12:06.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba2a000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2414e020 con 0x7fba240d0d60 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba2a000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2414e300 con 0x7fba240d3380 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba24160770 0x7fba2409d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba24160770 0x7fba2409d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38474/0 (socket says 172.21.15.99:38474) 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 -- 172.21.15.99:0/106724682 learned_addr learned my addr 172.21.15.99:0/106724682 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23400700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba240d0d60 0x7fba2409ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba22a00700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba240d3380 0x7fba240ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 -- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba240d3380 msgr2=0x7fba240ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba240d3380 0x7fba240ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 -- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba240d0d60 msgr2=0x7fba2409ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba240d0d60 0x7fba2409ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23e00700 1 -- 172.21.15.99:0/106724682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c009b30 con 0x7fba24160770 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba23400700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba240d0d60 0x7fba2409ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.630+0000 7fba22a00700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba240d3380 0x7fba240ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.631+0000 7fba23e00700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba24160770 0x7fba2409d060 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fba240d0d10 tx=0x7fba1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:06.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.631+0000 7fba1b400700 1 -- 172.21.15.99:0/106724682 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0099b0 con 0x7fba24160770 2024-03-29T09:12:06.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.631+0000 7fba1b400700 1 -- 172.21.15.99:0/106724682 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c004250 con 0x7fba24160770 2024-03-29T09:12:06.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.631+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2414e5e0 con 0x7fba24160770 2024-03-29T09:12:06.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.631+0000 7fba1b400700 1 -- 172.21.15.99:0/106724682 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c004600 con 0x7fba24160770 2024-03-29T09:12:06.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.631+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2414eba0 con 0x7fba24160770 2024-03-29T09:12:06.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.632+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2414ed60 con 0x7fba24160770 2024-03-29T09:12:06.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.633+0000 7fba1b400700 1 -- 172.21.15.99:0/106724682 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fba1c004760 con 0x7fba24160770 2024-03-29T09:12:06.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.634+0000 7fba1b400700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fba0406ac90 0x7fba0406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:06.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.634+0000 7fba1b400700 1 -- 172.21.15.99:0/106724682 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fba1c08a8b0 con 0x7fba24160770 2024-03-29T09:12:06.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.634+0000 7fba23400700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fba0406ac90 0x7fba0406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:06.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.635+0000 7fba23400700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fba0406ac90 0x7fba0406d150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fba14009b10 tx=0x7fba14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:06.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.638+0000 7fba1b400700 1 -- 172.21.15.99:0/106724682 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fba1c0559e0 con 0x7fba24160770 2024-03-29T09:12:06.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.844+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba240b3410 con 0x7fba24160770 2024-03-29T09:12:06.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.845+0000 7fba1b400700 1 -- 172.21.15.99:0/106724682 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba1c02a0a0 con 0x7fba24160770 2024-03-29T09:12:06.845 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fba0406ac90 msgr2=0x7fba0406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fba0406ac90 0x7fba0406d150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fba14009b10 tx=0x7fba14008040 comp rx=0 tx=0).stop 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba24160770 msgr2=0x7fba2409d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba24160770 0x7fba2409d060 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fba240d0d10 tx=0x7fba1c004ad0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 shutdown_connections 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fba0406ac90 0x7fba0406d150 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fba24160770 0x7fba2409d060 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba240d0d60 0x7fba2409ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 --2- 172.21.15.99:0/106724682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fba240d3380 0x7fba240ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:06.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 >> 172.21.15.99:0/106724682 conn(0x7fba2408afd0 msgr2=0x7fba240b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:06.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 shutdown_connections 2024-03-29T09:12:06.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:06.847+0000 7fba2a000700 1 -- 172.21.15.99:0/106724682 wait complete. 2024-03-29T09:12:06.865 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:12:07.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.059+0000 7f7beac00700 1 Processor -- start 2024-03-29T09:12:07.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.059+0000 7f7beac00700 1 -- start start 2024-03-29T09:12:07.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.060+0000 7f7beac00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 0x7f7be40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.060+0000 7f7beac00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40d1520 con 0x7f7be40d3550 2024-03-29T09:12:07.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.060+0000 7f7beac00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40d1800 con 0x7f7be40d0d60 2024-03-29T09:12:07.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.060+0000 7f7beac00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40d1ae0 con 0x7f7be4160770 2024-03-29T09:12:07.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be8c00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7be40d3550 0x7f7be40ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:36230/0 (socket says 172.21.15.99:36230) 2024-03-29T09:12:07.061 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be8c00700 1 -- 172.21.15.99:0/1833514888 learned_addr learned my addr 172.21.15.99:0/1833514888 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108303757 0 0) 0x7f7be40d1520 con 0x7f7be40d3550 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be3e00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 0x7f7be40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc80035b0 con 0x7f7be40d3550 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2363655540 0 0) 0x7f7bc80035b0 con 0x7f7be40d3550 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 msgr2=0x7f7be40b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 0x7f7be40b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 >> v1:172.21.15.133:6789/0 conn(0x7f7be40d0d60 legacy=0x7f7be40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40d1dc0 con 0x7f7be40d3550 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f7bdc0019f0 con 0x7f7be40d3550 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 >> v1:172.21.15.99:6789/0 conn(0x7f7be40d3550 legacy=0x7f7be40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.061+0000 7f7be2a00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7bc8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be2a00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bc8004850 0x7f7bc8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be2a00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 0x7f7bc8005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40d1ae0 con 0x7f7bc8003a30 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc800bb40 con 0x7f7bc8004850 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc800bd00 con 0x7f7be4160770 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3e00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7bc8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be8c00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 0x7f7bc8005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3400700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bc8004850 0x7f7bc8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3e00700 1 -- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 msgr2=0x7f7bc8005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3e00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be4160770 0x7f7bc8005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3e00700 1 -- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bc8004850 msgr2=0x7f7bc8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3e00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bc8004850 0x7f7bc8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3e00700 1 -- 172.21.15.99:0/1833514888 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80035b0 con 0x7f7bc8003a30 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.062+0000 7f7be3e00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7bc8003e50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009b60 tx=0x7f7bc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac9eea3f37e3ab8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400e6c0 con 0x7f7bc8003a30 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd400e820 con 0x7f7bc8003a30 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7be2a00700 1 -- 172.21.15.99:0/1833514888 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400eb90 con 0x7f7bc8003a30 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7beac00700 1 -- 172.21.15.99:0/1833514888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 msgr2=0x7f7bc8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7beac00700 1 --2- 172.21.15.99:0/1833514888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7bc8003e50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009b60 tx=0x7f7bc8005880 comp rx=0 tx=0).stop 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7be8c00700 1 -- 172.21.15.99:0/1833514888 reap_dead start 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7beac00700 1 -- 172.21.15.99:0/1833514888 shutdown_connections 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7beac00700 1 -- 172.21.15.99:0/1833514888 >> 172.21.15.99:0/1833514888 conn(0x7f7be408afd0 msgr2=0x7f7be409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7beac00700 1 -- 172.21.15.99:0/1833514888 shutdown_connections 2024-03-29T09:12:07.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.063+0000 7f7beac00700 1 -- 172.21.15.99:0/1833514888 wait complete. 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.065+0000 7f7beac00700 1 Processor -- start 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.065+0000 7f7beac00700 1 -- start start 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7beac00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7be40cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7beac00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be40b2060 0x7f7be415a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7beac00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be40b0f10 0x7f7be40b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7beac00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40b1900 con 0x7f7bc8003a30 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7beac00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be415aab0 con 0x7f7be40b0f10 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7beac00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40cc870 con 0x7f7be40b2060 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7be3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7be40cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7be3e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7be40cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38498/0 (socket says 172.21.15.99:38498) 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.066+0000 7f7be3e00700 1 -- 172.21.15.99:0/923019180 learned_addr learned my addr 172.21.15.99:0/923019180 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be3400700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be40b2060 0x7f7be415a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be8c00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be40b0f10 0x7f7be40b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.068 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be3e00700 1 -- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be40b2060 msgr2=0x7f7be415a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be3e00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be40b2060 0x7f7be415a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be3e00700 1 -- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be40b0f10 msgr2=0x7f7be40b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be3e00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be40b0f10 0x7f7be40b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be3e00700 1 -- 172.21.15.99:0/923019180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40ccb50 con 0x7f7bc8003a30 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be8c00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be40b0f10 0x7f7be40b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7be3e00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7be40cac70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800bb10 tx=0x7f7bd40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7bdbe00700 1 -- 172.21.15.99:0/923019180 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400e6c0 con 0x7f7bc8003a30 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7bdbe00700 1 -- 172.21.15.99:0/923019180 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4003ab0 con 0x7f7bc8003a30 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be40cce30 con 0x7f7bc8003a30 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.067+0000 7f7bdbe00700 1 -- 172.21.15.99:0/923019180 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4003e60 con 0x7f7bc8003a30 2024-03-29T09:12:07.069 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.068+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be40cd3f0 con 0x7f7bc8003a30 2024-03-29T09:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.069+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be40d33c0 con 0x7f7bc8003a30 2024-03-29T09:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.069+0000 7f7bdbe00700 1 -- 172.21.15.99:0/923019180 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7bd4013470 con 0x7f7bc8003a30 2024-03-29T09:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.070+0000 7f7bdbe00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7bc406ac90 0x7f7bc406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.070+0000 7f7bdbe00700 1 -- 172.21.15.99:0/923019180 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7bd408ad40 con 0x7f7bc8003a30 2024-03-29T09:12:07.070 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.070+0000 7f7be3400700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7bc406ac90 0x7f7bc406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.071 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.071+0000 7f7be3400700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7bc406ac90 0x7f7bc406d150 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7be40d3160 tx=0x7f7bd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.074 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.075+0000 7f7bdbe00700 1 -- 172.21.15.99:0/923019180 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7bd4055dd0 con 0x7f7bc8003a30 2024-03-29T09:12:07.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.282+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7be40678b0 con 0x7f7bc8003a30 2024-03-29T09:12:07.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.283+0000 7f7bdbe00700 1 -- 172.21.15.99:0/923019180 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7bd4017020 con 0x7f7bc8003a30 2024-03-29T09:12:07.283 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:12:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7bc406ac90 msgr2=0x7f7bc406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7bc406ac90 0x7f7bc406d150 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7be40d3160 tx=0x7f7bd0009040 comp rx=0 tx=0).stop 2024-03-29T09:12:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 msgr2=0x7f7be40cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7be40cac70 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800bb10 tx=0x7f7bd40047c0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 shutdown_connections 2024-03-29T09:12:07.285 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7bc406ac90 0x7f7bc406d150 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be40b0f10 0x7f7be40b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7bc8003a30 0x7f7be40cac70 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 --2- 172.21.15.99:0/923019180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7be40b2060 0x7f7be415a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 >> 172.21.15.99:0/923019180 conn(0x7f7be408afd0 msgr2=0x7f7be40b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 shutdown_connections 2024-03-29T09:12:07.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.285+0000 7f7beac00700 1 -- 172.21.15.99:0/923019180 wait complete. 2024-03-29T09:12:07.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:12:07.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.496+0000 7ff9f5800700 1 Processor -- start 2024-03-29T09:12:07.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.496+0000 7ff9f5800700 1 -- start start 2024-03-29T09:12:07.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9f5800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff9f00b3cc0 0x7ff9f00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9f5800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9f00d3380 0x7ff9f00ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9f5800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f00d3810 con 0x7ff9f0160770 2024-03-29T09:12:07.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9f5800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f00b4140 con 0x7ff9f00d3380 2024-03-29T09:12:07.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9f5800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f00d1520 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9ef400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9f00d3380 0x7ff9f00ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.498 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9ef400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff9f00d3380 0x7ff9f00ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38514/0 (socket says 172.21.15.99:38514) 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.497+0000 7ff9ef400700 1 -- 172.21.15.99:0/3214844278 learned_addr learned my addr 172.21.15.99:0/3214844278 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ee000700 1 --2- 172.21.15.99:0/3214844278 >> v2:172.21.15.133:3300/0 conn(0x7ff9f00b3cc0 0x7ff9f00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ef400700 1 -- 172.21.15.99:0/3214844278 >> v2:172.21.15.133:3300/0 conn(0x7ff9f00b3cc0 msgr2=0x7ff9f00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ef400700 1 --2- 172.21.15.99:0/3214844278 >> v2:172.21.15.133:3300/0 conn(0x7ff9f00b3cc0 0x7ff9f00d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ef400700 1 -- 172.21.15.99:0/3214844278 >> v1:172.21.15.133:6789/0 conn(0x7ff9f0160770 legacy=0x7ff9f00d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ef400700 1 -- 172.21.15.99:0/3214844278 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00d1800 con 0x7ff9f00d3380 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ee000700 1 --2- 172.21.15.99:0/3214844278 >> v2:172.21.15.133:3300/0 conn(0x7ff9f00b3cc0 0x7ff9f00d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ef400700 1 --2- 172.21.15.99:0/3214844278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00d3380 0x7ff9f00ce080 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00b5cf0 tx=0x7ff9e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d9d82f389dee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ed600700 1 -- 172.21.15.99:0/3214844278 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800e070 con 0x7ff9f00d3380 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ed600700 1 -- 172.21.15.99:0/3214844278 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800b9c0 con 0x7ff9f00d3380 2024-03-29T09:12:07.499 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.498+0000 7ff9ed600700 1 -- 172.21.15.99:0/3214844278 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800bd70 con 0x7ff9f00d3380 2024-03-29T09:12:07.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 -- 172.21.15.99:0/3214844278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00d3380 msgr2=0x7ff9f00ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3214844278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00d3380 0x7ff9f00ce080 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00b5cf0 tx=0x7ff9e800b330 comp rx=0 tx=0).stop 2024-03-29T09:12:07.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 -- 172.21.15.99:0/3214844278 shutdown_connections 2024-03-29T09:12:07.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3214844278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00d3380 0x7ff9f00ce080 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3214844278 >> v2:172.21.15.133:3300/0 conn(0x7ff9f00b3cc0 0x7ff9f00d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 -- 172.21.15.99:0/3214844278 >> 172.21.15.99:0/3214844278 conn(0x7ff9f008afd0 msgr2=0x7ff9f009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:07.500 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 -- 172.21.15.99:0/3214844278 shutdown_connections 2024-03-29T09:12:07.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.499+0000 7ff9f5800700 1 -- 172.21.15.99:0/3214844278 wait complete. 2024-03-29T09:12:07.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.501+0000 7ff9f5800700 1 Processor -- start 2024-03-29T09:12:07.501 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.501+0000 7ff9f5800700 1 -- start start 2024-03-29T09:12:07.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9f5800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00b3cc0 0x7ff9f00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9f5800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f00d3380 0x7ff9f015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9f5800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f0160770 0x7ff9f00cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9f5800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00b78e0 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9f5800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f015ab50 con 0x7ff9f0160770 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9f5800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00b6c60 con 0x7ff9f00d3380 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00b3cc0 0x7ff9f00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00b3cc0 0x7ff9f00cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:38520/0 (socket says 172.21.15.99:38520) 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 -- 172.21.15.99:0/3669772941 learned_addr learned my addr 172.21.15.99:0/3669772941 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9ee000700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f00d3380 0x7ff9f015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9ef400700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f0160770 0x7ff9f00cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 -- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f00d3380 msgr2=0x7ff9f015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f00d3380 0x7ff9f015a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 -- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f0160770 msgr2=0x7ff9f00cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f0160770 0x7ff9f00cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.502+0000 7ff9eea00700 1 -- 172.21.15.99:0/3669772941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009b30 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.503+0000 7ff9eea00700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00b3cc0 0x7ff9f00cd020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c450 tx=0x7ff9e000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.503+0000 7ff9e6a00700 1 -- 172.21.15.99:0/3669772941 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00052d0 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.503+0000 7ff9e6a00700 1 -- 172.21.15.99:0/3669772941 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0005430 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.503+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f00b6f40 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.503+0000 7ff9e6a00700 1 -- 172.21.15.99:0/3669772941 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00038c0 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.503+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f00b7500 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.504+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f00678b0 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.505 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.505+0000 7ff9e6a00700 1 -- 172.21.15.99:0/3669772941 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff9e0003a20 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.506+0000 7ff9e6a00700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff9d006ac90 0x7ff9d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.506+0000 7ff9e6a00700 1 -- 172.21.15.99:0/3669772941 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff9e0088780 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.506+0000 7ff9ee000700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff9d006ac90 0x7ff9d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.506+0000 7ff9ee000700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff9d006ac90 0x7ff9d006d150 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00b4630 tx=0x7ff9dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.510+0000 7ff9e6a00700 1 -- 172.21.15.99:0/3669772941 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7ff9e0053820 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.714 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.714+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9f00b76c0 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.715 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.715+0000 7ff9e6a00700 1 -- 172.21.15.99:0/3669772941 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9e000ce60 con 0x7ff9f00b3cc0 2024-03-29T09:12:07.715 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.717+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff9d006ac90 msgr2=0x7ff9d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.717+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff9d006ac90 0x7ff9d006d150 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00b4630 tx=0x7ff9dc008040 comp rx=0 tx=0).stop 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.717+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00b3cc0 msgr2=0x7ff9f00cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.717+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00b3cc0 0x7ff9f00cd020 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff9e000c450 tx=0x7ff9e000c760 comp rx=0 tx=0).stop 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 shutdown_connections 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff9d006ac90 0x7ff9d006d150 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f0160770 0x7ff9f00cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9f00b3cc0 0x7ff9f00cd020 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 --2- 172.21.15.99:0/3669772941 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f00d3380 0x7ff9f015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 >> 172.21.15.99:0/3669772941 conn(0x7ff9f008afd0 msgr2=0x7ff9f015f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 shutdown_connections 2024-03-29T09:12:07.718 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.718+0000 7ff9f5800700 1 -- 172.21.15.99:0/3669772941 wait complete. 2024-03-29T09:12:07.736 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:12:07.927 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.928+0000 7f1b4f600700 1 Processor -- start 2024-03-29T09:12:07.928 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.928+0000 7f1b4f600700 1 -- start start 2024-03-29T09:12:07.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4f600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b480d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4f600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1b48160cc0 0x7f1b480ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.935 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4f600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b480d3a70 0x7f1b480d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4f600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480cddc0 con 0x7f1b480d3a70 2024-03-29T09:12:07.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4f600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480ce0a0 con 0x7f1b48160cc0 2024-03-29T09:12:07.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4f600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480ce380 con 0x7f1b481608e0 2024-03-29T09:12:07.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4d600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b480d3a70 0x7f1b480d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4d600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1b480d3a70 0x7f1b480d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36084/0 (socket says 172.21.15.99:36084) 2024-03-29T09:12:07.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4d600700 1 -- 172.21.15.99:0/4160854555 learned_addr learned my addr 172.21.15.99:0/4160854555 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:07.936 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4cc00700 1 --2- 172.21.15.99:0/4160854555 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b480d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b47e00700 1 --2- 172.21.15.99:0/4160854555 >> v2:172.21.15.133:3300/0 conn(0x7f1b48160cc0 0x7f1b480ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4d600700 1 -- 172.21.15.99:0/4160854555 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 msgr2=0x7f1b480d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4d600700 1 --2- 172.21.15.99:0/4160854555 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b480d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.929+0000 7f1b4d600700 1 -- 172.21.15.99:0/4160854555 >> v2:172.21.15.133:3300/0 conn(0x7f1b48160cc0 msgr2=0x7f1b480ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b4d600700 1 --2- 172.21.15.99:0/4160854555 >> v2:172.21.15.133:3300/0 conn(0x7f1b48160cc0 0x7f1b480ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b4d600700 1 -- 172.21.15.99:0/4160854555 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480ce660 con 0x7f1b480d3a70 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b4d600700 1 --2- 172.21.15.99:0/4160854555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b480d3a70 0x7f1b480d1410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1b4808d120 tx=0x7f1b4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedb2dc853cba66b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b47400700 1 -- 172.21.15.99:0/4160854555 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000e070 con 0x7f1b480d3a70 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b47400700 1 -- 172.21.15.99:0/4160854555 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4000b9c0 con 0x7f1b480d3a70 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b47400700 1 -- 172.21.15.99:0/4160854555 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4000bd70 con 0x7f1b480d3a70 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b4f600700 1 -- 172.21.15.99:0/4160854555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b480d3a70 msgr2=0x7f1b480d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.930+0000 7f1b4f600700 1 --2- 172.21.15.99:0/4160854555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b480d3a70 0x7f1b480d1410 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1b4808d120 tx=0x7f1b4000b330 comp rx=0 tx=0).stop 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.931+0000 7f1b4f600700 1 -- 172.21.15.99:0/4160854555 shutdown_connections 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.931+0000 7f1b4f600700 1 --2- 172.21.15.99:0/4160854555 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b480d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.931+0000 7f1b4f600700 1 --2- 172.21.15.99:0/4160854555 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b480d3a70 0x7f1b480d1410 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.931+0000 7f1b4f600700 1 --2- 172.21.15.99:0/4160854555 >> v2:172.21.15.133:3300/0 conn(0x7f1b48160cc0 0x7f1b480ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.931+0000 7f1b4f600700 1 -- 172.21.15.99:0/4160854555 >> 172.21.15.99:0/4160854555 conn(0x7f1b4809c490 msgr2=0x7f1b480d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.931+0000 7f1b4f600700 1 -- 172.21.15.99:0/4160854555 shutdown_connections 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.931+0000 7f1b4f600700 1 -- 172.21.15.99:0/4160854555 wait complete. 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.933+0000 7f1b4f600700 1 Processor -- start 2024-03-29T09:12:07.938 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.933+0000 7f1b4f600700 1 -- start start 2024-03-29T09:12:07.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4f600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b480d3a70 0x7f1b480cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4f600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b481f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4f600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b48160cc0 0x7f1b4815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4f600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4814fef0 con 0x7f1b48160cc0 2024-03-29T09:12:07.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4f600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4815b610 con 0x7f1b480d3a70 2024-03-29T09:12:07.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4f600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480cd550 con 0x7f1b481608e0 2024-03-29T09:12:07.939 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b48160cc0 0x7f1b4815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b48160cc0 0x7f1b4815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36090/0 (socket says 172.21.15.99:36090) 2024-03-29T09:12:07.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 -- 172.21.15.99:0/1957555030 learned_addr learned my addr 172.21.15.99:0/1957555030 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:07.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4cc00700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b480d3a70 0x7f1b480cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b47e00700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b481f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 -- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 msgr2=0x7f1b481f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b481f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 -- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b480d3a70 msgr2=0x7f1b480cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:07.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b480d3a70 0x7f1b480cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:07.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.934+0000 7f1b4d600700 1 -- 172.21.15.99:0/1957555030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b40009b30 con 0x7f1b48160cc0 2024-03-29T09:12:07.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.935+0000 7f1b4d600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b48160cc0 0x7f1b4815baa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1b4808c9a0 tx=0x7f1b40004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.935+0000 7f1b44c00700 1 -- 172.21.15.99:0/1957555030 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400099b0 con 0x7f1b48160cc0 2024-03-29T09:12:07.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.935+0000 7f1b44c00700 1 -- 172.21.15.99:0/1957555030 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b40004250 con 0x7f1b48160cc0 2024-03-29T09:12:07.941 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.935+0000 7f1b44c00700 1 -- 172.21.15.99:0/1957555030 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40004600 con 0x7f1b48160cc0 2024-03-29T09:12:07.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.935+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4815a720 con 0x7f1b48160cc0 2024-03-29T09:12:07.942 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.935+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4815ace0 con 0x7f1b48160cc0 2024-03-29T09:12:07.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.936+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4815b7c0 con 0x7f1b48160cc0 2024-03-29T09:12:07.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.937+0000 7f1b44c00700 1 -- 172.21.15.99:0/1957555030 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1b40004760 con 0x7f1b48160cc0 2024-03-29T09:12:07.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.938+0000 7f1b44c00700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b2806ac90 0x7f1b2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:07.943 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.938+0000 7f1b44c00700 1 -- 172.21.15.99:0/1957555030 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1b4008a900 con 0x7f1b48160cc0 2024-03-29T09:12:07.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.938+0000 7f1b4cc00700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b2806ac90 0x7f1b2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:07.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.938+0000 7f1b4cc00700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b2806ac90 0x7f1b2806d150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006b20 tx=0x7f1b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:07.944 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:07.942+0000 7f1b44c00700 1 -- 172.21.15.99:0/1957555030 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1b4815b7c0 con 0x7f1b48160cc0 2024-03-29T09:12:08.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.161+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1b4815f510 con 0x7f1b48160cc0 2024-03-29T09:12:08.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.162+0000 7f1b44c00700 1 -- 172.21.15.99:0/1957555030 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1b4002a080 con 0x7f1b48160cc0 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b2806ac90 msgr2=0x7f1b2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b2806ac90 0x7f1b2806d150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006b20 tx=0x7f1b3c008040 comp rx=0 tx=0).stop 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b48160cc0 msgr2=0x7f1b4815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b48160cc0 0x7f1b4815baa0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1b4808c9a0 tx=0x7f1b40004ad0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 shutdown_connections 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1b2806ac90 0x7f1b2806d150 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1b48160cc0 0x7f1b4815baa0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b480d3a70 0x7f1b480cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 --2- 172.21.15.99:0/1957555030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1b481608e0 0x7f1b481f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 >> 172.21.15.99:0/1957555030 conn(0x7f1b4809c490 msgr2=0x7f1b4806ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 shutdown_connections 2024-03-29T09:12:08.165 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.164+0000 7f1b4f600700 1 -- 172.21.15.99:0/1957555030 wait complete. 2024-03-29T09:12:08.166 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:12:08.182 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:12:08.182 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:08.224 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:08.452 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.452+0000 7f67f8000700 1 Processor -- start 2024-03-29T09:12:08.453 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.453+0000 7f67f8000700 1 -- start start 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f8000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f67f00d1b20 0x7f67f00d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f8000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f67f0160cc0 0x7f67f00ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f8000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f00cddc0 con 0x7f67f01608e0 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f8000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f00ce0a0 con 0x7f67f00d1b20 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f8000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f00ce380 con 0x7f67f0160cc0 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f4c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f67f00d1b20 0x7f67f00d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f4c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f67f00d1b20 0x7f67f00d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36098/0 (socket says 172.21.15.99:36098) 2024-03-29T09:12:08.454 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f4c00700 1 -- 172.21.15.99:0/3072643605 learned_addr learned my addr 172.21.15.99:0/3072643605 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f6000700 1 --2- 172.21.15.99:0/3072643605 >> v2:172.21.15.133:3300/0 conn(0x7f67f0160cc0 0x7f67f00ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f4c00700 1 -- 172.21.15.99:0/3072643605 >> v2:172.21.15.133:3300/0 conn(0x7f67f0160cc0 msgr2=0x7f67f00ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f4c00700 1 --2- 172.21.15.99:0/3072643605 >> v2:172.21.15.133:3300/0 conn(0x7f67f0160cc0 0x7f67f00ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f4c00700 1 -- 172.21.15.99:0/3072643605 >> v1:172.21.15.133:6789/0 conn(0x7f67f01608e0 legacy=0x7f67f00d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.454+0000 7f67f4c00700 1 -- 172.21.15.99:0/3072643605 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f00ce660 con 0x7f67f00d1b20 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.455+0000 7f67f4c00700 1 --2- 172.21.15.99:0/3072643605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f00d0150 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f67f00b91e0 tx=0x7f67dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b41804ce5b104 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.455+0000 7f67efe00700 1 -- 172.21.15.99:0/3072643605 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00b870 con 0x7f67f00d1b20 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.455+0000 7f67efe00700 1 -- 172.21.15.99:0/3072643605 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc00b9d0 con 0x7f67f00d1b20 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.455+0000 7f67efe00700 1 -- 172.21.15.99:0/3072643605 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00bd60 con 0x7f67f00d1b20 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.455+0000 7f67f8000700 1 -- 172.21.15.99:0/3072643605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 msgr2=0x7f67f00d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.455+0000 7f67f8000700 1 --2- 172.21.15.99:0/3072643605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f00d0150 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f67f00b91e0 tx=0x7f67dc009bd0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.456+0000 7f67f8000700 1 -- 172.21.15.99:0/3072643605 shutdown_connections 2024-03-29T09:12:08.456 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.456+0000 7f67f8000700 1 --2- 172.21.15.99:0/3072643605 >> v2:172.21.15.133:3300/0 conn(0x7f67f0160cc0 0x7f67f00ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.456+0000 7f67f8000700 1 --2- 172.21.15.99:0/3072643605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f00d0150 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.456+0000 7f67f8000700 1 -- 172.21.15.99:0/3072643605 >> 172.21.15.99:0/3072643605 conn(0x7f67f009c490 msgr2=0x7f67f00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.456+0000 7f67f8000700 1 -- 172.21.15.99:0/3072643605 shutdown_connections 2024-03-29T09:12:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.456+0000 7f67f8000700 1 -- 172.21.15.99:0/3072643605 wait complete. 2024-03-29T09:12:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.457+0000 7f67f8000700 1 Processor -- start 2024-03-29T09:12:08.457 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.458+0000 7f67f8000700 1 -- start start 2024-03-29T09:12:08.458 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.458+0000 7f67f8000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.458+0000 7f67f8000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f01608e0 0x7f67f00a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f8000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67f0160cc0 0x7f67f00a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f8000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00a0e50 con 0x7f67f00d1b20 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f8000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00cd500 con 0x7f67f0160cc0 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f8000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00cd220 con 0x7f67f01608e0 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f015a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36112/0 (socket says 172.21.15.99:36112) 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 -- 172.21.15.99:0/3988513120 learned_addr learned my addr 172.21.15.99:0/3988513120 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f4c00700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f01608e0 0x7f67f00a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.459 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f6000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67f0160cc0 0x7f67f00a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 -- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f01608e0 msgr2=0x7f67f00a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f01608e0 0x7f67f00a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 -- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67f0160cc0 msgr2=0x7f67f00a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67f0160cc0 0x7f67f00a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 -- 172.21.15.99:0/3988513120 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc009550 con 0x7f67f00d1b20 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f4c00700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f01608e0 0x7f67f00a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f6000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67f0160cc0 0x7f67f00a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.459+0000 7f67f5600700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f015a5f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c4e0 tx=0x7f67e400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.460+0000 7f67ed600700 1 -- 172.21.15.99:0/3988513120 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e40052d0 con 0x7f67f00d1b20 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.460+0000 7f67ed600700 1 -- 172.21.15.99:0/3988513120 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e4005430 con 0x7f67f00d1b20 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.460+0000 7f67ed600700 1 -- 172.21.15.99:0/3988513120 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e40038c0 con 0x7f67f00d1b20 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.460+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f00ca350 con 0x7f67f00d1b20 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.460+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f00ca910 con 0x7f67f00d1b20 2024-03-29T09:12:08.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.461+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f00677a0 con 0x7f67f00d1b20 2024-03-29T09:12:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.462+0000 7f67ed600700 1 -- 172.21.15.99:0/3988513120 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f67e4003a20 con 0x7f67f00d1b20 2024-03-29T09:12:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.463+0000 7f67ed600700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f67d006ac90 0x7f67d006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.462 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.463+0000 7f67ed600700 1 -- 172.21.15.99:0/3988513120 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f67e4088890 con 0x7f67f00d1b20 2024-03-29T09:12:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.463+0000 7f67f4c00700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f67d006ac90 0x7f67d006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.463+0000 7f67f4c00700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f67d006ac90 0x7f67d006d150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000c00 tx=0x7f67dc008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:08.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.467+0000 7f67ed600700 1 -- 172.21.15.99:0/3988513120 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f67e40539c0 con 0x7f67f00d1b20 2024-03-29T09:12:08.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.674+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67f00cf390 con 0x7f67f00d1b20 2024-03-29T09:12:08.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.675+0000 7f67ed600700 1 -- 172.21.15.99:0/3988513120 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f67e4017090 con 0x7f67f00d1b20 2024-03-29T09:12:08.675 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:08.676 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:08.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f67d006ac90 msgr2=0x7f67d006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f67d006ac90 0x7f67d006d150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000c00 tx=0x7f67dc008030 comp rx=0 tx=0).stop 2024-03-29T09:12:08.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 msgr2=0x7f67f015a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f015a5f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f67e400c4e0 tx=0x7f67e400c8a0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 shutdown_connections 2024-03-29T09:12:08.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f67d006ac90 0x7f67d006d150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67f0160cc0 0x7f67f00a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67f00d1b20 0x7f67f015a5f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 --2- 172.21.15.99:0/3988513120 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f01608e0 0x7f67f00a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 >> 172.21.15.99:0/3988513120 conn(0x7f67f009c490 msgr2=0x7f67f015f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:08.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 shutdown_connections 2024-03-29T09:12:08.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.678+0000 7f67f8000700 1 -- 172.21.15.99:0/3988513120 wait complete. 2024-03-29T09:12:08.697 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:11:41.952804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:11:42.633561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:12:08.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:12:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.889+0000 7f88fc600700 1 Processor -- start 2024-03-29T09:12:08.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.889+0000 7f88fc600700 1 -- start start 2024-03-29T09:12:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.890+0000 7f88fc600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f4160770 0x7f88f40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.890 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.890+0000 7f88fc600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f88f40d3550 0x7f88f40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.890+0000 7f88fc600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f40d1520 con 0x7f88f40d0d60 2024-03-29T09:12:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.890+0000 7f88fc600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f40d1800 con 0x7f88f40d3550 2024-03-29T09:12:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.890+0000 7f88fc600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40d1ae0 con 0x7f88f4160770 2024-03-29T09:12:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88f9200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f88f40d0d60 0x7f88f40d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39248/0 (socket says 172.21.15.99:39248) 2024-03-29T09:12:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88f9200700 1 -- 172.21.15.99:0/2377626262 learned_addr learned my addr 172.21.15.99:0/2377626262 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88f9c00700 1 --2- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f4160770 0x7f88f40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.891 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88fa600700 1 --2- 172.21.15.99:0/2377626262 >> v2:172.21.15.133:3300/0 conn(0x7f88f40d3550 0x7f88f40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88f3e00700 1 -- 172.21.15.99:0/2377626262 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561235413 0 0) 0x7f88f40d1520 con 0x7f88f40d0d60 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88f3e00700 1 -- 172.21.15.99:0/2377626262 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88dc0035b0 con 0x7f88f40d0d60 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88fa600700 1 -- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f4160770 msgr2=0x7f88f40b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88fa600700 1 --2- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f4160770 0x7f88f40b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88fa600700 1 -- 172.21.15.99:0/2377626262 >> v1:172.21.15.99:6789/0 conn(0x7f88f40d0d60 legacy=0x7f88f40d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88fa600700 1 -- 172.21.15.99:0/2377626262 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f40d1dc0 con 0x7f88f40d3550 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.891+0000 7f88f9c00700 1 --2- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f4160770 0x7f88f40b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.892+0000 7f88fa600700 1 --2- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d3550 0x7f88f40ce080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009d90 tx=0x7f88ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd362cfe0ab4eef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.892+0000 7f88f3e00700 1 -- 172.21.15.99:0/2377626262 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00e070 con 0x7f88f40d3550 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.892+0000 7f88f3e00700 1 -- 172.21.15.99:0/2377626262 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec00bb20 con 0x7f88f40d3550 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.892+0000 7f88f3e00700 1 -- 172.21.15.99:0/2377626262 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec00be90 con 0x7f88f40d3550 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.892+0000 7f88fc600700 1 -- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d3550 msgr2=0x7f88f40ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.892+0000 7f88fc600700 1 --2- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d3550 0x7f88f40ce080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009d90 tx=0x7f88ec00b330 comp rx=0 tx=0).stop 2024-03-29T09:12:08.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.893+0000 7f88fc600700 1 -- 172.21.15.99:0/2377626262 shutdown_connections 2024-03-29T09:12:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.893+0000 7f88fc600700 1 --2- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d3550 0x7f88f40ce080 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.893+0000 7f88fc600700 1 --2- 172.21.15.99:0/2377626262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f4160770 0x7f88f40b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.893+0000 7f88fc600700 1 -- 172.21.15.99:0/2377626262 >> 172.21.15.99:0/2377626262 conn(0x7f88f408afd0 msgr2=0x7f88f409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.893+0000 7f88fc600700 1 -- 172.21.15.99:0/2377626262 shutdown_connections 2024-03-29T09:12:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.893+0000 7f88fc600700 1 -- 172.21.15.99:0/2377626262 wait complete. 2024-03-29T09:12:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.894+0000 7f88fc600700 1 Processor -- start 2024-03-29T09:12:08.894 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.894+0000 7f88fc600700 1 -- start start 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.895+0000 7f88fc600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d0d60 0x7f88f40b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.895+0000 7f88fc600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f40d3550 0x7f88f41f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.895+0000 7f88fc600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88f4160770 0x7f88f40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.895+0000 7f88fc600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f40b90a0 con 0x7f88f4160770 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.895+0000 7f88fc600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4150100 con 0x7f88f40d0d60 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.895+0000 7f88fc600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f414f120 con 0x7f88f40d3550 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88f4160770 0x7f88f40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88f4160770 0x7f88f40cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36118/0 (socket says 172.21.15.99:36118) 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 -- 172.21.15.99:0/1006407974 learned_addr learned my addr 172.21.15.99:0/1006407974 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:08.896 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88f9200700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f40d3550 0x7f88f41f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88f9c00700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d0d60 0x7f88f40b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 -- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f40d3550 msgr2=0x7f88f41f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f40d3550 0x7f88f41f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 -- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d0d60 msgr2=0x7f88f40b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:08.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d0d60 0x7f88f40b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:08.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 -- 172.21.15.99:0/1006407974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80094a0 con 0x7f88f4160770 2024-03-29T09:12:08.897 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.896+0000 7f88fa600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88f4160770 0x7f88f40cc300 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f88ec013000 tx=0x7f88ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:08.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.897+0000 7f88f1600700 1 -- 172.21.15.99:0/1006407974 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec007000 con 0x7f88f4160770 2024-03-29T09:12:08.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.897+0000 7f88f1600700 1 -- 172.21.15.99:0/1006407974 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec012650 con 0x7f88f4160770 2024-03-29T09:12:08.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.897+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec009a70 con 0x7f88f4160770 2024-03-29T09:12:08.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.897+0000 7f88f1600700 1 -- 172.21.15.99:0/1006407974 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec012a00 con 0x7f88f4160770 2024-03-29T09:12:08.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.897+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f414f3f0 con 0x7f88f4160770 2024-03-29T09:12:08.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.898+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f415ff40 con 0x7f88f4160770 2024-03-29T09:12:08.898 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.899+0000 7f88f1600700 1 -- 172.21.15.99:0/1006407974 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f88ec012b60 con 0x7f88f4160770 2024-03-29T09:12:08.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.900+0000 7f88f1600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88d406ac90 0x7f88d406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:08.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.900+0000 7f88f1600700 1 -- 172.21.15.99:0/1006407974 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f88ec08b560 con 0x7f88f4160770 2024-03-29T09:12:08.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.900+0000 7f88f9c00700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88d406ac90 0x7f88d406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:08.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.901+0000 7f88f9c00700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88d406ac90 0x7f88d406d150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009d70 tx=0x7f88e8009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:08.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:08.904+0000 7f88f1600700 1 -- 172.21.15.99:0/1006407974 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f88ec0565f0 con 0x7f88f4160770 2024-03-29T09:12:09.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.106+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88f40678b0 con 0x7f88f4160770 2024-03-29T09:12:09.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.106+0000 7f88f1600700 1 -- 172.21.15.99:0/1006407974 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88ec059c10 con 0x7f88f4160770 2024-03-29T09:12:09.106 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88d406ac90 msgr2=0x7f88d406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88d406ac90 0x7f88d406d150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f88e8009d70 tx=0x7f88e8009210 comp rx=0 tx=0).stop 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88f4160770 msgr2=0x7f88f40cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88f4160770 0x7f88f40cc300 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f88ec013000 tx=0x7f88ec004ad0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 shutdown_connections 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88d406ac90 0x7f88d406d150 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88f4160770 0x7f88f40cc300 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88f40d0d60 0x7f88f40b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 --2- 172.21.15.99:0/1006407974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88f40d3550 0x7f88f41f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 >> 172.21.15.99:0/1006407974 conn(0x7f88f408afd0 msgr2=0x7f88f415f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:09.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.108+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 shutdown_connections 2024-03-29T09:12:09.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.109+0000 7f88fc600700 1 -- 172.21.15.99:0/1006407974 wait complete. 2024-03-29T09:12:09.127 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:12:09.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.320+0000 7f2d83c00700 1 Processor -- start 2024-03-29T09:12:09.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.320+0000 7f2d83c00700 1 -- start start 2024-03-29T09:12:09.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.321+0000 7f2d83c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c0adc40 0x7f2d7c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.321+0000 7f2d83c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d7c072120 0x7f2d7c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.321+0000 7f2d83c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c06c580 con 0x7f2d7c06d230 2024-03-29T09:12:09.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.321+0000 7f2d83c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c15c7a0 con 0x7f2d7c072120 2024-03-29T09:12:09.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.321+0000 7f2d83c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c06c8d0 con 0x7f2d7c0adc40 2024-03-29T09:12:09.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.321+0000 7f2d81200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c0adc40 0x7f2d7c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d7be00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d7c072120 0x7f2d7c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d81c00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2d7c06d230 0x7f2d7c155070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:43544/0 (socket says 172.21.15.99:43544) 2024-03-29T09:12:09.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d81c00700 1 -- 172.21.15.99:0/4182490432 learned_addr learned my addr 172.21.15.99:0/4182490432 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d7be00700 1 -- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c0adc40 msgr2=0x7f2d7c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d7be00700 1 --2- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c0adc40 0x7f2d7c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d7be00700 1 -- 172.21.15.99:0/4182490432 >> v1:172.21.15.133:6789/0 conn(0x7f2d7c06d230 legacy=0x7f2d7c155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d7be00700 1 -- 172.21.15.99:0/4182490432 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0abf10 con 0x7f2d7c072120 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.322+0000 7f2d81200700 1 --2- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c0adc40 0x7f2d7c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d7be00700 1 --2- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c072120 0x7f2d7c06ccf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009840 tx=0x7f2d68009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ceb092601ab736 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d7b400700 1 -- 172.21.15.99:0/4182490432 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68011830 con 0x7f2d7c072120 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d7b400700 1 -- 172.21.15.99:0/4182490432 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d68011990 con 0x7f2d7c072120 2024-03-29T09:12:09.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d7b400700 1 -- 172.21.15.99:0/4182490432 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68011d20 con 0x7f2d7c072120 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d83c00700 1 -- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c072120 msgr2=0x7f2d7c06ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d83c00700 1 --2- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c072120 0x7f2d7c06ccf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009840 tx=0x7f2d68009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d83c00700 1 -- 172.21.15.99:0/4182490432 shutdown_connections 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d83c00700 1 --2- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c0adc40 0x7f2d7c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d83c00700 1 --2- 172.21.15.99:0/4182490432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c072120 0x7f2d7c06ccf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.323+0000 7f2d83c00700 1 -- 172.21.15.99:0/4182490432 >> 172.21.15.99:0/4182490432 conn(0x7f2d7c066540 msgr2=0x7f2d7c066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.324+0000 7f2d83c00700 1 -- 172.21.15.99:0/4182490432 shutdown_connections 2024-03-29T09:12:09.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.324+0000 7f2d83c00700 1 -- 172.21.15.99:0/4182490432 wait complete. 2024-03-29T09:12:09.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.325+0000 7f2d83c00700 1 Processor -- start 2024-03-29T09:12:09.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.325+0000 7f2d83c00700 1 -- start start 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.326+0000 7f2d83c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c06d230 0x7f2d7c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.326+0000 7f2d83c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c072120 0x7f2d7c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.326+0000 7f2d83c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d7c0adc40 0x7f2d7c156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.326+0000 7f2d83c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c158070 con 0x7f2d7c0adc40 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.326+0000 7f2d83c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c157460 con 0x7f2d7c06d230 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.326+0000 7f2d83c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c157740 con 0x7f2d7c072120 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.326+0000 7f2d81c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d7c0adc40 0x7f2d7c156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d7c0adc40 0x7f2d7c156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36124/0 (socket says 172.21.15.99:36124) 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 -- 172.21.15.99:0/819780084 learned_addr learned my addr 172.21.15.99:0/819780084 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81200700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c06d230 0x7f2d7c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d7be00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c072120 0x7f2d7c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 -- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c072120 msgr2=0x7f2d7c152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c072120 0x7f2d7c152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 -- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c06d230 msgr2=0x7f2d7c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c06d230 0x7f2d7c158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 -- 172.21.15.99:0/819780084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700094a0 con 0x7f2d7c0adc40 2024-03-29T09:12:09.328 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d81c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d7c0adc40 0x7f2d7c156a10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ca80 tx=0x7f2d7400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.327+0000 7f2d78c00700 1 -- 172.21.15.99:0/819780084 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400a9e0 con 0x7f2d7c0adc40 2024-03-29T09:12:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.328+0000 7f2d78c00700 1 -- 172.21.15.99:0/819780084 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7400ab40 con 0x7f2d7c0adc40 2024-03-29T09:12:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.328+0000 7f2d78c00700 1 -- 172.21.15.99:0/819780084 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74012400 con 0x7f2d7c0adc40 2024-03-29T09:12:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.328+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d680094a0 con 0x7f2d7c0adc40 2024-03-29T09:12:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.328+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c157d00 con 0x7f2d7c0adc40 2024-03-29T09:12:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.329+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c09ce00 con 0x7f2d7c0adc40 2024-03-29T09:12:09.329 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.330+0000 7f2d78c00700 1 -- 172.21.15.99:0/819780084 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2d7401b3b0 con 0x7f2d7c0adc40 2024-03-29T09:12:09.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.331+0000 7f2d78c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d5c06ac90 0x7f2d5c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.330 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.331+0000 7f2d78c00700 1 -- 172.21.15.99:0/819780084 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2d74088f00 con 0x7f2d7c0adc40 2024-03-29T09:12:09.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.331+0000 7f2d81200700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d5c06ac90 0x7f2d5c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.331 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.331+0000 7f2d81200700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d5c06ac90 0x7f2d5c06d150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d70 tx=0x7f2d70009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:09.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.335+0000 7f2d78c00700 1 -- 172.21.15.99:0/819780084 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f2d74054050 con 0x7f2d7c0adc40 2024-03-29T09:12:09.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.540+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d7c0678b0 con 0x7f2d7c0adc40 2024-03-29T09:12:09.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.541+0000 7f2d78c00700 1 -- 172.21.15.99:0/819780084 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d74017020 con 0x7f2d7c0adc40 2024-03-29T09:12:09.541 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d5c06ac90 msgr2=0x7f2d5c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d5c06ac90 0x7f2d5c06d150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009d70 tx=0x7f2d70009210 comp rx=0 tx=0).stop 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d7c0adc40 msgr2=0x7f2d7c156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d7c0adc40 0x7f2d7c156a10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ca80 tx=0x7f2d7400cd90 comp rx=0 tx=0).stop 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 shutdown_connections 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d5c06ac90 0x7f2d5c06d150 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d7c0adc40 0x7f2d7c156a10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d7c06d230 0x7f2d7c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.543+0000 7f2d83c00700 1 --2- 172.21.15.99:0/819780084 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d7c072120 0x7f2d7c152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.544+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 >> 172.21.15.99:0/819780084 conn(0x7f2d7c066540 msgr2=0x7f2d7c15acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.544+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 shutdown_connections 2024-03-29T09:12:09.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.544+0000 7f2d83c00700 1 -- 172.21.15.99:0/819780084 wait complete. 2024-03-29T09:12:09.561 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:12:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.754+0000 7f3050200700 1 Processor -- start 2024-03-29T09:12:09.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.754+0000 7f3050200700 1 -- start start 2024-03-29T09:12:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.755+0000 7f3050200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f30480b3cc0 0x7f30480d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.755+0000 7f3050200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480d3380 0x7f30480ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.755+0000 7f3050200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30480d3810 con 0x7f3048160770 2024-03-29T09:12:09.756 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.755+0000 7f3050200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30480b4140 con 0x7f30480b3cc0 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.755+0000 7f3050200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30480d1520 con 0x7f30480d3380 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f30480b3cc0 0x7f30480d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f30480b3cc0 0x7f30480d2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36136/0 (socket says 172.21.15.99:36136) 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 -- 172.21.15.99:0/3808349515 learned_addr learned my addr 172.21.15.99:0/3808349515 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304e200700 1 --2- 172.21.15.99:0/3808349515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480d3380 0x7f30480ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 -- 172.21.15.99:0/3808349515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480d3380 msgr2=0x7f30480ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 --2- 172.21.15.99:0/3808349515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480d3380 0x7f30480ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 -- 172.21.15.99:0/3808349515 >> v1:172.21.15.133:6789/0 conn(0x7f3048160770 legacy=0x7f30480d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 -- 172.21.15.99:0/3808349515 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480d1800 con 0x7f30480b3cc0 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.756+0000 7f304ce00700 1 --2- 172.21.15.99:0/3808349515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30480b3cc0 0x7f30480d2e40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f30340098f0 tx=0x7f3034009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4005860f55698fd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3047e00700 1 -- 172.21.15.99:0/3808349515 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303400b8a0 con 0x7f30480b3cc0 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3047e00700 1 -- 172.21.15.99:0/3808349515 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303400ba00 con 0x7f30480b3cc0 2024-03-29T09:12:09.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3047e00700 1 -- 172.21.15.99:0/3808349515 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303400bd90 con 0x7f30480b3cc0 2024-03-29T09:12:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 -- 172.21.15.99:0/3808349515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30480b3cc0 msgr2=0x7f30480d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.758 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 --2- 172.21.15.99:0/3808349515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30480b3cc0 0x7f30480d2e40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f30340098f0 tx=0x7f3034009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 -- 172.21.15.99:0/3808349515 shutdown_connections 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 --2- 172.21.15.99:0/3808349515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480d3380 0x7f30480ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 --2- 172.21.15.99:0/3808349515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f30480b3cc0 0x7f30480d2e40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 -- 172.21.15.99:0/3808349515 >> 172.21.15.99:0/3808349515 conn(0x7f304808afd0 msgr2=0x7f304809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 -- 172.21.15.99:0/3808349515 shutdown_connections 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.757+0000 7f3050200700 1 -- 172.21.15.99:0/3808349515 wait complete. 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.759+0000 7f3050200700 1 Processor -- start 2024-03-29T09:12:09.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.759+0000 7f3050200700 1 -- start start 2024-03-29T09:12:09.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f3050200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480b3cc0 0x7f30480cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f3050200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30480d3380 0x7f30481f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f3050200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3048160770 0x7f30480b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f3050200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30480cb9e0 con 0x7f3048160770 2024-03-29T09:12:09.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f3050200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30480cbcc0 con 0x7f30480d3380 2024-03-29T09:12:09.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f3050200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30480cbfa0 con 0x7f30480b3cc0 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f304e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3048160770 0x7f30480b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f304e200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3048160770 0x7f30480b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36150/0 (socket says 172.21.15.99:36150) 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f304e200700 1 -- 172.21.15.99:0/2246391513 learned_addr learned my addr 172.21.15.99:0/2246391513 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.760+0000 7f304d800700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480b3cc0 0x7f30480cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f304ce00700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30480d3380 0x7f30481f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f304e200700 1 -- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480b3cc0 msgr2=0x7f30480cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f304e200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480b3cc0 0x7f30480cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f304e200700 1 -- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30480d3380 msgr2=0x7f30481f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f304e200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30480d3380 0x7f30481f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f304e200700 1 -- 172.21.15.99:0/2246391513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3034009550 con 0x7f3048160770 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f304e200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3048160770 0x7f30480b06f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3040007e40 tx=0x7f304000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f3045600700 1 -- 172.21.15.99:0/2246391513 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304000ed30 con 0x7f3048160770 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f3045600700 1 -- 172.21.15.99:0/2246391513 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040014420 con 0x7f3048160770 2024-03-29T09:12:09.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f3045600700 1 -- 172.21.15.99:0/2246391513 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30400147d0 con 0x7f3048160770 2024-03-29T09:12:09.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.761+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30480cc280 con 0x7f3048160770 2024-03-29T09:12:09.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.762+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30481f29b0 con 0x7f3048160770 2024-03-29T09:12:09.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.762+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304809e790 con 0x7f3048160770 2024-03-29T09:12:09.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.763+0000 7f3045600700 1 -- 172.21.15.99:0/2246391513 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3040014930 con 0x7f3048160770 2024-03-29T09:12:09.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.765+0000 7f3045600700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f302806ac90 0x7f302806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:09.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.765+0000 7f3045600700 1 -- 172.21.15.99:0/2246391513 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3040089b20 con 0x7f3048160770 2024-03-29T09:12:09.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.765+0000 7f304d800700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f302806ac90 0x7f302806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:09.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.766+0000 7f304d800700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f302806ac90 0x7f302806d150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f30480d27e0 tx=0x7f303c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:09.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.769+0000 7f3045600700 1 -- 172.21.15.99:0/2246391513 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f304809e790 con 0x7f3048160770 2024-03-29T09:12:09.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.971+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f304809eb70 con 0x7f3048160770 2024-03-29T09:12:09.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.971+0000 7f3045600700 1 -- 172.21.15.99:0/2246391513 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f304809eb70 con 0x7f3048160770 2024-03-29T09:12:09.971 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f302806ac90 msgr2=0x7f302806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f302806ac90 0x7f302806d150 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f30480d27e0 tx=0x7f303c009040 comp rx=0 tx=0).stop 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3048160770 msgr2=0x7f30480b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3048160770 0x7f30480b06f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3040007e40 tx=0x7f304000e3c0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 shutdown_connections 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f302806ac90 0x7f302806d150 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3048160770 0x7f30480b06f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30480b3cc0 0x7f30480cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 --2- 172.21.15.99:0/2246391513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30480d3380 0x7f30481f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 >> 172.21.15.99:0/2246391513 conn(0x7f304808afd0 msgr2=0x7f3048073090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 shutdown_connections 2024-03-29T09:12:09.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:09.974+0000 7f3050200700 1 -- 172.21.15.99:0/2246391513 wait complete. 2024-03-29T09:12:09.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:12:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.184+0000 7f9c44e00700 1 Processor -- start 2024-03-29T09:12:10.184 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.184+0000 7f9c44e00700 1 -- start start 2024-03-29T09:12:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.185+0000 7f9c44e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c400d1b20 0x7f9c400d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.185+0000 7f9c44e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c40160cc0 0x7f9c400ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.185+0000 7f9c44e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c400cddc0 con 0x7f9c401608e0 2024-03-29T09:12:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.185+0000 7f9c44e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c400ce0a0 con 0x7f9c400d1b20 2024-03-29T09:12:10.185 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.185+0000 7f9c44e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400ce380 con 0x7f9c40160cc0 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3e800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c40160cc0 0x7f9c400ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3de00700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9c401608e0 0x7f9c400d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:43560/0 (socket says 172.21.15.99:43560) 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3d400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c400d1b20 0x7f9c400d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3de00700 1 -- 172.21.15.99:0/1777719911 learned_addr learned my addr 172.21.15.99:0/1777719911 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3ca00700 1 -- 172.21.15.99:0/1777719911 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2797075916 0 0) 0x7f9c400cddc0 con 0x7f9c401608e0 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3ca00700 1 -- 172.21.15.99:0/1777719911 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c240035b0 con 0x7f9c401608e0 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3e800700 1 -- 172.21.15.99:0/1777719911 >> v2:172.21.15.133:3300/0 conn(0x7f9c400d1b20 msgr2=0x7f9c400d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3e800700 1 --2- 172.21.15.99:0/1777719911 >> v2:172.21.15.133:3300/0 conn(0x7f9c400d1b20 0x7f9c400d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3e800700 1 -- 172.21.15.99:0/1777719911 >> v1:172.21.15.133:6789/0 conn(0x7f9c401608e0 legacy=0x7f9c400d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3e800700 1 -- 172.21.15.99:0/1777719911 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c400ce660 con 0x7f9c40160cc0 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.186+0000 7f9c3d400700 1 --2- 172.21.15.99:0/1777719911 >> v2:172.21.15.133:3300/0 conn(0x7f9c400d1b20 0x7f9c400d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:10.187 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.187+0000 7f9c3e800700 1 --2- 172.21.15.99:0/1777719911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c40160cc0 0x7f9c400ceb80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009d90 tx=0x7f9c3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edce2541baab0f7f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:10.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.187+0000 7f9c3ca00700 1 -- 172.21.15.99:0/1777719911 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3800e070 con 0x7f9c40160cc0 2024-03-29T09:12:10.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.187+0000 7f9c3ca00700 1 -- 172.21.15.99:0/1777719911 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3800bb20 con 0x7f9c40160cc0 2024-03-29T09:12:10.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.187+0000 7f9c3ca00700 1 -- 172.21.15.99:0/1777719911 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c38012430 con 0x7f9c40160cc0 2024-03-29T09:12:10.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.187+0000 7f9c44e00700 1 -- 172.21.15.99:0/1777719911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c40160cc0 msgr2=0x7f9c400ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.187+0000 7f9c44e00700 1 --2- 172.21.15.99:0/1777719911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c40160cc0 0x7f9c400ceb80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009d90 tx=0x7f9c3800b330 comp rx=0 tx=0).stop 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.187+0000 7f9c44e00700 1 -- 172.21.15.99:0/1777719911 shutdown_connections 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.188+0000 7f9c44e00700 1 --2- 172.21.15.99:0/1777719911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c40160cc0 0x7f9c400ceb80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.188+0000 7f9c44e00700 1 --2- 172.21.15.99:0/1777719911 >> v2:172.21.15.133:3300/0 conn(0x7f9c400d1b20 0x7f9c400d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.188+0000 7f9c44e00700 1 -- 172.21.15.99:0/1777719911 >> 172.21.15.99:0/1777719911 conn(0x7f9c4009c490 msgr2=0x7f9c400d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.188+0000 7f9c44e00700 1 -- 172.21.15.99:0/1777719911 shutdown_connections 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.188+0000 7f9c44e00700 1 -- 172.21.15.99:0/1777719911 wait complete. 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.189+0000 7f9c44e00700 1 Processor -- start 2024-03-29T09:12:10.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.189+0000 7f9c44e00700 1 -- start start 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.190+0000 7f9c44e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c400d1b20 0x7f9c401f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.190+0000 7f9c44e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c401608e0 0x7f9c401f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.190+0000 7f9c44e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c40160cc0 0x7f9c400cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.190+0000 7f9c44e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400b5070 con 0x7f9c400d1b20 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.190+0000 7f9c44e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400b5630 con 0x7f9c40160cc0 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.190+0000 7f9c44e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c401f3b70 con 0x7f9c401608e0 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c400d1b20 0x7f9c401f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c400d1b20 0x7f9c401f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36156/0 (socket says 172.21.15.99:36156) 2024-03-29T09:12:10.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 -- 172.21.15.99:0/3891288154 learned_addr learned my addr 172.21.15.99:0/3891288154 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3d400700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c401608e0 0x7f9c401f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3e800700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c40160cc0 0x7f9c400cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 -- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c401608e0 msgr2=0x7f9c401f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c401608e0 0x7f9c401f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 -- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c40160cc0 msgr2=0x7f9c400cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c40160cc0 0x7f9c400cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 -- 172.21.15.99:0/3891288154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0094a0 con 0x7f9c400d1b20 2024-03-29T09:12:10.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.191+0000 7f9c3de00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c400d1b20 0x7f9c401f4230 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c4d0 tx=0x7f9c3000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:10.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.192+0000 7f9c36000700 1 -- 172.21.15.99:0/3891288154 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30005d70 con 0x7f9c400d1b20 2024-03-29T09:12:10.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.192+0000 7f9c36000700 1 -- 172.21.15.99:0/3891288154 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300070d0 con 0x7f9c400d1b20 2024-03-29T09:12:10.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.192+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c38009a70 con 0x7f9c400d1b20 2024-03-29T09:12:10.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.192+0000 7f9c36000700 1 -- 172.21.15.99:0/3891288154 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30007440 con 0x7f9c400d1b20 2024-03-29T09:12:10.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.192+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c400b52c0 con 0x7f9c400d1b20 2024-03-29T09:12:10.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.193+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c400677a0 con 0x7f9c400d1b20 2024-03-29T09:12:10.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.194+0000 7f9c36000700 1 -- 172.21.15.99:0/3891288154 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9c300075a0 con 0x7f9c400d1b20 2024-03-29T09:12:10.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.195+0000 7f9c36000700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9c2006ac90 0x7f9c2006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:10.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.196+0000 7f9c36000700 1 -- 172.21.15.99:0/3891288154 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9c300887c0 con 0x7f9c400d1b20 2024-03-29T09:12:10.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.196+0000 7f9c3d400700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9c2006ac90 0x7f9c2006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:10.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.196+0000 7f9c3d400700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9c2006ac90 0x7f9c2006d150 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b8d50 tx=0x7f9c2c009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:10.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.200+0000 7f9c36000700 1 -- 172.21.15.99:0/3891288154 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9c30053870 con 0x7f9c400d1b20 2024-03-29T09:12:10.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.415+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9c400d0700 con 0x7f9c400d1b20 2024-03-29T09:12:10.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.416+0000 7f9c36000700 1 -- 172.21.15.99:0/3891288154 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9c30056e90 con 0x7f9c400d1b20 2024-03-29T09:12:10.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9c2006ac90 msgr2=0x7f9c2006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:10.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9c2006ac90 0x7f9c2006d150 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c400b8d50 tx=0x7f9c2c009210 comp rx=0 tx=0).stop 2024-03-29T09:12:10.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c400d1b20 msgr2=0x7f9c401f4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:10.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c400d1b20 0x7f9c401f4230 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c4d0 tx=0x7f9c3000c7e0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 shutdown_connections 2024-03-29T09:12:10.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f9c2006ac90 0x7f9c2006d150 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c40160cc0 0x7f9c400cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9c400d1b20 0x7f9c401f4230 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 --2- 172.21.15.99:0/3891288154 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c401608e0 0x7f9c401f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 >> 172.21.15.99:0/3891288154 conn(0x7f9c4009c490 msgr2=0x7f9c400b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 shutdown_connections 2024-03-29T09:12:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:10.418+0000 7f9c44e00700 1 -- 172.21.15.99:0/3891288154 wait complete. 2024-03-29T09:12:10.419 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:12:10.436 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:12:10.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:10.436 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:10.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[67469]: fuse finished with error 0 and tester_r 0 2024-03-29T09:12:10.500 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:12:16.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:12:16.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:16.603 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:12:16.651 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T09:12:16.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.843+0000 7fcbe1600700 1 Processor -- start 2024-03-29T09:12:16.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.844+0000 7fcbe1600700 1 -- start start 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbe1600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 0x7fcbdc0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbe1600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcbdc0d1740 0x7fcbdc0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbe1600700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbdc0afde0 con 0x7fcbdc15da70 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbe1600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbdc0d2130 con 0x7fcbdc0d1740 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbe1600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0d01d0 con 0x7fcbdc15d690 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbda000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcbdc15da70 0x7fcbdc0cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:39252/0 (socket says 172.21.15.99:39252) 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbda000700 1 -- 172.21.15.99:0/3546724962 learned_addr learned my addr 172.21.15.99:0/3546724962 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:16.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 0x7fcbdc0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbdb400700 1 --2- 172.21.15.99:0/3546724962 >> v2:172.21.15.133:3300/0 conn(0x7fcbdc0d1740 0x7fcbdc0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.845+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774075226 0 0) 0x7fcbdc0afde0 con 0x7fcbdc15da70 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbc00035b0 con 0x7fcbdc15da70 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3816048910 0 0) 0x7fcbc00035b0 con 0x7fcbdc15da70 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 msgr2=0x7fcbdc0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 0x7fcbdc0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 >> v2:172.21.15.133:3300/0 conn(0x7fcbdc0d1740 msgr2=0x7fcbdc0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 --2- 172.21.15.99:0/3546724962 >> v2:172.21.15.133:3300/0 conn(0x7fcbdc0d1740 0x7fcbdc0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0d04b0 con 0x7fcbdc15da70 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fcbc8001670 con 0x7fcbdc15da70 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 0x7fcbdc0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 >> v1:172.21.15.99:6789/0 conn(0x7fcbdc15da70 legacy=0x7fcbdc0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc0003a00 0x7fcbc0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.849 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 0x7fcbc0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0004d90 0x7fcbc0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0d2130 con 0x7fcbc0003a00 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0d01d0 con 0x7fcbc0004d90 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc000ba60 con 0x7fcbdc15d690 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc0003a00 0x7fcbc0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbda000700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 0x7fcbc0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.846+0000 7fcbdb400700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0004d90 0x7fcbc0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbdaa00700 1 -- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 msgr2=0x7fcbc0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc15d690 0x7fcbc0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbdaa00700 1 -- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0004d90 msgr2=0x7fcbc0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0004d90 0x7fcbc0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbdaa00700 1 -- 172.21.15.99:0/3546724962 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0094a0 con 0x7fcbc0003a00 2024-03-29T09:12:16.850 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc0003a00 0x7fcbc0003e20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009d60 tx=0x7fcbc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63fc762b17ac849 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00e9a0 con 0x7fcbc0003a00 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc00eb00 con 0x7fcbc0003a00 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.847+0000 7fcbd9600700 1 -- 172.21.15.99:0/3546724962 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00ee90 con 0x7fcbc0003a00 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.848+0000 7fcbe1600700 1 -- 172.21.15.99:0/3546724962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc0003a00 msgr2=0x7fcbc0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.848+0000 7fcbe1600700 1 --2- 172.21.15.99:0/3546724962 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbc0003a00 0x7fcbc0003e20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009d60 tx=0x7fcbc0005880 comp rx=0 tx=0).stop 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.848+0000 7fcbdb400700 1 -- 172.21.15.99:0/3546724962 reap_dead start 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.848+0000 7fcbe1600700 1 -- 172.21.15.99:0/3546724962 shutdown_connections 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.848+0000 7fcbe1600700 1 -- 172.21.15.99:0/3546724962 >> 172.21.15.99:0/3546724962 conn(0x7fcbdc09c480 msgr2=0x7fcbdc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.848+0000 7fcbe1600700 1 -- 172.21.15.99:0/3546724962 shutdown_connections 2024-03-29T09:12:16.851 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.848+0000 7fcbe1600700 1 -- 172.21.15.99:0/3546724962 wait complete. 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.849+0000 7fcbe1600700 1 Processor -- start 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.850+0000 7fcbe1600700 1 -- start start 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.850+0000 7fcbe1600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0003a00 0x7fcbdc15e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.850+0000 7fcbe1600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbdc1603a0 0x7fcbdc152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.850+0000 7fcbe1600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc0cbce0 0x7fcbdc0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.850+0000 7fcbe1600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc15ee20 con 0x7fcbdc1603a0 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.850+0000 7fcbe1600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc152cb0 con 0x7fcbc0003a00 2024-03-29T09:12:16.854 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.850+0000 7fcbe1600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc0cc6a0 con 0x7fcbdc0cbce0 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbdc1603a0 0x7fcbdc152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbdc1603a0 0x7fcbdc152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:36172/0 (socket says 172.21.15.99:36172) 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 -- 172.21.15.99:0/2872298969 learned_addr learned my addr 172.21.15.99:0/2872298969 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbdb400700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc0cbce0 0x7fcbdc0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 -- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc0cbce0 msgr2=0x7fcbdc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc0cbce0 0x7fcbdc0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 -- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0003a00 msgr2=0x7fcbdc15e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0003a00 0x7fcbdc15e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0003a00 0x7fcbdc15e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 -- 172.21.15.99:0/2872298969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0cbac0 con 0x7fcbdc1603a0 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbda000700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbdc1603a0 0x7fcbdc152770 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8007310 tx=0x7fcbc8007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbd2a00700 1 -- 172.21.15.99:0/2872298969 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8007800 con 0x7fcbdc1603a0 2024-03-29T09:12:16.855 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbd2a00700 1 -- 172.21.15.99:0/2872298969 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc800f870 con 0x7fcbdc1603a0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.851+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbdc0b6780 con 0x7fcbdc1603a0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.852+0000 7fcbd2a00700 1 -- 172.21.15.99:0/2872298969 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc800fc40 con 0x7fcbdc1603a0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.852+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbdc15f2f0 con 0x7fcbdc1603a0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.853+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbdc0677a0 con 0x7fcbdc1603a0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.853+0000 7fcbd2a00700 1 -- 172.21.15.99:0/2872298969 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fcbc801e3b0 con 0x7fcbdc1603a0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.854+0000 7fcbd2a00700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcbbc06ac90 0x7fcbbc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.854+0000 7fcbd2a00700 1 -- 172.21.15.99:0/2872298969 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcbc8088fd0 con 0x7fcbdc1603a0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.855+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcbbc06ac90 0x7fcbbc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:16.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.855+0000 7fcbdaa00700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcbbc06ac90 0x7fcbbc06d150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009d00 tx=0x7fcbcc01e040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:16.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:16.859+0000 7fcbd2a00700 1 -- 172.21.15.99:0/2872298969 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fcbc8053fd0 con 0x7fcbdc1603a0 2024-03-29T09:12:17.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.115+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcbdc0b6910 con 0x7fcbdc1603a0 2024-03-29T09:12:17.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.115+0000 7fcbd2a00700 1 -- 172.21.15.99:0/2872298969 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcbc801a020 con 0x7fcbdc1603a0 2024-03-29T09:12:17.116 INFO:teuthology.orchestra.run.smithi099.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T09:12:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcbbc06ac90 msgr2=0x7fcbbc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcbbc06ac90 0x7fcbbc06d150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009d00 tx=0x7fcbcc01e040 comp rx=0 tx=0).stop 2024-03-29T09:12:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbdc1603a0 msgr2=0x7fcbdc152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbdc1603a0 0x7fcbdc152770 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8007310 tx=0x7fcbc8007620 comp rx=0 tx=0).stop 2024-03-29T09:12:17.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 shutdown_connections 2024-03-29T09:12:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcbbc06ac90 0x7fcbbc06d150 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcbdc0cbce0 0x7fcbdc0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcbc0003a00 0x7fcbdc15e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 --2- 172.21.15.99:0/2872298969 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcbdc1603a0 0x7fcbdc152770 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 >> 172.21.15.99:0/2872298969 conn(0x7fcbdc09c480 msgr2=0x7fcbdc06bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 shutdown_connections 2024-03-29T09:12:17.119 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:17.118+0000 7fcbe1600700 1 -- 172.21.15.99:0/2872298969 wait complete. 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi099.front.sepia.ceph.com 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T09:12:17.137 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:12:17.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:17.138 DEBUG:teuthology.orchestra.run.smithi099:> ip addr 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 ::1/128 scope host 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f0 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:d8:a2:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout: altname enp7s0f1 2024-03-29T09:12:17.180 INFO:teuthology.orchestra.run.smithi099.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:02 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 0c:c4:7a:8f:d8:03 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: inet 172.21.15.99/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft 42051sec preferred_lft 42051sec 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::ec4:7aff:fe8f:d803/64 scope link 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::3caa:acff:feac:fccd/64 scope link 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:17.181 INFO:teuthology.orchestra.run.smithi099.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether 5a:32:ee:0b:ee:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::5832:eeff:fe0b:ee92/64 scope link 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run.smithi099.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run.smithi099.stdout: link/ether fe:d0:b6:f6:cb:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run.smithi099.stdout: inet6 fe80::fcd0:b6ff:fef6:cba2/64 scope link 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run.smithi099.stdout: valid_lft forever preferred_lft forever 2024-03-29T09:12:17.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:17.182 DEBUG:teuthology.orchestra.run.smithi099:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T09:12:17.282 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-03-29T09:12:17.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:17.286 DEBUG:teuthology.orchestra.run.smithi099:> route 2024-03-29T09:12:17.329 INFO:teuthology.orchestra.run.smithi099.stdout:Kernel IP routing table 2024-03-29T09:12:17.330 INFO:teuthology.orchestra.run.smithi099.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T09:12:17.330 INFO:teuthology.orchestra.run.smithi099.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T09:12:17.330 INFO:teuthology.orchestra.run.smithi099.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T09:12:17.330 INFO:teuthology.orchestra.run.smithi099.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T09:12:17.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:17.331 DEBUG:teuthology.orchestra.run.smithi099:> (cd / && exec bash -c ' 2024-03-29T09:12:17.331 DEBUG:teuthology.orchestra.run.smithi099:> set -e 2024-03-29T09:12:17.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T09:12:17.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T09:12:17.331 DEBUG:teuthology.orchestra.run.smithi099:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T09:12:17.331 DEBUG:teuthology.orchestra.run.smithi099:> ') 2024-03-29T09:12:17.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:17.531 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:12:17.572 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:12:17.573 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T09:12:17.573 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:12:17.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:17.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link set brx.0 up 2024-03-29T09:12:17.677 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T09:12:17.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:12:17.677 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:17.718 INFO:teuthology.orchestra.run.smithi099.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T09:12:17.718 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T09:12:17.718 DEBUG:teuthology.orchestra.run.smithi099:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:17.799 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:12:17.905 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:12:17.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:12:17.967 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:12:17.970 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:12:17.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:17.971 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:12:18.013 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T09:12:18.013 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T09:12:18.058 DEBUG:teuthology.orchestra.run.smithi099:> sudo modprobe fuse 2024-03-29T09:12:18.122 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T09:12:18.122 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T09:12:18.187 INFO:teuthology.orchestra.run.smithi099.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T09:12:18.189 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T09:12:18.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:18.190 DEBUG:teuthology.orchestra.run.smithi099:> ls /sys/fs/fuse/connections 2024-03-29T09:12:18.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:2024-03-29T09:12:18.221+0000 7f3f000355c0 -1 init, newargv = 0x55e96bafe0b0 newargc=15 2024-03-29T09:12:18.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[68573]: starting ceph client 2024-03-29T09:12:18.227 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[68573]: starting fuse 2024-03-29T09:12:18.234 INFO:teuthology.orchestra.run.smithi099.stdout:44 2024-03-29T09:12:18.234 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-03-29T09:12:18.234 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:12:18.234 DEBUG:teuthology.orchestra.run.smithi099:> import glob 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> import re 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> import subprocess 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> def _find_admin_socket(client_name): 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> files = glob.glob(asok_path) 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> # Given a non-glob path, it better be there 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> if "*" not in asok_path: 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> assert(len(files) == 1) 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> return files[0] 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:18.235 DEBUG:teuthology.orchestra.run.smithi099:> for f in files: 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> contents = proc_f.read() 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> if mountpoint in contents: 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> return f 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> print(_find_admin_socket("client.0")) 2024-03-29T09:12:18.236 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:12:18.413 INFO:teuthology.orchestra.run.smithi099.stdout:/var/run/ceph/ceph-client.0.68573.asok 2024-03-29T09:12:18.592 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.68573.asok 2024-03-29T09:12:18.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:18.592 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.68573.asok status 2024-03-29T09:12:18.810 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "metadata": { 2024-03-29T09:12:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T09:12:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T09:12:18.810 INFO:teuthology.orchestra.run.smithi099.stdout: "entity_id": "0", 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "hostname": "smithi099", 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "pid": "68573", 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "root": "/" 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_count": 0, 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "dentry_pinned_count": 0, 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "id": 6462, 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "inst": { 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "name": { 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "client", 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: "num": 6462 2024-03-29T09:12:18.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2276467563 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": { 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type": "v1", 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "addr": "192.168.0.1:0", 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "nonce": 2276467563 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "inst_str": "client.6462 192.168.0.1:0/2276467563", 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "addr_str": "192.168.0.1:0/2276467563", 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "inode_count": 1, 2024-03-29T09:12:18.812 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_epoch": 25, 2024-03-29T09:12:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch": 55, 2024-03-29T09:12:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_epoch_barrier": 0, 2024-03-29T09:12:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "blocklisted": false, 2024-03-29T09:12:18.813 INFO:teuthology.orchestra.run.smithi099.stdout: "fs_name": "cephfs" 2024-03-29T09:12:18.813 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:18.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:18.824 DEBUG:teuthology.orchestra.run.smithi099:> stat --file-system '--printf=%T 2024-03-29T09:12:18.824 DEBUG:teuthology.orchestra.run.smithi099:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:18.866 INFO:teuthology.orchestra.run.smithi099.stdout:fuseblk 2024-03-29T09:12:18.867 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:18.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:18.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> import os 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> import errno 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> try: 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> if True: 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> ret = os.read(fd, 1024) 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> os.write(fd, b'"'"'content'"'"') 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> except OSError as e: 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> if e.errno != errno.EPERM: 2024-03-29T09:12:18.972 DEBUG:teuthology.orchestra.run.smithi099:> raise 2024-03-29T09:12:18.973 DEBUG:teuthology.orchestra.run.smithi099:> else: 2024-03-29T09:12:18.973 DEBUG:teuthology.orchestra.run.smithi099:> raise RuntimeError("client does not check permission of data pool") 2024-03-29T09:12:18.973 DEBUG:teuthology.orchestra.run.smithi099:> ' 2024-03-29T09:12:19.085 INFO:teuthology.orchestra.run.smithi099.stderr:Traceback (most recent call last): 2024-03-29T09:12:19.085 INFO:teuthology.orchestra.run.smithi099.stderr: File "", line 15, in 2024-03-29T09:12:19.085 INFO:teuthology.orchestra.run.smithi099.stderr:RuntimeError: client does not check permission of data pool 2024-03-29T09:12:19.260 INFO:teuthology.orchestra.run.smithi099.stderr:daemon-helper: command failed with exit status 1 2024-03-29T09:12:19.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:12:19.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-03-29T09:12:19.503 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.503+0000 7f6403c00700 1 Processor -- start 2024-03-29T09:12:19.504 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.504+0000 7f6403c00700 1 -- start start 2024-03-29T09:12:19.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f6403c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:19.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f6403c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63fc0d1520 con 0x7f63fc0d0d60 2024-03-29T09:12:19.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f6403c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63fc0d1800 con 0x7f63fc0d3550 2024-03-29T09:12:19.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f6403c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc0d1ae0 con 0x7f63fc160770 2024-03-29T09:12:19.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f63fbe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f63fc0d0d60 0x7f63fc0d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40932/0 (socket says 172.21.15.99:40932) 2024-03-29T09:12:19.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f63fbe00700 1 -- 172.21.15.99:0/2297549391 learned_addr learned my addr 172.21.15.99:0/2297549391 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f6401200700 1 --2- 172.21.15.99:0/2297549391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.505+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164150097 0 0) 0x7f63fc0d1520 con 0x7f63fc0d0d60 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63e40035b0 con 0x7f63fc0d0d60 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490345380 0 0) 0x7f63fc0d1800 con 0x7f63fc0d3550 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63fc0d1520 con 0x7f63fc0d3550 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f6401200700 1 -- 172.21.15.99:0/2297549391 >> v1:172.21.15.133:6789/0 conn(0x7f63fc0d3550 legacy=0x7f63fc0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f6401200700 1 -- 172.21.15.99:0/2297549391 >> v1:172.21.15.99:6789/0 conn(0x7f63fc0d0d60 legacy=0x7f63fc0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2908705191 0 0) 0x7f63e40035b0 con 0x7f63fc0d0d60 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f6401200700 1 -- 172.21.15.99:0/2297549391 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc0d1dc0 con 0x7f63fc160770 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f6401200700 1 --2- 172.21.15.99:0/2297549391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0b3cc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009870 tx=0x7f63f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4788de9db77c4200 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0011830 con 0x7f63fc160770 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f0011990 con 0x7f63fc160770 2024-03-29T09:12:19.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.506+0000 7f63fb400700 1 -- 172.21.15.99:0/2297549391 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0011d20 con 0x7f63fc160770 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.507+0000 7f6403c00700 1 -- 172.21.15.99:0/2297549391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 msgr2=0x7f63fc0b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.507+0000 7f6403c00700 1 --2- 172.21.15.99:0/2297549391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0b3cc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009870 tx=0x7f63f0009b80 comp rx=0 tx=0).stop 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.507+0000 7f6403c00700 1 -- 172.21.15.99:0/2297549391 shutdown_connections 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.507+0000 7f6403c00700 1 --2- 172.21.15.99:0/2297549391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0b3cc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.507+0000 7f6403c00700 1 -- 172.21.15.99:0/2297549391 >> 172.21.15.99:0/2297549391 conn(0x7f63fc08afd0 msgr2=0x7f63fc09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.507+0000 7f6403c00700 1 -- 172.21.15.99:0/2297549391 shutdown_connections 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.507+0000 7f6403c00700 1 -- 172.21.15.99:0/2297549391 wait complete. 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.509+0000 7f6403c00700 1 Processor -- start 2024-03-29T09:12:19.509 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.509+0000 7f6403c00700 1 -- start start 2024-03-29T09:12:19.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6403c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc0d0d60 0x7f63fc15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:19.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6403c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc0d3550 0x7f63fc0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:19.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6403c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6403c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc0cb9c0 con 0x7f63fc0d3550 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6403c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc0b86f0 con 0x7f63fc0d0d60 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6403c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc15c1d0 con 0x7f63fc160770 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc0d3550 0x7f63fc0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc0d3550 0x7f63fc0cc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46702/0 (socket says 172.21.15.99:46702) 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 -- 172.21.15.99:0/3294022660 learned_addr learned my addr 172.21.15.99:0/3294022660 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6401c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f6401200700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc0d0d60 0x7f63fc15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 -- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 msgr2=0x7f63fc0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 -- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc0d0d60 msgr2=0x7f63fc15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc0d0d60 0x7f63fc15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:19.512 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.510+0000 7f63fbe00700 1 -- 172.21.15.99:0/3294022660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f00094a0 con 0x7f63fc0d3550 2024-03-29T09:12:19.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.511+0000 7f63fbe00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc0d3550 0x7f63fc0cc120 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c750 tx=0x7f63e800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:19.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.511+0000 7f63f8c00700 1 -- 172.21.15.99:0/3294022660 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8005d70 con 0x7f63fc0d3550 2024-03-29T09:12:19.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.511+0000 7f63f8c00700 1 -- 172.21.15.99:0/3294022660 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e80070d0 con 0x7f63fc0d3550 2024-03-29T09:12:19.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.511+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63fc0cae40 con 0x7f63fc0d3550 2024-03-29T09:12:19.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.511+0000 7f63f8c00700 1 -- 172.21.15.99:0/3294022660 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8007440 con 0x7f63fc0d3550 2024-03-29T09:12:19.513 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.511+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc0cb400 con 0x7f63fc0d3550 2024-03-29T09:12:19.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.512+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63fc0678b0 con 0x7f63fc0d3550 2024-03-29T09:12:19.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.513+0000 7f63f8c00700 1 -- 172.21.15.99:0/3294022660 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f63e80075a0 con 0x7f63fc0d3550 2024-03-29T09:12:19.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.514+0000 7f63f8c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f63dc06ac90 0x7f63dc06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:19.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.514+0000 7f63f8c00700 1 -- 172.21.15.99:0/3294022660 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f63e80887b0 con 0x7f63fc0d3550 2024-03-29T09:12:19.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.514+0000 7f6401200700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f63dc06ac90 0x7f63dc06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:19.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.514+0000 7f6401200700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f63dc06ac90 0x7f63dc06d150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f63f00097c0 tx=0x7f63f001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:19.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.518+0000 7f63f8c00700 1 -- 172.21.15.99:0/3294022660 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f63e80538f0 con 0x7f63fc0d3550 2024-03-29T09:12:19.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.801+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f63fc15cd50 con 0x7f63fc0d3550 2024-03-29T09:12:19.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.806+0000 7f63f8c00700 1 -- 172.21.15.99:0/3294022660 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f63e8056f10 con 0x7f63fc0d3550 2024-03-29T09:12:19.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f63dc06ac90 msgr2=0x7f63dc06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:19.808 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f63dc06ac90 0x7f63dc06d150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f63f00097c0 tx=0x7f63f001c040 comp rx=0 tx=0).stop 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc0d3550 msgr2=0x7f63fc0cc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc0d3550 0x7f63fc0cc120 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c750 tx=0x7f63e800ca60 comp rx=0 tx=0).stop 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 shutdown_connections 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f63dc06ac90 0x7f63dc06d150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63fc160770 0x7f63fc0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63fc0d0d60 0x7f63fc15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 --2- 172.21.15.99:0/3294022660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63fc0d3550 0x7f63fc0cc120 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 >> 172.21.15.99:0/3294022660 conn(0x7f63fc08afd0 msgr2=0x7f63fc0d0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 shutdown_connections 2024-03-29T09:12:19.809 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:19.808+0000 7f6403c00700 1 -- 172.21.15.99:0/3294022660 wait complete. 2024-03-29T09:12:19.810 INFO:teuthology.orchestra.run.smithi099.stderr:updated caps for client.0 2024-03-29T09:12:19.826 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T09:12:19.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:12:19.924 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T09:12:20.014 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T09:12:20.014 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:12:20.014 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:20.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.288+0000 7f346fc00700 1 Processor -- start 2024-03-29T09:12:20.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.288+0000 7f346fc00700 1 -- start start 2024-03-29T09:12:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346fc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f346815e7a0 0x7f34680d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346fc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34680d12f0 0x7f34680cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346fc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f34680cfad0 0x7f34680cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346fc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34680d0460 con 0x7f346815e7a0 2024-03-29T09:12:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346fc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34680cdf80 con 0x7f34680cfad0 2024-03-29T09:12:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346fc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346815eb80 con 0x7f34680d12f0 2024-03-29T09:12:20.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346d200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f346815e7a0 0x7f34680d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346d200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f346815e7a0 0x7f34680d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46706/0 (socket says 172.21.15.99:46706) 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346d200700 1 -- 172.21.15.99:0/2590051776 learned_addr learned my addr 172.21.15.99:0/2590051776 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f3467e00700 1 --2- 172.21.15.99:0/2590051776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34680d12f0 0x7f34680cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.289+0000 7f346dc00700 1 --2- 172.21.15.99:0/2590051776 >> v2:172.21.15.133:3300/0 conn(0x7f34680cfad0 0x7f34680cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f346d200700 1 -- 172.21.15.99:0/2590051776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34680d12f0 msgr2=0x7f34680cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f346d200700 1 --2- 172.21.15.99:0/2590051776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34680d12f0 0x7f34680cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f346d200700 1 -- 172.21.15.99:0/2590051776 >> v2:172.21.15.133:3300/0 conn(0x7f34680cfad0 msgr2=0x7f34680cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f346d200700 1 --2- 172.21.15.99:0/2590051776 >> v2:172.21.15.133:3300/0 conn(0x7f34680cfad0 0x7f34680cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f346d200700 1 -- 172.21.15.99:0/2590051776 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34680d2140 con 0x7f346815e7a0 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f346d200700 1 --2- 172.21.15.99:0/2590051776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346815e7a0 0x7f34680d1a40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f34680cd910 tx=0x7f345c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a904de76d941ac50 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f3467400700 1 -- 172.21.15.99:0/2590051776 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c011830 con 0x7f346815e7a0 2024-03-29T09:12:20.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f3467400700 1 -- 172.21.15.99:0/2590051776 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c011990 con 0x7f346815e7a0 2024-03-29T09:12:20.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.290+0000 7f3467400700 1 -- 172.21.15.99:0/2590051776 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c011d20 con 0x7f346815e7a0 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 -- 172.21.15.99:0/2590051776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346815e7a0 msgr2=0x7f34680d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 --2- 172.21.15.99:0/2590051776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346815e7a0 0x7f34680d1a40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f34680cd910 tx=0x7f345c009bd0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 -- 172.21.15.99:0/2590051776 shutdown_connections 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 --2- 172.21.15.99:0/2590051776 >> v2:172.21.15.133:3300/0 conn(0x7f34680cfad0 0x7f34680cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 --2- 172.21.15.99:0/2590051776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f346815e7a0 0x7f34680d1a40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 --2- 172.21.15.99:0/2590051776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f34680d12f0 0x7f34680cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 -- 172.21.15.99:0/2590051776 >> 172.21.15.99:0/2590051776 conn(0x7f346809c480 msgr2=0x7f34680d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 -- 172.21.15.99:0/2590051776 shutdown_connections 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.291+0000 7f346fc00700 1 -- 172.21.15.99:0/2590051776 wait complete. 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.293+0000 7f346fc00700 1 Processor -- start 2024-03-29T09:12:20.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.293+0000 7f346fc00700 1 -- start start 2024-03-29T09:12:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346fc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34680cfad0 0x7f34681507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346fc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34680d12f0 0x7f34680c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346fc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f346815e7a0 0x7f34680c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346fc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3468152d00 con 0x7f34680cfad0 2024-03-29T09:12:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346fc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34680b7350 con 0x7f34680d12f0 2024-03-29T09:12:20.295 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346fc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34680c97f0 con 0x7f346815e7a0 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34680cfad0 0x7f34681507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34680cfad0 0x7f34681507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46708/0 (socket says 172.21.15.99:46708) 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 -- 172.21.15.99:0/3703618745 learned_addr learned my addr 172.21.15.99:0/3703618745 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346dc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f346815e7a0 0x7f34680c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 -- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f346815e7a0 msgr2=0x7f34680c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f3467e00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34680d12f0 0x7f34680c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f346815e7a0 0x7f34680c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 -- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34680d12f0 msgr2=0x7f34680c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34680d12f0 0x7f34680c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346d200700 1 -- 172.21.15.99:0/3703618745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c009550 con 0x7f34680cfad0 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f346dc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f346815e7a0 0x7f34680c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.294+0000 7f3467e00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34680d12f0 0x7f34680c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.295+0000 7f346d200700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34680cfad0 0x7f34681507e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f345c009c10 tx=0x7f345c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:20.296 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.295+0000 7f3464c00700 1 -- 172.21.15.99:0/3703618745 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c004510 con 0x7f34680cfad0 2024-03-29T09:12:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.295+0000 7f3464c00700 1 -- 172.21.15.99:0/3703618745 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c01e070 con 0x7f34680cfad0 2024-03-29T09:12:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.295+0000 7f3464c00700 1 -- 172.21.15.99:0/3703618745 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c0038e0 con 0x7f34680cfad0 2024-03-29T09:12:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.295+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3468150d20 con 0x7f34680cfad0 2024-03-29T09:12:20.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.295+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34680c7620 con 0x7f34680cfad0 2024-03-29T09:12:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.296+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34680c7810 con 0x7f34680cfad0 2024-03-29T09:12:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.297+0000 7f3464c00700 1 -- 172.21.15.99:0/3703618745 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f345c003a40 con 0x7f34680cfad0 2024-03-29T09:12:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.298+0000 7f3464c00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f344806acc0 0x7f344806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.298+0000 7f3464c00700 1 -- 172.21.15.99:0/3703618745 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f345c08a350 con 0x7f34680cfad0 2024-03-29T09:12:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.298+0000 7f3467e00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f344806acc0 0x7f344806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.299+0000 7f3467e00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f344806acc0 0x7f344806d180 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3468157f00 tx=0x7f3454008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:20.302 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.302+0000 7f3464c00700 1 -- 172.21.15.99:0/3703618745 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f345c055350 con 0x7f34680cfad0 2024-03-29T09:12:20.506 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.506+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34680677a0 con 0x7f34680cfad0 2024-03-29T09:12:20.507 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.507+0000 7f3464c00700 1 -- 172.21.15.99:0/3703618745 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f345c058970 con 0x7f34680cfad0 2024-03-29T09:12:20.507 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:20.508 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:20.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f344806acc0 msgr2=0x7f344806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f344806acc0 0x7f344806d180 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3468157f00 tx=0x7f3454008040 comp rx=0 tx=0).stop 2024-03-29T09:12:20.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34680cfad0 msgr2=0x7f34681507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34680cfad0 0x7f34681507e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f345c009c10 tx=0x7f345c004750 comp rx=0 tx=0).stop 2024-03-29T09:12:20.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 shutdown_connections 2024-03-29T09:12:20.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f344806acc0 0x7f344806d180 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.510 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f346815e7a0 0x7f34680c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f34680cfad0 0x7f34681507e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 --2- 172.21.15.99:0/3703618745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34680d12f0 0x7f34680c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 >> 172.21.15.99:0/3703618745 conn(0x7f346809c480 msgr2=0x7f346806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:20.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 shutdown_connections 2024-03-29T09:12:20.511 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.510+0000 7f346fc00700 1 -- 172.21.15.99:0/3703618745 wait complete. 2024-03-29T09:12:20.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:11:41.952804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:11:42.633561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:12:20.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:12:20.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.720+0000 7f583a000700 1 Processor -- start 2024-03-29T09:12:20.720 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.720+0000 7f583a000700 1 -- start start 2024-03-29T09:12:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.721+0000 7f583a000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5834160770 0x7f58340b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.721+0000 7f583a000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340d1520 con 0x7f58340d0d60 2024-03-29T09:12:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.721+0000 7f583a000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340d1800 con 0x7f58340d3550 2024-03-29T09:12:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.721+0000 7f583a000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58340d1ae0 con 0x7f5834160770 2024-03-29T09:12:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.721+0000 7f5832a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f58340d0d60 0x7f58340d2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40944/0 (socket says 172.21.15.99:40944) 2024-03-29T09:12:20.721 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.721+0000 7f5832a00700 1 -- 172.21.15.99:0/1377293876 learned_addr learned my addr 172.21.15.99:0/1377293876 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5833400700 1 --2- 172.21.15.99:0/1377293876 >> v2:172.21.15.133:3300/0 conn(0x7f5834160770 0x7f58340b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775294741 0 0) 0x7f58340d1520 con 0x7f58340d0d60 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58180035b0 con 0x7f58340d0d60 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974604409 0 0) 0x7f58340d1800 con 0x7f58340d3550 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58340d1520 con 0x7f58340d3550 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1388652548 0 0) 0x7f58180035b0 con 0x7f58340d0d60 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 >> v2:172.21.15.133:3300/0 conn(0x7f5834160770 msgr2=0x7f58340b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 --2- 172.21.15.99:0/1377293876 >> v2:172.21.15.133:3300/0 conn(0x7f5834160770 0x7f58340b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 >> v1:172.21.15.133:6789/0 conn(0x7f58340d3550 legacy=0x7f58340ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340d1dc0 con 0x7f58340d0d60 2024-03-29T09:12:20.725 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f5820001670 con 0x7f58340d0d60 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 >> v1:172.21.15.99:6789/0 conn(0x7f58340d0d60 legacy=0x7f58340d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5818003be0 0x7f5818003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f5818004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5818005260 0x7f581800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340d1ae0 con 0x7f5818004850 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581800c190 con 0x7f5818003be0 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.722+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581800c350 con 0x7f5818005260 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832a00700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f5818004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5833400700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5818003be0 0x7f5818003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5833e00700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5818005260 0x7f581800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.726 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832a00700 1 -- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5818005260 msgr2=0x7f581800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832a00700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5818005260 0x7f581800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832a00700 1 -- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5818003be0 msgr2=0x7f5818003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832a00700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5818003be0 0x7f5818003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832a00700 1 -- 172.21.15.99:0/1377293876 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180035b0 con 0x7f5818004850 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5833e00700 1 -- 172.21.15.99:0/1377293876 reap_dead start 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832a00700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f5818004cd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f582000c110 tx=0x7f5818005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f4307ee36913fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000ccb0 con 0x7f5818004850 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582000ce10 con 0x7f5818004850 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.723+0000 7f5832000700 1 -- 172.21.15.99:0/1377293876 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820005ba0 con 0x7f5818004850 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.724+0000 7f583a000700 1 -- 172.21.15.99:0/1377293876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 msgr2=0x7f5818004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.724+0000 7f583a000700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f5818004cd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f582000c110 tx=0x7f5818005880 comp rx=0 tx=0).stop 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.724+0000 7f583a000700 1 -- 172.21.15.99:0/1377293876 shutdown_connections 2024-03-29T09:12:20.727 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.724+0000 7f583a000700 1 --2- 172.21.15.99:0/1377293876 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f5818004cd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.724+0000 7f583a000700 1 -- 172.21.15.99:0/1377293876 >> 172.21.15.99:0/1377293876 conn(0x7f583408afd0 msgr2=0x7f583409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.724+0000 7f583a000700 1 -- 172.21.15.99:0/1377293876 shutdown_connections 2024-03-29T09:12:20.728 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.724+0000 7f583a000700 1 -- 172.21.15.99:0/1377293876 wait complete. 2024-03-29T09:12:20.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.726+0000 7f583a000700 1 Processor -- start 2024-03-29T09:12:20.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.726+0000 7f583a000700 1 -- start start 2024-03-29T09:12:20.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f583a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f58340b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f583a000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58340cd1d0 0x7f58341f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.730 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f583a000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58341f3120 0x7f58340cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f583a000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58341f3c40 con 0x7f5818004850 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f583a000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583415cf70 con 0x7f58340cd1d0 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f583a000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340b02d0 con 0x7f58341f3120 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f5833400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f58340b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f5833400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f58340b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46728/0 (socket says 172.21.15.99:46728) 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f5833400700 1 -- 172.21.15.99:0/4214388208 learned_addr learned my addr 172.21.15.99:0/4214388208 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f5832a00700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58340cd1d0 0x7f58341f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f5833e00700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58341f3120 0x7f58340cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.727+0000 7f5833400700 1 -- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58341f3120 msgr2=0x7f58340cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f5833400700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58341f3120 0x7f58340cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f5833400700 1 -- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58340cd1d0 msgr2=0x7f58341f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.731 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f5833400700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58340cd1d0 0x7f58341f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f5833400700 1 -- 172.21.15.99:0/4214388208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58341f3680 con 0x7f5818004850 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f5833400700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f58340b0760 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5824011aa0 tx=0x7f5824011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f582b400700 1 -- 172.21.15.99:0/4214388208 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824011ee0 con 0x7f5818004850 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f582b400700 1 -- 172.21.15.99:0/4214388208 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582400f940 con 0x7f5818004850 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f582b400700 1 -- 172.21.15.99:0/4214388208 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400fcd0 con 0x7f5818004850 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58341f3960 con 0x7f5818004850 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.728+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58340b0ca0 con 0x7f5818004850 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.729+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58340b05b0 con 0x7f5818004850 2024-03-29T09:12:20.732 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.730+0000 7f582b400700 1 -- 172.21.15.99:0/4214388208 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f5824014440 con 0x7f5818004850 2024-03-29T09:12:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.731+0000 7f582b400700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f581406ac90 0x7f581406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.731+0000 7f582b400700 1 -- 172.21.15.99:0/4214388208 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5824089a30 con 0x7f5818004850 2024-03-29T09:12:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.732+0000 7f5832a00700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f581406ac90 0x7f581406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:20.733 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.732+0000 7f5832a00700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f581406ac90 0x7f581406d150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f582000c110 tx=0x7f5820007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:20.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.736+0000 7f582b400700 1 -- 172.21.15.99:0/4214388208 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5824054ba0 con 0x7f5818004850 2024-03-29T09:12:20.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.937+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58340afdc0 con 0x7f5818004850 2024-03-29T09:12:20.937 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.937+0000 7f582b400700 1 -- 172.21.15.99:0/4214388208 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58340afdc0 con 0x7f5818004850 2024-03-29T09:12:20.937 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.939+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f581406ac90 msgr2=0x7f581406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.939+0000 7f583a000700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f581406ac90 0x7f581406d150 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f582000c110 tx=0x7f5820007000 comp rx=0 tx=0).stop 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.939+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 msgr2=0x7f58340b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.939+0000 7f583a000700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f58340b0760 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5824011aa0 tx=0x7f5824011db0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.939+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 shutdown_connections 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.939+0000 7f583a000700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f581406ac90 0x7f581406d150 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.939+0000 7f583a000700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58341f3120 0x7f58340cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.940+0000 7f583a000700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5818004850 0x7f58340b0760 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.940+0000 7f583a000700 1 --2- 172.21.15.99:0/4214388208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58340cd1d0 0x7f58341f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.940+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 >> 172.21.15.99:0/4214388208 conn(0x7f583408afd0 msgr2=0x7f583415f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.940+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 shutdown_connections 2024-03-29T09:12:20.940 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:20.940+0000 7f583a000700 1 -- 172.21.15.99:0/4214388208 wait complete. 2024-03-29T09:12:20.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:12:21.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5049c00700 1 Processor -- start 2024-03-29T09:12:21.153 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5049c00700 1 -- start start 2024-03-29T09:12:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5049c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f50440b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5049c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f50440d3550 0x7f50440ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5049c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50440d1520 con 0x7f50440d0d60 2024-03-29T09:12:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5049c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50440d1800 con 0x7f50440d3550 2024-03-29T09:12:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5049c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440d1ae0 con 0x7f5044160770 2024-03-29T09:12:21.155 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5043400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f50440d3550 0x7f50440ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5043400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f50440d3550 0x7f50440ce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46732/0 (socket says 172.21.15.99:46732) 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5043400700 1 -- 172.21.15.99:0/1712144944 learned_addr learned my addr 172.21.15.99:0/1712144944 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.154+0000 7f5042a00700 1 --2- 172.21.15.99:0/1712144944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f50440b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5043400700 1 -- 172.21.15.99:0/1712144944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 msgr2=0x7f50440b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5043400700 1 --2- 172.21.15.99:0/1712144944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f50440b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5041600700 1 -- 172.21.15.99:0/1712144944 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628607211 0 0) 0x7f50440d1520 con 0x7f50440d0d60 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5043400700 1 -- 172.21.15.99:0/1712144944 >> v1:172.21.15.133:6789/0 conn(0x7f50440d0d60 legacy=0x7f50440d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5043400700 1 -- 172.21.15.99:0/1712144944 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440d1dc0 con 0x7f50440d3550 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5043400700 1 --2- 172.21.15.99:0/1712144944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d3550 0x7f50440ce080 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f503c009e50 tx=0x7f503c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6fe32fe39dc69 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5041600700 1 -- 172.21.15.99:0/1712144944 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00e070 con 0x7f50440d3550 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5041600700 1 -- 172.21.15.99:0/1712144944 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c00b910 con 0x7f50440d3550 2024-03-29T09:12:21.156 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.155+0000 7f5041600700 1 -- 172.21.15.99:0/1712144944 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c00bcc0 con 0x7f50440d3550 2024-03-29T09:12:21.157 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 -- 172.21.15.99:0/1712144944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d3550 msgr2=0x7f50440ce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 --2- 172.21.15.99:0/1712144944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d3550 0x7f50440ce080 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f503c009e50 tx=0x7f503c00b330 comp rx=0 tx=0).stop 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 -- 172.21.15.99:0/1712144944 shutdown_connections 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 --2- 172.21.15.99:0/1712144944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d3550 0x7f50440ce080 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 --2- 172.21.15.99:0/1712144944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f50440b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 -- 172.21.15.99:0/1712144944 >> 172.21.15.99:0/1712144944 conn(0x7f504408afd0 msgr2=0x7f504409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 -- 172.21.15.99:0/1712144944 shutdown_connections 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.156+0000 7f5049c00700 1 -- 172.21.15.99:0/1712144944 wait complete. 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.158+0000 7f5049c00700 1 Processor -- start 2024-03-29T09:12:21.158 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.158+0000 7f5049c00700 1 -- start start 2024-03-29T09:12:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5049c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d0d60 0x7f50440b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5049c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50440d3550 0x7f50441f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5049c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f504415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5049c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50441f2970 con 0x7f50440d0d60 2024-03-29T09:12:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5049c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440cb950 con 0x7f50440d3550 2024-03-29T09:12:21.160 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5049c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5044156fa0 con 0x7f5044160770 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d0d60 0x7f50440b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d0d60 0x7f50440b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46744/0 (socket says 172.21.15.99:46744) 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 -- 172.21.15.99:0/2647520791 learned_addr learned my addr 172.21.15.99:0/2647520791 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5043400700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f504415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042000700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50440d3550 0x7f50441f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 -- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 msgr2=0x7f504415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f504415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 -- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50440d3550 msgr2=0x7f50441f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50440d3550 0x7f50441f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 -- 172.21.15.99:0/2647520791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c009b30 con 0x7f50440d0d60 2024-03-29T09:12:21.161 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5043400700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f504415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042000700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50440d3550 0x7f50441f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.159+0000 7f5042a00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d0d60 0x7f50440b07d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f503400e880 tx=0x7f503400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.160+0000 7f503aa00700 1 -- 172.21.15.99:0/2647520791 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50340095b0 con 0x7f50440d0d60 2024-03-29T09:12:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.160+0000 7f503aa00700 1 -- 172.21.15.99:0/2647520791 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503400ee80 con 0x7f50440d0d60 2024-03-29T09:12:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.160+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50440cb4c0 con 0x7f50440d0d60 2024-03-29T09:12:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.160+0000 7f503aa00700 1 -- 172.21.15.99:0/2647520791 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400ca70 con 0x7f50440d0d60 2024-03-29T09:12:21.162 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.160+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50440cab90 con 0x7f50440d0d60 2024-03-29T09:12:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.161+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50440678b0 con 0x7f50440d0d60 2024-03-29T09:12:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.162+0000 7f503aa00700 1 -- 172.21.15.99:0/2647520791 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f503400cbd0 con 0x7f50440d0d60 2024-03-29T09:12:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.163+0000 7f503aa00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f502406ac90 0x7f502406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.163+0000 7f503aa00700 1 -- 172.21.15.99:0/2647520791 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f50340894d0 con 0x7f50440d0d60 2024-03-29T09:12:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.163+0000 7f5042000700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f502406ac90 0x7f502406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.163 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.163+0000 7f5042000700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f502406ac90 0x7f502406d150 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f60 tx=0x7f5030009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:21.167 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.167+0000 7f503aa00700 1 -- 172.21.15.99:0/2647520791 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f50340544d0 con 0x7f50440d0d60 2024-03-29T09:12:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.371+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50440d31c0 con 0x7f50440d0d60 2024-03-29T09:12:21.371 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.371+0000 7f503aa00700 1 -- 172.21.15.99:0/2647520791 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5034057af0 con 0x7f50440d0d60 2024-03-29T09:12:21.371 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:12:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f502406ac90 msgr2=0x7f502406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.374 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f502406ac90 0x7f502406d150 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5030000f60 tx=0x7f5030009040 comp rx=0 tx=0).stop 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d0d60 msgr2=0x7f50440b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d0d60 0x7f50440b07d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f503400e880 tx=0x7f503400ec40 comp rx=0 tx=0).stop 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 shutdown_connections 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f502406ac90 0x7f502406d150 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5044160770 0x7f504415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f50440d0d60 0x7f50440b07d0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 --2- 172.21.15.99:0/2647520791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50440d3550 0x7f50441f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 >> 172.21.15.99:0/2647520791 conn(0x7f504408afd0 msgr2=0x7f50440b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 shutdown_connections 2024-03-29T09:12:21.375 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.374+0000 7f5049c00700 1 -- 172.21.15.99:0/2647520791 wait complete. 2024-03-29T09:12:21.392 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:12:21.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0eaf200700 1 Processor -- start 2024-03-29T09:12:21.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0eaf200700 1 -- start start 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0eaf200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0ea80ae020 0x7f0ea815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0eaf200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea806c580 con 0x7f0ea80adc40 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0eaf200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea80abf10 con 0x7f0ea8072120 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0eaf200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea815c7a0 con 0x7f0ea80ae020 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0ea7e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0ea80adc40 0x7f0ea806ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40954/0 (socket says 172.21.15.99:40954) 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.586+0000 7f0ea7e00700 1 -- 172.21.15.99:0/1541499020 learned_addr learned my addr 172.21.15.99:0/1541499020 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ead200700 1 --2- 172.21.15.99:0/1541499020 >> v2:172.21.15.133:3300/0 conn(0x7f0ea80ae020 0x7f0ea815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414892983 0 0) 0x7f0ea806c580 con 0x7f0ea80adc40 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e8c0035b0 con 0x7f0ea80adc40 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996288337 0 0) 0x7f0ea80abf10 con 0x7f0ea8072120 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea806c580 con 0x7f0ea8072120 2024-03-29T09:12:21.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217626342 0 0) 0x7f0e8c0035b0 con 0x7f0ea80adc40 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 >> v2:172.21.15.133:3300/0 conn(0x7f0ea80ae020 msgr2=0x7f0ea815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 --2- 172.21.15.99:0/1541499020 >> v2:172.21.15.133:3300/0 conn(0x7f0ea80ae020 0x7f0ea815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 >> v1:172.21.15.133:6789/0 conn(0x7f0ea8072120 legacy=0x7f0ea8155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80ac1f0 con 0x7f0ea80adc40 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f0e98001670 con 0x7f0ea80adc40 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 >> v1:172.21.15.99:6789/0 conn(0x7f0ea80adc40 legacy=0x7f0ea806ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c003b90 0x7f0e8c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c004850 0x7f0e8c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0e8c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea815c7a0 con 0x7f0e8c005210 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c00c140 con 0x7f0e8c003b90 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.587+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c00c300 con 0x7f0e8c004850 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0e8c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ea7e00700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c003b90 0x7f0e8c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ea7400700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c004850 0x7f0e8c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 -- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c004850 msgr2=0x7f0e8c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c004850 0x7f0e8c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 -- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c003b90 msgr2=0x7f0e8c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c003b90 0x7f0e8c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 -- 172.21.15.99:0/1541499020 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0035b0 con 0x7f0e8c005210 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 -- 172.21.15.99:0/1541499020 reap_dead start 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ead200700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0e8c00ba10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009fc0 tx=0x7f0e8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740b25b53f9c10c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0003a20 con 0x7f0e8c005210 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000f040 con 0x7f0e8c005210 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.588+0000 7f0ea6a00700 1 -- 172.21.15.99:0/1541499020 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0003da0 con 0x7f0e8c005210 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.589+0000 7f0eaf200700 1 -- 172.21.15.99:0/1541499020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 msgr2=0x7f0e8c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.589+0000 7f0eaf200700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0e8c00ba10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009fc0 tx=0x7f0e8c005880 comp rx=0 tx=0).stop 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.589+0000 7f0eaf200700 1 -- 172.21.15.99:0/1541499020 shutdown_connections 2024-03-29T09:12:21.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.589+0000 7f0eaf200700 1 --2- 172.21.15.99:0/1541499020 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0e8c00ba10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.589+0000 7f0eaf200700 1 -- 172.21.15.99:0/1541499020 >> 172.21.15.99:0/1541499020 conn(0x7f0ea8066540 msgr2=0x7f0ea8066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.589+0000 7f0eaf200700 1 -- 172.21.15.99:0/1541499020 shutdown_connections 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.589+0000 7f0eaf200700 1 -- 172.21.15.99:0/1541499020 wait complete. 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.591+0000 7f0eaf200700 1 Processor -- start 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.591+0000 7f0eaf200700 1 -- start start 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0eaf200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0ea814aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0eaf200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea809a770 0x7f0ea809b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0eaf200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea809b7c0 0x7f0ea81583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0eaf200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea806c8d0 con 0x7f0e8c005210 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0eaf200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea809bc70 con 0x7f0ea809a770 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0eaf200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea814af60 con 0x7f0ea809b7c0 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0ea814aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0ea814aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46772/0 (socket says 172.21.15.99:46772) 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 -- 172.21.15.99:0/3967602274 learned_addr learned my addr 172.21.15.99:0/3967602274 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ead200700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea809b7c0 0x7f0ea81583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7400700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea809a770 0x7f0ea809b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 -- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea809b7c0 msgr2=0x7f0ea81583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea809b7c0 0x7f0ea81583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 -- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea809a770 msgr2=0x7f0ea809b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea809a770 0x7f0ea809b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.592+0000 7f0ea7e00700 1 -- 172.21.15.99:0/3967602274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea8099a40 con 0x7f0e8c005210 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.593+0000 7f0ea7e00700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0ea814aa20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0e98004fb0 tx=0x7f0e980073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.593+0000 7f0e9fe00700 1 -- 172.21.15.99:0/3967602274 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98011580 con 0x7f0e8c005210 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.593+0000 7f0e9fe00700 1 -- 172.21.15.99:0/3967602274 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e980116e0 con 0x7f0e8c005210 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.593+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea8159140 con 0x7f0e8c005210 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.593+0000 7f0e9fe00700 1 -- 172.21.15.99:0/3967602274 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98011a70 con 0x7f0e8c005210 2024-03-29T09:12:21.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.593+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea8099f70 con 0x7f0e8c005210 2024-03-29T09:12:21.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.594+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea80ac6b0 con 0x7f0e8c005210 2024-03-29T09:12:21.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.595+0000 7f0e9fe00700 1 -- 172.21.15.99:0/3967602274 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0e98011bd0 con 0x7f0e8c005210 2024-03-29T09:12:21.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.596+0000 7f0e9fe00700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e8806ac90 0x7f0e8806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:21.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.596+0000 7f0e9fe00700 1 -- 172.21.15.99:0/3967602274 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0e98088cf0 con 0x7f0e8c005210 2024-03-29T09:12:21.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.596+0000 7f0ea7400700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e8806ac90 0x7f0e8806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:21.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.596+0000 7f0ea7400700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e8806ac90 0x7f0e8806d150 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c00 tx=0x7f0e94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:21.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.600+0000 7f0e9fe00700 1 -- 172.21.15.99:0/3967602274 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0e98053d80 con 0x7f0e8c005210 2024-03-29T09:12:21.801 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.801+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ea80678b0 con 0x7f0e8c005210 2024-03-29T09:12:21.802 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.802+0000 7f0e9fe00700 1 -- 172.21.15.99:0/3967602274 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e9801a020 con 0x7f0e8c005210 2024-03-29T09:12:21.802 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e8806ac90 msgr2=0x7f0e8806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e8806ac90 0x7f0e8806d150 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0e94000c00 tx=0x7f0e94009040 comp rx=0 tx=0).stop 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 msgr2=0x7f0ea814aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0ea814aa20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f0e98004fb0 tx=0x7f0e980073f0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 shutdown_connections 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f0e8806ac90 0x7f0e8806d150 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea809b7c0 0x7f0ea81583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0e8c005210 0x7f0ea814aa20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 --2- 172.21.15.99:0/3967602274 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea809a770 0x7f0ea809b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 >> 172.21.15.99:0/3967602274 conn(0x7f0ea8066540 msgr2=0x7f0ea8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 shutdown_connections 2024-03-29T09:12:21.805 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:21.805+0000 7f0eaf200700 1 -- 172.21.15.99:0/3967602274 wait complete. 2024-03-29T09:12:21.823 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T09:12:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.014+0000 7fedfd200700 1 Processor -- start 2024-03-29T09:12:22.013 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.014+0000 7fedfd200700 1 -- start start 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedfd200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fedf80d1b20 0x7fedf80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedfd200700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fedf8160cc0 0x7fedf80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedfd200700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf80cddc0 con 0x7fedf81608e0 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedfd200700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf80ce0a0 con 0x7fedf8160cc0 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedfd200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf80ce380 con 0x7fedf80d1b20 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedf6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fedf8160cc0 0x7fedf80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedf6800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fedf8160cc0 0x7fedf80ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46774/0 (socket says 172.21.15.99:46774) 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.015+0000 7fedf6800700 1 -- 172.21.15.99:0/921540197 learned_addr learned my addr 172.21.15.99:0/921540197 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:22.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf5400700 1 --2- 172.21.15.99:0/921540197 >> v2:172.21.15.133:3300/0 conn(0x7fedf80d1b20 0x7fedf80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf6800700 1 -- 172.21.15.99:0/921540197 >> v2:172.21.15.133:3300/0 conn(0x7fedf80d1b20 msgr2=0x7fedf80d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf6800700 1 --2- 172.21.15.99:0/921540197 >> v2:172.21.15.133:3300/0 conn(0x7fedf80d1b20 0x7fedf80d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf6800700 1 -- 172.21.15.99:0/921540197 >> v1:172.21.15.133:6789/0 conn(0x7fedf81608e0 legacy=0x7fedf80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf6800700 1 -- 172.21.15.99:0/921540197 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80ce660 con 0x7fedf8160cc0 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf5400700 1 --2- 172.21.15.99:0/921540197 >> v2:172.21.15.133:3300/0 conn(0x7fedf80d1b20 0x7fedf80d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf6800700 1 --2- 172.21.15.99:0/921540197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf80ceb80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fedf806ac50 tx=0x7fedf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16db76e9e94a76bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf4a00700 1 -- 172.21.15.99:0/921540197 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000e070 con 0x7fedf8160cc0 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf4a00700 1 -- 172.21.15.99:0/921540197 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf0003a20 con 0x7fedf8160cc0 2024-03-29T09:12:22.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.016+0000 7fedf4a00700 1 -- 172.21.15.99:0/921540197 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0003dd0 con 0x7fedf8160cc0 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 -- 172.21.15.99:0/921540197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 msgr2=0x7fedf80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 --2- 172.21.15.99:0/921540197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf80ceb80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fedf806ac50 tx=0x7fedf000b330 comp rx=0 tx=0).stop 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 -- 172.21.15.99:0/921540197 shutdown_connections 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 --2- 172.21.15.99:0/921540197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf80ceb80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 --2- 172.21.15.99:0/921540197 >> v2:172.21.15.133:3300/0 conn(0x7fedf80d1b20 0x7fedf80d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 -- 172.21.15.99:0/921540197 >> 172.21.15.99:0/921540197 conn(0x7fedf809c490 msgr2=0x7fedf80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 -- 172.21.15.99:0/921540197 shutdown_connections 2024-03-29T09:12:22.017 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.017+0000 7fedfd200700 1 -- 172.21.15.99:0/921540197 wait complete. 2024-03-29T09:12:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.019+0000 7fedfd200700 1 Processor -- start 2024-03-29T09:12:22.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.019+0000 7fedfd200700 1 -- start start 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedfd200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedf80d1b20 0x7fedf81f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedfd200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fedf81608e0 0x7fedf814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedfd200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf81f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedfd200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf814cd70 con 0x7fedf8160cc0 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedfd200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf81f5150 con 0x7fedf80d1b20 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedfd200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf81f3d60 con 0x7fedf81608e0 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf81f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf81f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46776/0 (socket says 172.21.15.99:46776) 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 -- 172.21.15.99:0/2697497177 learned_addr learned my addr 172.21.15.99:0/2697497177 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf5e00700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedf80d1b20 0x7fedf81f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf5400700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fedf81608e0 0x7fedf814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 -- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fedf81608e0 msgr2=0x7fedf814c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fedf81608e0 0x7fedf814c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 -- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedf80d1b20 msgr2=0x7fedf81f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedf80d1b20 0x7fedf81f3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.021+0000 7fedf6800700 1 -- 172.21.15.99:0/2697497177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0009b30 con 0x7fedf8160cc0 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.022+0000 7fedf6800700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf81f3820 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fedf808c9a0 tx=0x7fedf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.022+0000 7fedee000700 1 -- 172.21.15.99:0/2697497177 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0004440 con 0x7fedf8160cc0 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.022+0000 7fedee000700 1 -- 172.21.15.99:0/2697497177 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf00092e0 con 0x7fedf8160cc0 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.022+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf81f4540 con 0x7fedf8160cc0 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.022+0000 7fedee000700 1 -- 172.21.15.99:0/2697497177 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf00047e0 con 0x7fedf8160cc0 2024-03-29T09:12:22.022 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.022+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf81f4b00 con 0x7fedf8160cc0 2024-03-29T09:12:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.023+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf81f5300 con 0x7fedf8160cc0 2024-03-29T09:12:22.023 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.024+0000 7fedee000700 1 -- 172.21.15.99:0/2697497177 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fedf001c440 con 0x7fedf8160cc0 2024-03-29T09:12:22.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.025+0000 7fedee000700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fedd806ac90 0x7fedd806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.025+0000 7fedee000700 1 -- 172.21.15.99:0/2697497177 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fedf008a260 con 0x7fedf8160cc0 2024-03-29T09:12:22.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.025+0000 7fedf5e00700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fedd806ac90 0x7fedd806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.025 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.026+0000 7fedf5e00700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fedd806ac90 0x7fedd806d150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fd0 tx=0x7fede8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:22.028 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.029+0000 7fedee000700 1 -- 172.21.15.99:0/2697497177 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fedf0055300 con 0x7fedf8160cc0 2024-03-29T09:12:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.248+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fedf80677a0 con 0x7fedf8160cc0 2024-03-29T09:12:22.247 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.248+0000 7fedee000700 1 -- 172.21.15.99:0/2697497177 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) v1 ==== 80+0+0 (secure 0 0 0) 0x7fedf0058920 con 0x7fedf8160cc0 2024-03-29T09:12:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fedd806ac90 msgr2=0x7fedd806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fedd806ac90 0x7fedd806d150 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fede8007fd0 tx=0x7fede8007ec0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 msgr2=0x7fedf81f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf81f3820 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fedf808c9a0 tx=0x7fedf0004ad0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 shutdown_connections 2024-03-29T09:12:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fedd806ac90 0x7fedd806d150 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.250 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedf8160cc0 0x7fedf81f3820 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedf80d1b20 0x7fedf81f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 --2- 172.21.15.99:0/2697497177 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fedf81608e0 0x7fedf814c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 >> 172.21.15.99:0/2697497177 conn(0x7fedf809c490 msgr2=0x7fedf815d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 shutdown_connections 2024-03-29T09:12:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.251+0000 7fedfd200700 1 -- 172.21.15.99:0/2697497177 wait complete. 2024-03-29T09:12:22.251 INFO:teuthology.orchestra.run.smithi099.stderr:listed 0 entries 2024-03-29T09:12:22.269 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi099.front.sepia.ceph.com... 2024-03-29T09:12:22.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:22.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T09:12:22.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi099.stderr:ceph-fuse[68573]: fuse finished with error 0 and tester_r 0 2024-03-29T09:12:22.334 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:12:22.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:22.334 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T09:12:22.409 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:12:22.409 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:22.456 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:12:22.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:12:22.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:22.456 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T09:12:22.539 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T09:12:22.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:12:22.540 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:12:22.541 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:22.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345608k,nr_inodes=4086402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-03-29T09:12:22.582 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:12:22.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:22.582 DEBUG:teuthology.orchestra.run.smithi099:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T09:12:22.663 INFO:teuthology.orchestra.run.smithi099.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T09:12:22.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:12:22.664 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:12:22.664 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:22.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:12:22.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:12:22.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:22.708 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T09:12:22.788 INFO:teuthology.orchestra.run.smithi133.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T09:12:22.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:12:22.789 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:12:22.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.979+0000 7fd7bdc00700 1 Processor -- start 2024-03-29T09:12:22.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.980+0000 7fd7bdc00700 1 -- start start 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7bdc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7bdc00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7b8160cc0 0x7fd7b80ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7bdc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80cddc0 con 0x7fd7b80d3a70 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7bdc00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b80ce0a0 con 0x7fd7b8160cc0 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7bdc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80ce380 con 0x7fd7b81608e0 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7b6000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7b8160cc0 0x7fd7b80ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7b6000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fd7b8160cc0 0x7fd7b80ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46788/0 (socket says 172.21.15.99:46788) 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7b6000700 1 -- 172.21.15.99:0/608748549 learned_addr learned my addr 172.21.15.99:0/608748549 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.981+0000 7fd7b6a00700 1 --2- 172.21.15.99:0/608748549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b6000700 1 -- 172.21.15.99:0/608748549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 msgr2=0x7fd7b80d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b6000700 1 --2- 172.21.15.99:0/608748549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b80d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b6000700 1 -- 172.21.15.99:0/608748549 >> v1:172.21.15.133:6789/0 conn(0x7fd7b80d3a70 legacy=0x7fd7b80d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b6000700 1 -- 172.21.15.99:0/608748549 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80ce660 con 0x7fd7b8160cc0 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b6000700 1 --2- 172.21.15.99:0/608748549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b80ceb80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40098f0 tx=0x7fd7a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a203a9bcb66da1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b5600700 1 -- 172.21.15.99:0/608748549 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4011830 con 0x7fd7b8160cc0 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b5600700 1 -- 172.21.15.99:0/608748549 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a4011990 con 0x7fd7b8160cc0 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7b5600700 1 -- 172.21.15.99:0/608748549 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4011d20 con 0x7fd7b8160cc0 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7bdc00700 1 -- 172.21.15.99:0/608748549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 msgr2=0x7fd7b80ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.982+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/608748549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b80ceb80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40098f0 tx=0x7fd7a4009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.983+0000 7fd7bdc00700 1 -- 172.21.15.99:0/608748549 shutdown_connections 2024-03-29T09:12:22.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.983+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/608748549 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b80d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.983+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/608748549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b80ceb80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.983+0000 7fd7bdc00700 1 -- 172.21.15.99:0/608748549 >> 172.21.15.99:0/608748549 conn(0x7fd7b809c490 msgr2=0x7fd7b80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.983+0000 7fd7bdc00700 1 -- 172.21.15.99:0/608748549 shutdown_connections 2024-03-29T09:12:22.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.983+0000 7fd7bdc00700 1 -- 172.21.15.99:0/608748549 wait complete. 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.984+0000 7fd7bdc00700 1 Processor -- start 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.985+0000 7fd7bdc00700 1 -- start start 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.985+0000 7fd7bdc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b80d3a70 0x7fd7b815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.985+0000 7fd7bdc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b8145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.985+0000 7fd7bdc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7bdc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b815ccc0 con 0x7fd7b8160cc0 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7bdc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80cc6b0 con 0x7fd7b80d3a70 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7bdc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80cbd80 con 0x7fd7b81608e0 2024-03-29T09:12:22.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b814b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46790/0 (socket says 172.21.15.99:46790) 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 -- 172.21.15.99:0/1985197165 learned_addr learned my addr 172.21.15.99:0/1985197165 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b6a00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b80d3a70 0x7fd7b815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b6000700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b8145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 -- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 msgr2=0x7fd7b8145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b8145f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 -- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b80d3a70 msgr2=0x7fd7b815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b80d3a70 0x7fd7b815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 -- 172.21.15.99:0/1985197165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4009550 con 0x7fd7b8160cc0 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b6a00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b80d3a70 0x7fd7b815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:22.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.986+0000 7fd7b7400700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b814b6a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000cc50 tx=0x7fd7b000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:22.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.987+0000 7fd7aea00700 1 -- 172.21.15.99:0/1985197165 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000ad80 con 0x7fd7b8160cc0 2024-03-29T09:12:22.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.987+0000 7fd7aea00700 1 -- 172.21.15.99:0/1985197165 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b000ce50 con 0x7fd7b8160cc0 2024-03-29T09:12:22.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.987+0000 7fd7aea00700 1 -- 172.21.15.99:0/1985197165 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0003c40 con 0x7fd7b8160cc0 2024-03-29T09:12:22.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.987+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b80cc060 con 0x7fd7b8160cc0 2024-03-29T09:12:22.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.987+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b80cba70 con 0x7fd7b8160cc0 2024-03-29T09:12:22.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.988+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b80cc840 con 0x7fd7b8160cc0 2024-03-29T09:12:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.989+0000 7fd7aea00700 1 -- 172.21.15.99:0/1985197165 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fd7b00050d0 con 0x7fd7b8160cc0 2024-03-29T09:12:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.990+0000 7fd7aea00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd79806ac90 0x7fd79806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.990+0000 7fd7aea00700 1 -- 172.21.15.99:0/1985197165 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fd7b0089630 con 0x7fd7b8160cc0 2024-03-29T09:12:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.990+0000 7fd7b6a00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd79806ac90 0x7fd79806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:22.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.991+0000 7fd7b6a00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd79806ac90 0x7fd79806d150 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009b60 tx=0x7fd7a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:22.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:22.995+0000 7fd7aea00700 1 -- 172.21.15.99:0/1985197165 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fd7b0054630 con 0x7fd7b8160cc0 2024-03-29T09:12:23.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.246+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b80677a0 con 0x7fd7b8160cc0 2024-03-29T09:12:23.246 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.247+0000 7fd7aea00700 1 -- 172.21.15.99:0/1985197165 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fd7b0057c50 con 0x7fd7b8160cc0 2024-03-29T09:12:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:23.246 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/790352630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":790352630},{"type":"v1","addr":"172.21.15.133:6833","nonce":790352630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1265449168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1265449168},{"type":"v1","addr":"172.21.15.133:6835","nonce":1265449168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6102,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1161439746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1161439746},{"type":"v1","addr":"172.21.15.99:6836","nonce":1161439746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:11:43.652592+0000","modified":"2024-03-29T09:11:46.662472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6105":{"gid":6105,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2396189132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2396189132},{"type":"v1","addr":"172.21.15.99:6837","nonce":2396189132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T09:12:23.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd79806ac90 msgr2=0x7fd79806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.248 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd79806ac90 0x7fd79806d150 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009b60 tx=0x7fd7a8008040 comp rx=0 tx=0).stop 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 msgr2=0x7fd7b814b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b814b6a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000cc50 tx=0x7fd7b000a340 comp rx=0 tx=0).stop 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 shutdown_connections 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fd79806ac90 0x7fd79806d150 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd7b8160cc0 0x7fd7b814b6a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7b80d3a70 0x7fd7b815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 --2- 172.21.15.99:0/1985197165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd7b81608e0 0x7fd7b8145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 >> 172.21.15.99:0/1985197165 conn(0x7fd7b809c490 msgr2=0x7fd7b80b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.249+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 shutdown_connections 2024-03-29T09:12:23.249 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.250+0000 7fd7bdc00700 1 -- 172.21.15.99:0/1985197165 wait complete. 2024-03-29T09:12:23.250 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-03-29T09:12:23.267 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:23.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:12:23.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:12:23.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.462+0000 7efdfa800700 1 Processor -- start 2024-03-29T09:12:23.461 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.462+0000 7efdfa800700 1 -- start start 2024-03-29T09:12:23.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.463+0000 7efdfa800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efdf415ae70 0x7efdf415b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.463+0000 7efdfa800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf40b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.463+0000 7efdfa800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efdf40b36f0 0x7efdf40b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.463+0000 7efdfa800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf40b14f0 con 0x7efdf40b36f0 2024-03-29T09:12:23.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.463+0000 7efdfa800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdf415c220 con 0x7efdf415ae70 2024-03-29T09:12:23.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.463+0000 7efdfa800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf40b40c0 con 0x7efdf40b0b50 2024-03-29T09:12:23.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.463+0000 7efdf3e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efdf40b36f0 0x7efdf40b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7efdf40b36f0 0x7efdf40b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46792/0 (socket says 172.21.15.99:46792) 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 -- 172.21.15.99:0/3394412450 learned_addr learned my addr 172.21.15.99:0/3394412450 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf2a00700 1 --2- 172.21.15.99:0/3394412450 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf40b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 -- 172.21.15.99:0/3394412450 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 msgr2=0x7efdf40b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 --2- 172.21.15.99:0/3394412450 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf40b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3400700 1 --2- 172.21.15.99:0/3394412450 >> v2:172.21.15.133:3300/0 conn(0x7efdf415ae70 0x7efdf415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 -- 172.21.15.99:0/3394412450 >> v2:172.21.15.133:3300/0 conn(0x7efdf415ae70 msgr2=0x7efdf415b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 --2- 172.21.15.99:0/3394412450 >> v2:172.21.15.133:3300/0 conn(0x7efdf415ae70 0x7efdf415b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 -- 172.21.15.99:0/3394412450 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf415b8f0 con 0x7efdf40b36f0 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3400700 1 --2- 172.21.15.99:0/3394412450 >> v2:172.21.15.133:3300/0 conn(0x7efdf415ae70 0x7efdf415b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf3e00700 1 --2- 172.21.15.99:0/3394412450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf40b3b80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7efdf4067330 tx=0x7efdec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1043b39958af202 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf2000700 1 -- 172.21.15.99:0/3394412450 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00e070 con 0x7efdf40b36f0 2024-03-29T09:12:23.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf2000700 1 -- 172.21.15.99:0/3394412450 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec00b9c0 con 0x7efdf40b36f0 2024-03-29T09:12:23.465 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.464+0000 7efdf2000700 1 -- 172.21.15.99:0/3394412450 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec00bd70 con 0x7efdf40b36f0 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 -- 172.21.15.99:0/3394412450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 msgr2=0x7efdf40b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 --2- 172.21.15.99:0/3394412450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf40b3b80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7efdf4067330 tx=0x7efdec00b330 comp rx=0 tx=0).stop 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 -- 172.21.15.99:0/3394412450 shutdown_connections 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 --2- 172.21.15.99:0/3394412450 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf40b3b80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 --2- 172.21.15.99:0/3394412450 >> v2:172.21.15.133:3300/0 conn(0x7efdf415ae70 0x7efdf415b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 --2- 172.21.15.99:0/3394412450 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf40b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 -- 172.21.15.99:0/3394412450 >> 172.21.15.99:0/3394412450 conn(0x7efdf4066540 msgr2=0x7efdf4066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 -- 172.21.15.99:0/3394412450 shutdown_connections 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.465+0000 7efdfa800700 1 -- 172.21.15.99:0/3394412450 wait complete. 2024-03-29T09:12:23.466 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.467+0000 7efdfa800700 1 Processor -- start 2024-03-29T09:12:23.467 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.467+0000 7efdfa800700 1 -- start start 2024-03-29T09:12:23.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdfa800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf4157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdfa800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf41f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdfa800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf415ae70 0x7efdf40af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdfa800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4142290 con 0x7efdf40b36f0 2024-03-29T09:12:23.468 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdfa800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf4157090 con 0x7efdf415ae70 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdfa800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf41453c0 con 0x7efdf40b0b50 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdf2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf41f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdf3400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf4157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdf2a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf41f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46796/0 (socket says 172.21.15.99:46796) 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdf2a00700 1 -- 172.21.15.99:0/1408416123 learned_addr learned my addr 172.21.15.99:0/1408416123 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.468+0000 7efdf3e00700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf415ae70 0x7efdf40af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdf2a00700 1 -- 172.21.15.99:0/1408416123 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 msgr2=0x7efdf4157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdf2a00700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf4157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdf2a00700 1 -- 172.21.15.99:0/1408416123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf415ae70 msgr2=0x7efdf40af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdf2a00700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf415ae70 0x7efdf40af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdf2a00700 1 -- 172.21.15.99:0/1408416123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec009b30 con 0x7efdf40b36f0 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdf2a00700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf41f2250 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7efde000e810 tx=0x7efde000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdeb400700 1 -- 172.21.15.99:0/1408416123 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000c070 con 0x7efdf40b36f0 2024-03-29T09:12:23.469 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdeb400700 1 -- 172.21.15.99:0/1408416123 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde0013610 con 0x7efdf40b36f0 2024-03-29T09:12:23.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf4157330 con 0x7efdf40b36f0 2024-03-29T09:12:23.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdeb400700 1 -- 172.21.15.99:0/1408416123 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde00139c0 con 0x7efdf40b36f0 2024-03-29T09:12:23.470 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.469+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf41578f0 con 0x7efdf40b36f0 2024-03-29T09:12:23.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.470+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf4157c60 con 0x7efdf40b36f0 2024-03-29T09:12:23.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.471+0000 7efdeb400700 1 -- 172.21.15.99:0/1408416123 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7efde0013b20 con 0x7efdf40b36f0 2024-03-29T09:12:23.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.472+0000 7efdeb400700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efdd406ac90 0x7efdd406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.472+0000 7efdeb400700 1 -- 172.21.15.99:0/1408416123 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efde0089210 con 0x7efdf40b36f0 2024-03-29T09:12:23.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.472+0000 7efdf3400700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efdd406ac90 0x7efdd406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.471 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.473+0000 7efdf3400700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efdd406ac90 0x7efdd406d150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efde4006fd0 tx=0x7efde4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:23.475 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.476+0000 7efdeb400700 1 -- 172.21.15.99:0/1408416123 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7efde0054220 con 0x7efdf40b36f0 2024-03-29T09:12:23.683 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.684+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdf4068060 con 0x7efdf40b36f0 2024-03-29T09:12:23.685 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.686+0000 7efdeb400700 1 -- 172.21.15.99:0/1408416123 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7efde0057840 con 0x7efdf40b36f0 2024-03-29T09:12:23.685 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:23.685 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:23.687 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efdd406ac90 msgr2=0x7efdd406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efdd406ac90 0x7efdd406d150 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7efde4006fd0 tx=0x7efde4008040 comp rx=0 tx=0).stop 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 msgr2=0x7efdf41f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf41f2250 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7efde000e810 tx=0x7efde000eb20 comp rx=0 tx=0).stop 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 shutdown_connections 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7efdd406ac90 0x7efdd406d150 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdf415ae70 0x7efdf40af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efdf40b0b50 0x7efdf4157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 --2- 172.21.15.99:0/1408416123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdf40b36f0 0x7efdf41f2250 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 >> 172.21.15.99:0/1408416123 conn(0x7efdf4066540 msgr2=0x7efdf40ada60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.688+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 shutdown_connections 2024-03-29T09:12:23.688 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.689+0000 7efdfa800700 1 -- 172.21.15.99:0/1408416123 wait complete. 2024-03-29T09:12:23.706 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T09:08:46.511608+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T09:11:41.952804+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T09:11:42.633561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T09:12:23.706 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T09:12:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.901+0000 7fe30cc00700 1 Processor -- start 2024-03-29T09:12:23.900 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.901+0000 7fe30cc00700 1 -- start start 2024-03-29T09:12:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.902+0000 7fe30cc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.902+0000 7fe30cc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe3080d3550 0x7fe3080ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.902+0000 7fe30cc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3080d1520 con 0x7fe3080d0d60 2024-03-29T09:12:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.902+0000 7fe30cc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3080d1800 con 0x7fe3080d3550 2024-03-29T09:12:23.901 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.902+0000 7fe30cc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080d1ae0 con 0x7fe308160770 2024-03-29T09:12:23.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:48792/0 (socket says 172.21.15.99:48792) 2024-03-29T09:12:23.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 -- 172.21.15.99:0/2839005652 learned_addr learned my addr 172.21.15.99:0/2839005652 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:23.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe306800700 1 --2- 172.21.15.99:0/2839005652 >> v2:172.21.15.133:3300/0 conn(0x7fe3080d3550 0x7fe3080ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.903 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 -- 172.21.15.99:0/2839005652 >> v2:172.21.15.133:3300/0 conn(0x7fe3080d3550 msgr2=0x7fe3080ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 --2- 172.21.15.99:0/2839005652 >> v2:172.21.15.133:3300/0 conn(0x7fe3080d3550 0x7fe3080ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 -- 172.21.15.99:0/2839005652 >> v1:172.21.15.133:6789/0 conn(0x7fe3080d0d60 legacy=0x7fe3080d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 -- 172.21.15.99:0/2839005652 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3080d1dc0 con 0x7fe308160770 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe304a00700 1 -- 172.21.15.99:0/2839005652 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086118638 0 0) 0x7fe3080d1520 con 0x7fe3080d0d60 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe306800700 1 --2- 172.21.15.99:0/2839005652 >> v2:172.21.15.133:3300/0 conn(0x7fe3080d3550 0x7fe3080ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.903+0000 7fe305e00700 1 --2- 172.21.15.99:0/2839005652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080b3cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80098f0 tx=0x7fe2f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=588fe8e8bdc9efcb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe304a00700 1 -- 172.21.15.99:0/2839005652 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8011830 con 0x7fe308160770 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe304a00700 1 -- 172.21.15.99:0/2839005652 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f8011990 con 0x7fe308160770 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe304a00700 1 -- 172.21.15.99:0/2839005652 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8011d20 con 0x7fe308160770 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 -- 172.21.15.99:0/2839005652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 msgr2=0x7fe3080b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2839005652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080b3cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80098f0 tx=0x7fe2f8009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 -- 172.21.15.99:0/2839005652 shutdown_connections 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2839005652 >> v2:172.21.15.133:3300/0 conn(0x7fe3080d3550 0x7fe3080ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2839005652 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080b3cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 -- 172.21.15.99:0/2839005652 >> 172.21.15.99:0/2839005652 conn(0x7fe30808afd0 msgr2=0x7fe30809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:23.904 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 -- 172.21.15.99:0/2839005652 shutdown_connections 2024-03-29T09:12:23.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.904+0000 7fe30cc00700 1 -- 172.21.15.99:0/2839005652 wait complete. 2024-03-29T09:12:23.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.906+0000 7fe30cc00700 1 Processor -- start 2024-03-29T09:12:23.905 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.906+0000 7fe30cc00700 1 -- start start 2024-03-29T09:12:23.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe30cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080d0d60 0x7fe3080a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.907 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe30cc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3080d3550 0x7fe3080b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe30cc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe30cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080b7310 con 0x7fe3080d0d60 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe30cc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080b75f0 con 0x7fe3080d3550 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe30cc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3080b78d0 con 0x7fe308160770 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe305e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080d0d60 0x7fe3080a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe305e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080d0d60 0x7fe3080a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46812/0 (socket says 172.21.15.99:46812) 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.907+0000 7fe305e00700 1 -- 172.21.15.99:0/2418419672 learned_addr learned my addr 172.21.15.99:0/2418419672 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305400700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3080d3550 0x7fe3080b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe306800700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305e00700 1 -- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 msgr2=0x7fe3080cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305e00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305e00700 1 -- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3080d3550 msgr2=0x7fe3080b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305e00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3080d3550 0x7fe3080b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305e00700 1 -- 172.21.15.99:0/2418419672 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe300009a70 con 0x7fe3080d0d60 2024-03-29T09:12:23.908 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305400700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3080d3550 0x7fe3080b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:23.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe305e00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080d0d60 0x7fe3080a08f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009c40 tx=0x7fe2f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:23.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe2fe000700 1 -- 172.21.15.99:0/2418419672 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8004590 con 0x7fe3080d0d60 2024-03-29T09:12:23.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.908+0000 7fe2fe000700 1 -- 172.21.15.99:0/2418419672 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f801e070 con 0x7fe3080d0d60 2024-03-29T09:12:23.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.909+0000 7fe2fe000700 1 -- 172.21.15.99:0/2418419672 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80038c0 con 0x7fe3080d0d60 2024-03-29T09:12:23.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.909+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f8009550 con 0x7fe3080d0d60 2024-03-29T09:12:23.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.909+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3080b7e90 con 0x7fe3080d0d60 2024-03-29T09:12:23.909 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.910+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3080678b0 con 0x7fe3080d0d60 2024-03-29T09:12:23.910 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.911+0000 7fe2fe000700 1 -- 172.21.15.99:0/2418419672 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe2f8003a20 con 0x7fe3080d0d60 2024-03-29T09:12:23.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.912+0000 7fe2fe000700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2e806ac90 0x7fe2e806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:23.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.912+0000 7fe2fe000700 1 -- 172.21.15.99:0/2418419672 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe2f808a440 con 0x7fe3080d0d60 2024-03-29T09:12:23.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.912+0000 7fe305400700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2e806ac90 0x7fe2e806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:23.911 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.913+0000 7fe305400700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2e806ac90 0x7fe2e806d150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3080d0480 tx=0x7fe2f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:23.915 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:23.916+0000 7fe2fe000700 1 -- 172.21.15.99:0/2418419672 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe2f80554d0 con 0x7fe3080d0d60 2024-03-29T09:12:24.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.119+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe308073370 con 0x7fe3080d0d60 2024-03-29T09:12:24.118 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.119+0000 7fe2fe000700 1 -- 172.21.15.99:0/2418419672 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2f8058af0 con 0x7fe3080d0d60 2024-03-29T09:12:24.118 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 1 2024-03-29T09:12:24.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2e806ac90 msgr2=0x7fe2e806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2e806ac90 0x7fe2e806d150 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe3080d0480 tx=0x7fe2f4009040 comp rx=0 tx=0).stop 2024-03-29T09:12:24.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080d0d60 msgr2=0x7fe3080a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080d0d60 0x7fe3080a08f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009c40 tx=0x7fe2f8004750 comp rx=0 tx=0).stop 2024-03-29T09:12:24.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 shutdown_connections 2024-03-29T09:12:24.121 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe2e806ac90 0x7fe2e806d150 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe308160770 0x7fe3080cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3080d0d60 0x7fe3080a08f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 --2- 172.21.15.99:0/2418419672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3080d3550 0x7fe3080b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 >> 172.21.15.99:0/2418419672 conn(0x7fe30808afd0 msgr2=0x7fe3080b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:24.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 shutdown_connections 2024-03-29T09:12:24.122 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.122+0000 7fe30cc00700 1 -- 172.21.15.99:0/2418419672 wait complete. 2024-03-29T09:12:24.140 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T09:12:24.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.335+0000 7f3519000700 1 Processor -- start 2024-03-29T09:12:24.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.335+0000 7f3519000700 1 -- start start 2024-03-29T09:12:24.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.336+0000 7f3519000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140b3cc0 0x7f35140d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.336+0000 7f3519000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35140d3810 con 0x7f35140d3380 2024-03-29T09:12:24.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.336+0000 7f3519000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35140d1520 con 0x7f3514160770 2024-03-29T09:12:24.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.336+0000 7f3519000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35140b4140 con 0x7f35140b3cc0 2024-03-29T09:12:24.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.336+0000 7f3512800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f35140d3380 0x7f35140ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40968/0 (socket says 172.21.15.99:40968) 2024-03-29T09:12:24.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.336+0000 7f3512800700 1 -- 172.21.15.99:0/947969812 learned_addr learned my addr 172.21.15.99:0/947969812 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:24.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3511400700 1 --2- 172.21.15.99:0/947969812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140b3cc0 0x7f35140d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3510a00700 1 -- 172.21.15.99:0/947969812 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783224540 0 0) 0x7f35140d3810 con 0x7f35140d3380 2024-03-29T09:12:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3510a00700 1 -- 172.21.15.99:0/947969812 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f80035b0 con 0x7f35140d3380 2024-03-29T09:12:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3510a00700 1 -- 172.21.15.99:0/947969812 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330437225 0 0) 0x7f35140d1520 con 0x7f3514160770 2024-03-29T09:12:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3510a00700 1 -- 172.21.15.99:0/947969812 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35140d3810 con 0x7f3514160770 2024-03-29T09:12:24.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3511400700 1 -- 172.21.15.99:0/947969812 >> v1:172.21.15.133:6789/0 conn(0x7f3514160770 legacy=0x7f35140d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3511400700 1 -- 172.21.15.99:0/947969812 >> v1:172.21.15.99:6789/0 conn(0x7f35140d3380 legacy=0x7f35140ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3511400700 1 -- 172.21.15.99:0/947969812 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35140d1800 con 0x7f35140b3cc0 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.337+0000 7f3511400700 1 --2- 172.21.15.99:0/947969812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140b3cc0 0x7f35140d2e40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3500009870 tx=0x7f3500009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d83b2f204a3caf2f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3510a00700 1 -- 172.21.15.99:0/947969812 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500011830 con 0x7f35140b3cc0 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3510a00700 1 -- 172.21.15.99:0/947969812 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500011990 con 0x7f35140b3cc0 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3510a00700 1 -- 172.21.15.99:0/947969812 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500011d20 con 0x7f35140b3cc0 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3519000700 1 -- 172.21.15.99:0/947969812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140b3cc0 msgr2=0x7f35140d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3519000700 1 --2- 172.21.15.99:0/947969812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140b3cc0 0x7f35140d2e40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3500009870 tx=0x7f3500009b80 comp rx=0 tx=0).stop 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3519000700 1 -- 172.21.15.99:0/947969812 shutdown_connections 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3519000700 1 --2- 172.21.15.99:0/947969812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140b3cc0 0x7f35140d2e40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3519000700 1 -- 172.21.15.99:0/947969812 >> 172.21.15.99:0/947969812 conn(0x7f351408afd0 msgr2=0x7f351409c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3519000700 1 -- 172.21.15.99:0/947969812 shutdown_connections 2024-03-29T09:12:24.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.338+0000 7f3519000700 1 -- 172.21.15.99:0/947969812 wait complete. 2024-03-29T09:12:24.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.340+0000 7f3519000700 1 Processor -- start 2024-03-29T09:12:24.339 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.340+0000 7f3519000700 1 -- start start 2024-03-29T09:12:24.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3519000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35140b3cc0 0x7f35140c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3519000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140d3380 0x7f351415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3519000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3514160770 0x7f351414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.341 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3519000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140cb690 con 0x7f3514160770 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3519000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351414a190 con 0x7f35140b3cc0 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3519000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351415cbe0 con 0x7f35140d3380 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3512800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3514160770 0x7f351414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3512800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3514160770 0x7f351414c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46828/0 (socket says 172.21.15.99:46828) 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3512800700 1 -- 172.21.15.99:0/809340541 learned_addr learned my addr 172.21.15.99:0/809340541 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3511400700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140d3380 0x7f351415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.341+0000 7f3511e00700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35140b3cc0 0x7f35140c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3512800700 1 -- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140d3380 msgr2=0x7f351415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3512800700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140d3380 0x7f351415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3512800700 1 -- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35140b3cc0 msgr2=0x7f35140c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3512800700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35140b3cc0 0x7f35140c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3512800700 1 -- 172.21.15.99:0/809340541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000094a0 con 0x7f3514160770 2024-03-29T09:12:24.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3512800700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3514160770 0x7f351414c250 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f350c00cba0 tx=0x7f350c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:24.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f350a000700 1 -- 172.21.15.99:0/809340541 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00aa40 con 0x7f3514160770 2024-03-29T09:12:24.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f350a000700 1 -- 172.21.15.99:0/809340541 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c00aba0 con 0x7f3514160770 2024-03-29T09:12:24.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351415cec0 con 0x7f3514160770 2024-03-29T09:12:24.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f350a000700 1 -- 172.21.15.99:0/809340541 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c012400 con 0x7f3514160770 2024-03-29T09:12:24.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.342+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35140cb2b0 con 0x7f3514160770 2024-03-29T09:12:24.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.343+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35140cb7d0 con 0x7f3514160770 2024-03-29T09:12:24.343 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.344+0000 7f350a000700 1 -- 172.21.15.99:0/809340541 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f350c01b3b0 con 0x7f3514160770 2024-03-29T09:12:24.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.346+0000 7f350a000700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f34f406ac90 0x7f34f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.346+0000 7f350a000700 1 -- 172.21.15.99:0/809340541 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f350c089120 con 0x7f3514160770 2024-03-29T09:12:24.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.346+0000 7f3511e00700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f34f406ac90 0x7f34f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.345 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.347+0000 7f3511e00700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f34f406ac90 0x7f34f406d150 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f35040011e0 tx=0x7f3504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:24.349 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.350+0000 7f350a000700 1 -- 172.21.15.99:0/809340541 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f35140cb7d0 con 0x7f3514160770 2024-03-29T09:12:24.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.554+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35140cb7d0 con 0x7f3514160770 2024-03-29T09:12:24.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.555+0000 7f350a000700 1 -- 172.21.15.99:0/809340541 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35140cb7d0 con 0x7f3514160770 2024-03-29T09:12:24.554 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 32 2024-03-29T09:12:24.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f34f406ac90 msgr2=0x7f34f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f34f406ac90 0x7f34f406d150 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f35040011e0 tx=0x7f3504009040 comp rx=0 tx=0).stop 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3514160770 msgr2=0x7f351414c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3514160770 0x7f351414c250 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f350c00cba0 tx=0x7f350c00ceb0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 shutdown_connections 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f34f406ac90 0x7f34f406d150 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3514160770 0x7f351414c250 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35140b3cc0 0x7f35140c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 --2- 172.21.15.99:0/809340541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35140d3380 0x7f351415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 >> 172.21.15.99:0/809340541 conn(0x7f351408afd0 msgr2=0x7f351406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 shutdown_connections 2024-03-29T09:12:24.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.557+0000 7f3519000700 1 -- 172.21.15.99:0/809340541 wait complete. 2024-03-29T09:12:24.575 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T09:12:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.767+0000 7f1a4d400700 1 Processor -- start 2024-03-29T09:12:24.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.767+0000 7f1a4d400700 1 -- start start 2024-03-29T09:12:24.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.768+0000 7f1a4d400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a48160770 0x7f1a480b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.768+0000 7f1a4d400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a480d1520 con 0x7f1a480d3550 2024-03-29T09:12:24.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.768+0000 7f1a4d400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a480d1800 con 0x7f1a480d0d60 2024-03-29T09:12:24.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.768+0000 7f1a4d400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480d1ae0 con 0x7f1a48160770 2024-03-29T09:12:24.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a47200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1a480d3550 0x7f1a480ce080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40984/0 (socket says 172.21.15.99:40984) 2024-03-29T09:12:24.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a47200700 1 -- 172.21.15.99:0/1982550489 learned_addr learned my addr 172.21.15.99:0/1982550489 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:24.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a46800700 1 --2- 172.21.15.99:0/1982550489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a48160770 0x7f1a480b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.769 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a45400700 1 -- 172.21.15.99:0/1982550489 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218974526 0 0) 0x7f1a480d1520 con 0x7f1a480d3550 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a45400700 1 -- 172.21.15.99:0/1982550489 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a2c0035b0 con 0x7f1a480d3550 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a45400700 1 -- 172.21.15.99:0/1982550489 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314463992 0 0) 0x7f1a480d1800 con 0x7f1a480d0d60 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a45400700 1 -- 172.21.15.99:0/1982550489 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a480d1520 con 0x7f1a480d0d60 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a46800700 1 -- 172.21.15.99:0/1982550489 >> v1:172.21.15.133:6789/0 conn(0x7f1a480d0d60 legacy=0x7f1a480d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a46800700 1 -- 172.21.15.99:0/1982550489 >> v1:172.21.15.99:6789/0 conn(0x7f1a480d3550 legacy=0x7f1a480ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.769+0000 7f1a46800700 1 -- 172.21.15.99:0/1982550489 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480d1dc0 con 0x7f1a48160770 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.770+0000 7f1a46800700 1 --2- 172.21.15.99:0/1982550489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a48160770 0x7f1a480b3cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a38009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ec45b6a5fc9e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.770+0000 7f1a45400700 1 -- 172.21.15.99:0/1982550489 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38011830 con 0x7f1a48160770 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.770+0000 7f1a45400700 1 -- 172.21.15.99:0/1982550489 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a38011990 con 0x7f1a48160770 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.770+0000 7f1a45400700 1 -- 172.21.15.99:0/1982550489 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38011d00 con 0x7f1a48160770 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.770+0000 7f1a4d400700 1 -- 172.21.15.99:0/1982550489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a48160770 msgr2=0x7f1a480b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.770+0000 7f1a4d400700 1 --2- 172.21.15.99:0/1982550489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a48160770 0x7f1a480b3cc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a38009b80 comp rx=0 tx=0).stop 2024-03-29T09:12:24.770 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.771+0000 7f1a4d400700 1 -- 172.21.15.99:0/1982550489 shutdown_connections 2024-03-29T09:12:24.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.771+0000 7f1a4d400700 1 --2- 172.21.15.99:0/1982550489 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a48160770 0x7f1a480b3cc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.771+0000 7f1a4d400700 1 -- 172.21.15.99:0/1982550489 >> 172.21.15.99:0/1982550489 conn(0x7f1a4808afd0 msgr2=0x7f1a4809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:24.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.771+0000 7f1a4d400700 1 -- 172.21.15.99:0/1982550489 shutdown_connections 2024-03-29T09:12:24.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.771+0000 7f1a4d400700 1 -- 172.21.15.99:0/1982550489 wait complete. 2024-03-29T09:12:24.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.772+0000 7f1a4d400700 1 Processor -- start 2024-03-29T09:12:24.772 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.772+0000 7f1a4d400700 1 -- start start 2024-03-29T09:12:24.773 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.773+0000 7f1a4d400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a480d0d60 0x7f1a480b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.773+0000 7f1a4d400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a480d3550 0x7f1a4806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.773+0000 7f1a4d400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a48160770 0x7f1a480b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.773+0000 7f1a4d400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480cbce0 con 0x7f1a480d0d60 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.773+0000 7f1a4d400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480cbf10 con 0x7f1a48160770 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.773+0000 7f1a4d400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480cc1f0 con 0x7f1a480d3550 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a480d0d60 0x7f1a480b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a480d0d60 0x7f1a480b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46830/0 (socket says 172.21.15.99:46830) 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 -- 172.21.15.99:0/924043252 learned_addr learned my addr 172.21.15.99:0/924043252 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a45e00700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a480d3550 0x7f1a4806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 -- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a480d3550 msgr2=0x7f1a4806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a480d3550 0x7f1a4806b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.774 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 -- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a48160770 msgr2=0x7f1a480b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a48160770 0x7f1a480b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 -- 172.21.15.99:0/924043252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a380094a0 con 0x7f1a480d0d60 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a45e00700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a480d3550 0x7f1a4806b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.774+0000 7f1a46800700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a480d0d60 0x7f1a480b18d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1a38014040 tx=0x7f1a38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.775+0000 7f1a3ea00700 1 -- 172.21.15.99:0/924043252 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a380045c0 con 0x7f1a480d0d60 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.775+0000 7f1a3ea00700 1 -- 172.21.15.99:0/924043252 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3801e070 con 0x7f1a480d0d60 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.775+0000 7f1a3ea00700 1 -- 172.21.15.99:0/924043252 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38003900 con 0x7f1a480d0d60 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.775+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a480cc4d0 con 0x7f1a480d0d60 2024-03-29T09:12:24.775 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.775+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a480cca90 con 0x7f1a480d0d60 2024-03-29T09:12:24.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.776+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a480678b0 con 0x7f1a480d0d60 2024-03-29T09:12:24.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.776+0000 7f1a3ea00700 1 -- 172.21.15.99:0/924043252 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1a38003a60 con 0x7f1a480d0d60 2024-03-29T09:12:24.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.777+0000 7f1a3ea00700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a2806ac90 0x7f1a2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:24.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.777+0000 7f1a3ea00700 1 -- 172.21.15.99:0/924043252 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1a3808a410 con 0x7f1a480d0d60 2024-03-29T09:12:24.776 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.778+0000 7f1a45e00700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a2806ac90 0x7f1a2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:24.777 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.778+0000 7f1a45e00700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a2806ac90 0x7f1a2806d150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1a340011e0 tx=0x7f1a34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:24.780 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.782+0000 7f1a3ea00700 1 -- 172.21.15.99:0/924043252 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1a380554a0 con 0x7f1a480d0d60 2024-03-29T09:12:24.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.987+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a480ccc80 con 0x7f1a480d0d60 2024-03-29T09:12:24.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.988+0000 7f1a3ea00700 1 -- 172.21.15.99:0/924043252 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a38006cc0 con 0x7f1a480d0d60 2024-03-29T09:12:24.987 INFO:teuthology.orchestra.run.smithi099.stdout:pg_num: 64 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a2806ac90 msgr2=0x7f1a2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a2806ac90 0x7f1a2806d150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1a340011e0 tx=0x7f1a34009040 comp rx=0 tx=0).stop 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a480d0d60 msgr2=0x7f1a480b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a480d0d60 0x7f1a480b18d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1a38014040 tx=0x7f1a38004750 comp rx=0 tx=0).stop 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 shutdown_connections 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1a2806ac90 0x7f1a2806d150 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a48160770 0x7f1a480b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a480d0d60 0x7f1a480b18d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 --2- 172.21.15.99:0/924043252 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a480d3550 0x7f1a4806b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 >> 172.21.15.99:0/924043252 conn(0x7f1a4808afd0 msgr2=0x7f1a4815e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 shutdown_connections 2024-03-29T09:12:24.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:24.990+0000 7f1a4d400700 1 -- 172.21.15.99:0/924043252 wait complete. 2024-03-29T09:12:25.008 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:12:25.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.200+0000 7fe405800700 1 Processor -- start 2024-03-29T09:12:25.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.200+0000 7fe405800700 1 -- start start 2024-03-29T09:12:25.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe405800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4000d0150 0x7fe400161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe405800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cdcb0 0x7fe4000ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe405800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000d1be0 con 0x7fe4000cfd70 2024-03-29T09:12:25.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe405800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000b4980 con 0x7fe4000d0150 2024-03-29T09:12:25.201 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe405800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b46a0 con 0x7fe4000cdcb0 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe3fe000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe4000d0150 0x7fe400161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe3fea00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fe4000cfd70 0x7fe4001615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40994/0 (socket says 172.21.15.99:40994) 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.201+0000 7fe3fea00700 1 -- 172.21.15.99:0/3626653828 learned_addr learned my addr 172.21.15.99:0/3626653828 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3ff400700 1 --2- 172.21.15.99:0/3626653828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cdcb0 0x7fe4000ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fe000700 1 -- 172.21.15.99:0/3626653828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cdcb0 msgr2=0x7fe4000ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fe000700 1 --2- 172.21.15.99:0/3626653828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cdcb0 0x7fe4000ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fe000700 1 -- 172.21.15.99:0/3626653828 >> v1:172.21.15.99:6789/0 conn(0x7fe4000cfd70 legacy=0x7fe4001615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fe000700 1 -- 172.21.15.99:0/3626653828 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000d1590 con 0x7fe4000d0150 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fd600700 1 -- 172.21.15.99:0/3626653828 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704077253 0 0) 0x7fe4000d1be0 con 0x7fe4000cfd70 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fe000700 1 --2- 172.21.15.99:0/3626653828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000d0150 0x7fe400161cb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bffce73fe99338 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fd600700 1 -- 172.21.15.99:0/3626653828 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec011830 con 0x7fe4000d0150 2024-03-29T09:12:25.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fd600700 1 -- 172.21.15.99:0/3626653828 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec011990 con 0x7fe4000d0150 2024-03-29T09:12:25.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.202+0000 7fe3fd600700 1 -- 172.21.15.99:0/3626653828 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec011d20 con 0x7fe4000d0150 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 -- 172.21.15.99:0/3626653828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000d0150 msgr2=0x7fe400161cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 --2- 172.21.15.99:0/3626653828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000d0150 0x7fe400161cb0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec009b80 comp rx=0 tx=0).stop 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 -- 172.21.15.99:0/3626653828 shutdown_connections 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 --2- 172.21.15.99:0/3626653828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cdcb0 0x7fe4000ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 --2- 172.21.15.99:0/3626653828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000d0150 0x7fe400161cb0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 -- 172.21.15.99:0/3626653828 >> 172.21.15.99:0/3626653828 conn(0x7fe40009c4a0 msgr2=0x7fe4000d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 -- 172.21.15.99:0/3626653828 shutdown_connections 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.203+0000 7fe405800700 1 -- 172.21.15.99:0/3626653828 wait complete. 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.205+0000 7fe405800700 1 Processor -- start 2024-03-29T09:12:25.204 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.205+0000 7fe405800700 1 -- start start 2024-03-29T09:12:25.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe405800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000cdcb0 0x7fe4001562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe405800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cfd70 0x7fe4000b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe405800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4000d0150 0x7fe400157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe405800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b7d30 con 0x7fe4000cdcb0 2024-03-29T09:12:25.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe405800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b70b0 con 0x7fe4000d0150 2024-03-29T09:12:25.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe405800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b7390 con 0x7fe4000cfd70 2024-03-29T09:12:25.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000cdcb0 0x7fe4001562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000cdcb0 0x7fe4001562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46846/0 (socket says 172.21.15.99:46846) 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 -- 172.21.15.99:0/2152465288 learned_addr learned my addr 172.21.15.99:0/2152465288 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fe000700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cfd70 0x7fe4000b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3ff400700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4000d0150 0x7fe400157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 -- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cfd70 msgr2=0x7fe4000b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cfd70 0x7fe4000b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 -- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4000d0150 msgr2=0x7fe400157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4000d0150 0x7fe400157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.206+0000 7fe3fea00700 1 -- 172.21.15.99:0/2152465288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0094a0 con 0x7fe4000cdcb0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.207+0000 7fe3fea00700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000cdcb0 0x7fe4001562f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00011e0 tx=0x7fe3f000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.207+0000 7fe3f6a00700 1 -- 172.21.15.99:0/2152465288 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0004c80 con 0x7fe4000cdcb0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.207+0000 7fe3f6a00700 1 -- 172.21.15.99:0/2152465288 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0004de0 con 0x7fe4000cdcb0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.207+0000 7fe3f6a00700 1 -- 172.21.15.99:0/2152465288 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0007310 con 0x7fe4000cdcb0 2024-03-29T09:12:25.208 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.207+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000b7670 con 0x7fe4000cdcb0 2024-03-29T09:12:25.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.207+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4000b6ca0 con 0x7fe4000cdcb0 2024-03-29T09:12:25.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.208+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4000d2050 con 0x7fe4000cdcb0 2024-03-29T09:12:25.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.209+0000 7fe3f6a00700 1 -- 172.21.15.99:0/2152465288 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe3f0007470 con 0x7fe4000cdcb0 2024-03-29T09:12:25.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.210+0000 7fe3f6a00700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3e006ac90 0x7fe3e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.210+0000 7fe3f6a00700 1 -- 172.21.15.99:0/2152465288 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe3f0089180 con 0x7fe4000cdcb0 2024-03-29T09:12:25.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.210+0000 7fe3fe000700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3e006ac90 0x7fe3e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.211+0000 7fe3fe000700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3e006ac90 0x7fe3e006d150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:25.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.214+0000 7fe3f6a00700 1 -- 172.21.15.99:0/2152465288 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe4000d2050 con 0x7fe4000cdcb0 2024-03-29T09:12:25.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.461+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4000d2050 con 0x7fe4000cdcb0 2024-03-29T09:12:25.460 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.461+0000 7fe3f6a00700 1 -- 172.21.15.99:0/2152465288 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fe4000d2050 con 0x7fe4000cdcb0 2024-03-29T09:12:25.460 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:25.460 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/790352630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":790352630},{"type":"v1","addr":"172.21.15.133:6833","nonce":790352630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1265449168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1265449168},{"type":"v1","addr":"172.21.15.133:6835","nonce":1265449168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6102,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1161439746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1161439746},{"type":"v1","addr":"172.21.15.99:6836","nonce":1161439746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:11:43.652592+0000","modified":"2024-03-29T09:11:46.662472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6105":{"gid":6105,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2396189132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2396189132},{"type":"v1","addr":"172.21.15.99:6837","nonce":2396189132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3e006ac90 msgr2=0x7fe3e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3e006ac90 0x7fe3e006d150 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009870 tx=0x7fe3ec008030 comp rx=0 tx=0).stop 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000cdcb0 msgr2=0x7fe4001562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000cdcb0 0x7fe4001562f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00011e0 tx=0x7fe3f000ca30 comp rx=0 tx=0).stop 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 shutdown_connections 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe3e006ac90 0x7fe3e006d150 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4000d0150 0x7fe400157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4000cdcb0 0x7fe4001562f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 --2- 172.21.15.99:0/2152465288 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4000cfd70 0x7fe4000b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 >> 172.21.15.99:0/2152465288 conn(0x7fe40009c4a0 msgr2=0x7fe40006c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:25.463 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 shutdown_connections 2024-03-29T09:12:25.464 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.464+0000 7fe405800700 1 -- 172.21.15.99:0/2152465288 wait complete. 2024-03-29T09:12:25.464 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-03-29T09:12:25.481 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:25.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.671+0000 7f7e76c00700 1 Processor -- start 2024-03-29T09:12:25.670 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.671+0000 7f7e76c00700 1 -- start start 2024-03-29T09:12:25.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.672+0000 7f7e76c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f7e70160cc0 0x7f7e700d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.672+0000 7f7e76c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700cddc0 con 0x7f7e701608e0 2024-03-29T09:12:25.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.672+0000 7f7e76c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700ce0a0 con 0x7f7e700d1b20 2024-03-29T09:12:25.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.672+0000 7f7e76c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e700ce380 con 0x7f7e70160cc0 2024-03-29T09:12:25.671 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e6fe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f7e701608e0 0x7f7e700d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41006/0 (socket says 172.21.15.99:41006) 2024-03-29T09:12:25.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e6fe00700 1 -- 172.21.15.99:0/2676246844 learned_addr learned my addr 172.21.15.99:0/2676246844 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:25.672 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e74c00700 1 --2- 172.21.15.99:0/2676246844 >> v2:172.21.15.99:3300/0 conn(0x7f7e70160cc0 0x7f7e700d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e74c00700 1 -- 172.21.15.99:0/2676246844 >> v1:172.21.15.133:6789/0 conn(0x7f7e700d1b20 legacy=0x7f7e700afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e74c00700 1 -- 172.21.15.99:0/2676246844 >> v1:172.21.15.99:6789/0 conn(0x7f7e701608e0 legacy=0x7f7e700d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e74c00700 1 -- 172.21.15.99:0/2676246844 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700ce660 con 0x7f7e70160cc0 2024-03-29T09:12:25.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e6ea00700 1 -- 172.21.15.99:0/2676246844 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290951724 0 0) 0x7f7e700cddc0 con 0x7f7e701608e0 2024-03-29T09:12:25.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.673+0000 7f7e74c00700 1 --2- 172.21.15.99:0/2676246844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70160cc0 0x7f7e700d0150 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009d90 tx=0x7f7e6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ceb6c6614868aeb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:25.673 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e6ea00700 1 -- 172.21.15.99:0/2676246844 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6800e070 con 0x7f7e70160cc0 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e6ea00700 1 -- 172.21.15.99:0/2676246844 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6800bbd0 con 0x7f7e70160cc0 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e6ea00700 1 -- 172.21.15.99:0/2676246844 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68012430 con 0x7f7e70160cc0 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e76c00700 1 -- 172.21.15.99:0/2676246844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70160cc0 msgr2=0x7f7e700d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e76c00700 1 --2- 172.21.15.99:0/2676246844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70160cc0 0x7f7e700d0150 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009d90 tx=0x7f7e6800b330 comp rx=0 tx=0).stop 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e76c00700 1 -- 172.21.15.99:0/2676246844 shutdown_connections 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e76c00700 1 --2- 172.21.15.99:0/2676246844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e70160cc0 0x7f7e700d0150 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e76c00700 1 -- 172.21.15.99:0/2676246844 >> 172.21.15.99:0/2676246844 conn(0x7f7e7009c490 msgr2=0x7f7e700d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e76c00700 1 -- 172.21.15.99:0/2676246844 shutdown_connections 2024-03-29T09:12:25.674 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.674+0000 7f7e76c00700 1 -- 172.21.15.99:0/2676246844 wait complete. 2024-03-29T09:12:25.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.676+0000 7f7e76c00700 1 Processor -- start 2024-03-29T09:12:25.675 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.676+0000 7f7e76c00700 1 -- start start 2024-03-29T09:12:25.676 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e76c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e700d1b20 0x7f7e7015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e76c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e701608e0 0x7f7e701f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e76c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e70160cc0 0x7f7e700b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e76c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7015ccc0 con 0x7f7e701608e0 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e76c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b54a0 con 0x7f7e70160cc0 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e76c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b5780 con 0x7f7e700d1b20 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e6f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e701608e0 0x7f7e701f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e6f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e701608e0 0x7f7e701f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46850/0 (socket says 172.21.15.99:46850) 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e6f400700 1 -- 172.21.15.99:0/1694833294 learned_addr learned my addr 172.21.15.99:0/1694833294 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.677+0000 7f7e6fe00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e700d1b20 0x7f7e7015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e74c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e70160cc0 0x7f7e700b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e6f400700 1 -- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e700d1b20 msgr2=0x7f7e7015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e6f400700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e700d1b20 0x7f7e7015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.677 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e6f400700 1 -- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e70160cc0 msgr2=0x7f7e700b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e6f400700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e70160cc0 0x7f7e700b8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e6f400700 1 -- 172.21.15.99:0/1694833294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68009a70 con 0x7f7e701608e0 2024-03-29T09:12:25.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e74c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e70160cc0 0x7f7e700b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:25.678 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e6f400700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e701608e0 0x7f7e701f42d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c4e0 tx=0x7f7e5c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:25.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e67e00700 1 -- 172.21.15.99:0/1694833294 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0052d0 con 0x7f7e701608e0 2024-03-29T09:12:25.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e67e00700 1 -- 172.21.15.99:0/1694833294 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c005430 con 0x7f7e701608e0 2024-03-29T09:12:25.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e67e00700 1 -- 172.21.15.99:0/1694833294 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0038c0 con 0x7f7e701608e0 2024-03-29T09:12:25.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e700b5a60 con 0x7f7e701608e0 2024-03-29T09:12:25.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.678+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e700b6020 con 0x7f7e701608e0 2024-03-29T09:12:25.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.679+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e700677a0 con 0x7f7e701608e0 2024-03-29T09:12:25.679 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.680+0000 7f7e67e00700 1 -- 172.21.15.99:0/1694833294 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7e5c003a20 con 0x7f7e701608e0 2024-03-29T09:12:25.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.681+0000 7f7e67e00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e5006acc0 0x7f7e5006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:25.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.681+0000 7f7e67e00700 1 -- 172.21.15.99:0/1694833294 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7e5c088820 con 0x7f7e701608e0 2024-03-29T09:12:25.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.681+0000 7f7e6fe00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e5006acc0 0x7f7e5006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:25.680 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.682+0000 7f7e6fe00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e5006acc0 0x7f7e5006d180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7e60007b10 tx=0x7f7e60007a60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:25.684 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.685+0000 7f7e67e00700 1 -- 172.21.15.99:0/1694833294 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7e5c053950 con 0x7f7e701608e0 2024-03-29T09:12:25.888 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.889+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e700d00a0 con 0x7f7e701608e0 2024-03-29T09:12:25.889 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.890+0000 7f7e67e00700 1 -- 172.21.15.99:0/1694833294 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f7e5c017090 con 0x7f7e701608e0 2024-03-29T09:12:25.889 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:25.890 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:25.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e5006acc0 msgr2=0x7f7e5006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e5006acc0 0x7f7e5006d180 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7e60007b10 tx=0x7f7e60007a60 comp rx=0 tx=0).stop 2024-03-29T09:12:25.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e701608e0 msgr2=0x7f7e701f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:25.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e701608e0 0x7f7e701f42d0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00c4e0 tx=0x7f7e5c00c7f0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 shutdown_connections 2024-03-29T09:12:25.892 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7e5006acc0 0x7f7e5006d180 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e70160cc0 0x7f7e700b8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e700d1b20 0x7f7e7015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 --2- 172.21.15.99:0/1694833294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7e701608e0 0x7f7e701f42d0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:25.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 >> 172.21.15.99:0/1694833294 conn(0x7f7e7009c490 msgr2=0x7f7e7015fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:25.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 shutdown_connections 2024-03-29T09:12:25.893 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:25.893+0000 7f7e76c00700 1 -- 172.21.15.99:0/1694833294 wait complete. 2024-03-29T09:12:25.910 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T09:12:25.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:12:26.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.102+0000 7f88ed000700 1 Processor -- start 2024-03-29T09:12:26.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.102+0000 7f88ed000700 1 -- start start 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.103+0000 7f88ed000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80cfd70 0x7f88e81615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.103+0000 7f88ed000700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f88e80d0150 0x7f88e8161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.103+0000 7f88ed000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e80d1be0 con 0x7f88e8162020 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.103+0000 7f88ed000700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e80b4980 con 0x7f88e80d0150 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.103+0000 7f88ed000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80b46a0 con 0x7f88e80cfd70 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e6800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f88e8162020 0x7f88e80b2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41008/0 (socket says 172.21.15.99:41008) 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e6800700 1 -- 172.21.15.99:0/3362572029 learned_addr learned my addr 172.21.15.99:0/3362572029 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e5e00700 1 --2- 172.21.15.99:0/3362572029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80cfd70 0x7f88e81615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e5400700 1 --2- 172.21.15.99:0/3362572029 >> v2:172.21.15.133:3300/0 conn(0x7f88e80d0150 0x7f88e8161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e4a00700 1 -- 172.21.15.99:0/3362572029 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488829918 0 0) 0x7f88e80d1be0 con 0x7f88e8162020 2024-03-29T09:12:26.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e4a00700 1 -- 172.21.15.99:0/3362572029 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88cc0035b0 con 0x7f88e8162020 2024-03-29T09:12:26.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e5e00700 1 -- 172.21.15.99:0/3362572029 >> v2:172.21.15.133:3300/0 conn(0x7f88e80d0150 msgr2=0x7f88e8161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:26.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e5e00700 1 --2- 172.21.15.99:0/3362572029 >> v2:172.21.15.133:3300/0 conn(0x7f88e80d0150 0x7f88e8161ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e5e00700 1 -- 172.21.15.99:0/3362572029 >> v1:172.21.15.99:6789/0 conn(0x7f88e8162020 legacy=0x7f88e80b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.104+0000 7f88e5e00700 1 -- 172.21.15.99:0/3362572029 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e80d1590 con 0x7f88e80cfd70 2024-03-29T09:12:26.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88e5e00700 1 --2- 172.21.15.99:0/3362572029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80cfd70 0x7f88e81615a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009840 tx=0x7f88d8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509c976deec26915 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:26.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88e4a00700 1 -- 172.21.15.99:0/3362572029 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8011830 con 0x7f88e80cfd70 2024-03-29T09:12:26.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88e4a00700 1 -- 172.21.15.99:0/3362572029 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8011990 con 0x7f88e80cfd70 2024-03-29T09:12:26.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88e4a00700 1 -- 172.21.15.99:0/3362572029 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8011d20 con 0x7f88e80cfd70 2024-03-29T09:12:26.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88ed000700 1 -- 172.21.15.99:0/3362572029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80cfd70 msgr2=0x7f88e81615a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88ed000700 1 --2- 172.21.15.99:0/3362572029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80cfd70 0x7f88e81615a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009840 tx=0x7f88d8009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:26.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88ed000700 1 -- 172.21.15.99:0/3362572029 shutdown_connections 2024-03-29T09:12:26.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88ed000700 1 --2- 172.21.15.99:0/3362572029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80cfd70 0x7f88e81615a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88ed000700 1 --2- 172.21.15.99:0/3362572029 >> v2:172.21.15.133:3300/0 conn(0x7f88e80d0150 0x7f88e8161ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88ed000700 1 -- 172.21.15.99:0/3362572029 >> 172.21.15.99:0/3362572029 conn(0x7f88e809c4a0 msgr2=0x7f88e80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:26.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.105+0000 7f88ed000700 1 -- 172.21.15.99:0/3362572029 shutdown_connections 2024-03-29T09:12:26.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.106+0000 7f88ed000700 1 -- 172.21.15.99:0/3362572029 wait complete. 2024-03-29T09:12:26.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.108+0000 7f88ed000700 1 Processor -- start 2024-03-29T09:12:26.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.108+0000 7f88ed000700 1 -- start start 2024-03-29T09:12:26.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88ed000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e80cfd70 0x7f88e80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88ed000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80d0150 0x7f88e80ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88ed000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88e8162020 0x7f88e8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.109 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88ed000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80cc7a0 con 0x7f88e80cfd70 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88ed000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e80cb490 con 0x7f88e8162020 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88ed000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e8155c50 con 0x7f88e80d0150 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88e5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e80cfd70 0x7f88e80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88e5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e80cfd70 0x7f88e80cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46862/0 (socket says 172.21.15.99:46862) 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.109+0000 7f88e5e00700 1 -- 172.21.15.99:0/3909490526 learned_addr learned my addr 172.21.15.99:0/3909490526 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5400700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80d0150 0x7f88e80ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e6800700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88e8162020 0x7f88e8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5e00700 1 -- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80d0150 msgr2=0x7f88e80ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5e00700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80d0150 0x7f88e80ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5e00700 1 -- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88e8162020 msgr2=0x7f88e8155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5e00700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88e8162020 0x7f88e8155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.110 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5e00700 1 -- 172.21.15.99:0/3909490526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80094a0 con 0x7f88e80cfd70 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5400700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80d0150 0x7f88e80ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88e5e00700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e80cfd70 0x7f88e80cb8b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88de000700 1 -- 172.21.15.99:0/3909490526 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80036a0 con 0x7f88e80cfd70 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88de000700 1 -- 172.21.15.99:0/3909490526 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d801f070 con 0x7f88e80cfd70 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e80cc200 con 0x7f88e80cfd70 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.110+0000 7f88de000700 1 -- 172.21.15.99:0/3909490526 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8003a40 con 0x7f88e80cfd70 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.111+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e80cadd0 con 0x7f88e80cfd70 2024-03-29T09:12:26.111 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.111+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e80cac30 con 0x7f88e80cfd70 2024-03-29T09:12:26.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.112+0000 7f88de000700 1 -- 172.21.15.99:0/3909490526 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f88d8003ba0 con 0x7f88e80cfd70 2024-03-29T09:12:26.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.113+0000 7f88de000700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88c806ac90 0x7f88c806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.114+0000 7f88de000700 1 -- 172.21.15.99:0/3909490526 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f88d8021070 con 0x7f88e80cfd70 2024-03-29T09:12:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.114+0000 7f88e5400700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88c806ac90 0x7f88c806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.114+0000 7f88e5400700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88c806ac90 0x7f88c806d150 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88e806be40 tx=0x7f88d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:26.116 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.118+0000 7f88de000700 1 -- 172.21.15.99:0/3909490526 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f88e80cac30 con 0x7f88e80cfd70 2024-03-29T09:12:26.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.364+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88e80d2050 con 0x7f88e80cfd70 2024-03-29T09:12:26.363 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.365+0000 7f88de000700 1 -- 172.21.15.99:0/3909490526 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f88e80d2050 con 0x7f88e80cfd70 2024-03-29T09:12:26.364 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:26.364 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/790352630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":790352630},{"type":"v1","addr":"172.21.15.133:6833","nonce":790352630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1265449168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1265449168},{"type":"v1","addr":"172.21.15.133:6835","nonce":1265449168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6102,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1161439746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1161439746},{"type":"v1","addr":"172.21.15.99:6836","nonce":1161439746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:11:43.652592+0000","modified":"2024-03-29T09:11:46.662472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6105":{"gid":6105,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2396189132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2396189132},{"type":"v1","addr":"172.21.15.99:6837","nonce":2396189132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T09:12:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88c806ac90 msgr2=0x7f88c806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88c806ac90 0x7f88c806d150 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88e806be40 tx=0x7f88d4008040 comp rx=0 tx=0).stop 2024-03-29T09:12:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e80cfd70 msgr2=0x7f88e80cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.366 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e80cfd70 0x7f88e80cb8b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f88d8000c00 tx=0x7f88d8004750 comp rx=0 tx=0).stop 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 shutdown_connections 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f88c806ac90 0x7f88c806d150 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88e8162020 0x7f88e8155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e80cfd70 0x7f88e80cb8b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 --2- 172.21.15.99:0/3909490526 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88e80d0150 0x7f88e80ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 >> 172.21.15.99:0/3909490526 conn(0x7f88e809c4a0 msgr2=0x7f88e80b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 shutdown_connections 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.367+0000 7f88ed000700 1 -- 172.21.15.99:0/3909490526 wait complete. 2024-03-29T09:12:26.367 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-03-29T09:12:26.385 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:12:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.577+0000 7f3d61200700 1 Processor -- start 2024-03-29T09:12:26.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.577+0000 7f3d61200700 1 -- start start 2024-03-29T09:12:26.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.578+0000 7f3d61200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c15d690 0x7f3d5c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.578+0000 7f3d61200700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3d5c15da70 0x7f3d5c0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.578+0000 7f3d61200700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c0afde0 con 0x7f3d5c0d1740 2024-03-29T09:12:26.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.578+0000 7f3d61200700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c0cf660 con 0x7f3d5c15da70 2024-03-29T09:12:26.578 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.578+0000 7f3d61200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0d01d0 con 0x7f3d5c15d690 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d5a800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f3d5c0d1740 0x7f3d5c0cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41010/0 (socket says 172.21.15.99:41010) 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d5a800700 1 -- 172.21.15.99:0/2245193481 learned_addr learned my addr 172.21.15.99:0/2245193481 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d59e00700 1 --2- 172.21.15.99:0/2245193481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c15d690 0x7f3d5c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d59400700 1 --2- 172.21.15.99:0/2245193481 >> v2:172.21.15.133:3300/0 conn(0x7f3d5c15da70 0x7f3d5c0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d58a00700 1 -- 172.21.15.99:0/2245193481 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104967086 0 0) 0x7f3d5c0afde0 con 0x7f3d5c0d1740 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d58a00700 1 -- 172.21.15.99:0/2245193481 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d400035b0 con 0x7f3d5c0d1740 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d59e00700 1 -- 172.21.15.99:0/2245193481 >> v2:172.21.15.133:3300/0 conn(0x7f3d5c15da70 msgr2=0x7f3d5c0cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d59e00700 1 --2- 172.21.15.99:0/2245193481 >> v2:172.21.15.133:3300/0 conn(0x7f3d5c15da70 0x7f3d5c0cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d59e00700 1 -- 172.21.15.99:0/2245193481 >> v1:172.21.15.99:6789/0 conn(0x7f3d5c0d1740 legacy=0x7f3d5c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.579+0000 7f3d59e00700 1 -- 172.21.15.99:0/2245193481 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0d04b0 con 0x7f3d5c15d690 2024-03-29T09:12:26.579 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d59e00700 1 --2- 172.21.15.99:0/2245193481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c15d690 0x7f3d5c0cfb60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009840 tx=0x7f3d4c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee4e98651ad1ec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d58a00700 1 -- 172.21.15.99:0/2245193481 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c011830 con 0x7f3d5c15d690 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d58a00700 1 -- 172.21.15.99:0/2245193481 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c011990 con 0x7f3d5c15d690 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d58a00700 1 -- 172.21.15.99:0/2245193481 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c011d20 con 0x7f3d5c15d690 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d61200700 1 -- 172.21.15.99:0/2245193481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c15d690 msgr2=0x7f3d5c0cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d61200700 1 --2- 172.21.15.99:0/2245193481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c15d690 0x7f3d5c0cfb60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009840 tx=0x7f3d4c009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d61200700 1 -- 172.21.15.99:0/2245193481 shutdown_connections 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d61200700 1 --2- 172.21.15.99:0/2245193481 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c15d690 0x7f3d5c0cfb60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d61200700 1 --2- 172.21.15.99:0/2245193481 >> v2:172.21.15.133:3300/0 conn(0x7f3d5c15da70 0x7f3d5c0cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.581 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.580+0000 7f3d61200700 1 -- 172.21.15.99:0/2245193481 >> 172.21.15.99:0/2245193481 conn(0x7f3d5c09c480 msgr2=0x7f3d5c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:26.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.581+0000 7f3d61200700 1 -- 172.21.15.99:0/2245193481 shutdown_connections 2024-03-29T09:12:26.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.581+0000 7f3d61200700 1 -- 172.21.15.99:0/2245193481 wait complete. 2024-03-29T09:12:26.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.582+0000 7f3d61200700 1 Processor -- start 2024-03-29T09:12:26.582 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.582+0000 7f3d61200700 1 -- start start 2024-03-29T09:12:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d61200700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c0d1740 0x7f3d5c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d61200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d5c15d690 0x7f3d5c1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d61200700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d5c15da70 0x7f3d5c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d61200700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c1f39b0 con 0x7f3d5c15da70 2024-03-29T09:12:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d61200700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c15f440 con 0x7f3d5c15d690 2024-03-29T09:12:26.584 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d61200700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c15e8c0 con 0x7f3d5c0d1740 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d5a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d5c15da70 0x7f3d5c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.583+0000 7f3d5a800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d5c15da70 0x7f3d5c0b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46876/0 (socket says 172.21.15.99:46876) 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d5a800700 1 -- 172.21.15.99:0/193236094 learned_addr learned my addr 172.21.15.99:0/193236094 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d59e00700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c0d1740 0x7f3d5c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d59400700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d5c15d690 0x7f3d5c1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d5a800700 1 -- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c0d1740 msgr2=0x7f3d5c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d5a800700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c0d1740 0x7f3d5c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d5a800700 1 -- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d5c15d690 msgr2=0x7f3d5c1f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d5a800700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d5c15d690 0x7f3d5c1f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d5a800700 1 -- 172.21.15.99:0/193236094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0094a0 con 0x7f3d5c15da70 2024-03-29T09:12:26.585 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d59400700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d5c15d690 0x7f3d5c1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d5a800700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d5c15da70 0x7f3d5c0b4be0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400caa0 tx=0x7f3d5400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d52000700 1 -- 172.21.15.99:0/193236094 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400a9e0 con 0x7f3d5c15da70 2024-03-29T09:12:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d52000700 1 -- 172.21.15.99:0/193236094 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5400ab40 con 0x7f3d5c15da70 2024-03-29T09:12:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d52000700 1 -- 172.21.15.99:0/193236094 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54012400 con 0x7f3d5c15da70 2024-03-29T09:12:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.584+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c15eba0 con 0x7f3d5c15da70 2024-03-29T09:12:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.585+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c15f160 con 0x7f3d5c15da70 2024-03-29T09:12:26.586 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.585+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c0677a0 con 0x7f3d5c15da70 2024-03-29T09:12:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.586+0000 7f3d52000700 1 -- 172.21.15.99:0/193236094 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3d5401b3b0 con 0x7f3d5c15da70 2024-03-29T09:12:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.587+0000 7f3d52000700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3d3c06ac90 0x7f3d3c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.587+0000 7f3d52000700 1 -- 172.21.15.99:0/193236094 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3d540891f0 con 0x7f3d5c15da70 2024-03-29T09:12:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.587+0000 7f3d59e00700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3d3c06ac90 0x7f3d3c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:26.587 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.588+0000 7f3d59e00700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3d3c06ac90 0x7f3d3c06d150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009fd0 tx=0x7f3d4c01c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:26.591 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.592+0000 7f3d52000700 1 -- 172.21.15.99:0/193236094 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3d54054280 con 0x7f3d5c15da70 2024-03-29T09:12:26.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.840+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d5c0b3f70 con 0x7f3d5c15da70 2024-03-29T09:12:26.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.841+0000 7f3d52000700 1 -- 172.21.15.99:0/193236094 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f3d540578a0 con 0x7f3d5c15da70 2024-03-29T09:12:26.840 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:26.840 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/790352630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":790352630},{"type":"v1","addr":"172.21.15.133:6833","nonce":790352630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1265449168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1265449168},{"type":"v1","addr":"172.21.15.133:6835","nonce":1265449168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6102,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1161439746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1161439746},{"type":"v1","addr":"172.21.15.99:6836","nonce":1161439746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T09:11:43.652592+0000","modified":"2024-03-29T09:11:46.662472+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6105},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6105":{"gid":6105,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.99:6837/2396189132","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":2396189132},{"type":"v1","addr":"172.21.15.99:6837","nonce":2396189132}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T09:12:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.843+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3d3c06ac90 msgr2=0x7f3d3c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3d3c06ac90 0x7f3d3c06d150 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009fd0 tx=0x7f3d4c01c040 comp rx=0 tx=0).stop 2024-03-29T09:12:26.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d5c15da70 msgr2=0x7f3d5c0b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d5c15da70 0x7f3d5c0b4be0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400caa0 tx=0x7f3d5400cdb0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 shutdown_connections 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f3d3c06ac90 0x7f3d3c06d150 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d5c15da70 0x7f3d5c0b4be0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d5c0d1740 0x7f3d5c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 --2- 172.21.15.99:0/193236094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d5c15d690 0x7f3d5c1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 >> 172.21.15.99:0/193236094 conn(0x7f3d5c09c480 msgr2=0x7f3d5c15c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 shutdown_connections 2024-03-29T09:12:26.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:26.844+0000 7f3d61200700 1 -- 172.21.15.99:0/193236094 wait complete. 2024-03-29T09:12:26.845 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 25 2024-03-29T09:12:26.861 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:27.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.051+0000 7f12d6e00700 1 Processor -- start 2024-03-29T09:12:27.050 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.051+0000 7f12d6e00700 1 -- start start 2024-03-29T09:12:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.052+0000 7f12d6e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12d0153d20 0x7f12d0070540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.052+0000 7f12d6e00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f12d015d330 0x7f12d015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.052+0000 7f12d6e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 0x7f12d015a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.052+0000 7f12d6e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d015dd00 con 0x7f12d0153d20 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.052+0000 7f12d6e00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d00ccf60 con 0x7f12d015d330 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.052+0000 7f12d6e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0071750 con 0x7f12d0159d40 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12d0153d20 0x7f12d0070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f12d0153d20 0x7f12d0070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46878/0 (socket says 172.21.15.99:46878) 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 -- 172.21.15.99:0/1612740824 learned_addr learned my addr 172.21.15.99:0/1612740824 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12d4e00700 1 --2- 172.21.15.99:0/1612740824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 0x7f12d015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cf400700 1 --2- 172.21.15.99:0/1612740824 >> v2:172.21.15.133:3300/0 conn(0x7f12d015d330 0x7f12d015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 -- 172.21.15.99:0/1612740824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 msgr2=0x7f12d015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 --2- 172.21.15.99:0/1612740824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 0x7f12d015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 -- 172.21.15.99:0/1612740824 >> v2:172.21.15.133:3300/0 conn(0x7f12d015d330 msgr2=0x7f12d015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 --2- 172.21.15.99:0/1612740824 >> v2:172.21.15.133:3300/0 conn(0x7f12d015d330 0x7f12d015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 -- 172.21.15.99:0/1612740824 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0070eb0 con 0x7f12d0153d20 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.053+0000 7f12cfe00700 1 --2- 172.21.15.99:0/1612740824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d0070540 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f12d006ac20 tx=0x7f12c000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8909b13392a4696 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12cea00700 1 -- 172.21.15.99:0/1612740824 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000dc20 con 0x7f12d0153d20 2024-03-29T09:12:27.053 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12cea00700 1 -- 172.21.15.99:0/1612740824 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000dd80 con 0x7f12d0153d20 2024-03-29T09:12:27.054 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12cea00700 1 -- 172.21.15.99:0/1612740824 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0017630 con 0x7f12d0153d20 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 -- 172.21.15.99:0/1612740824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 msgr2=0x7f12d0070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 --2- 172.21.15.99:0/1612740824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d0070540 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f12d006ac20 tx=0x7f12c000be70 comp rx=0 tx=0).stop 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 -- 172.21.15.99:0/1612740824 shutdown_connections 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 --2- 172.21.15.99:0/1612740824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 0x7f12d015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 --2- 172.21.15.99:0/1612740824 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d0070540 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 --2- 172.21.15.99:0/1612740824 >> v2:172.21.15.133:3300/0 conn(0x7f12d015d330 0x7f12d015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 -- 172.21.15.99:0/1612740824 >> 172.21.15.99:0/1612740824 conn(0x7f12d0066540 msgr2=0x7f12d0066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 -- 172.21.15.99:0/1612740824 shutdown_connections 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.054+0000 7f12d6e00700 1 -- 172.21.15.99:0/1612740824 wait complete. 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.056+0000 7f12d6e00700 1 Processor -- start 2024-03-29T09:12:27.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.056+0000 7f12d6e00700 1 -- start start 2024-03-29T09:12:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12d6e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d00cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12d6e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 0x7f12d0144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12d6e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d015d330 0x7f12d01543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12d6e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d01554c0 con 0x7f12d0153d20 2024-03-29T09:12:27.058 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12d6e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0155930 con 0x7f12d015d330 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12d6e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d0144eb0 con 0x7f12d0159d40 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d00cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d00cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46894/0 (socket says 172.21.15.99:46894) 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 -- 172.21.15.99:0/3010710761 learned_addr learned my addr 172.21.15.99:0/3010710761 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 -- 172.21.15.99:0/3010710761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 msgr2=0x7f12d0144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 0x7f12d0144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 -- 172.21.15.99:0/3010710761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d015d330 msgr2=0x7f12d01543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.059 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d015d330 0x7f12d01543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.057+0000 7f12cfe00700 1 -- 172.21.15.99:0/3010710761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c000b7f0 con 0x7f12d0153d20 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.058+0000 7f12cfe00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d00cacb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f12c0019040 tx=0x7f12c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.058+0000 7f12c7e00700 1 -- 172.21.15.99:0/3010710761 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0021410 con 0x7f12d0153d20 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.058+0000 7f12c7e00700 1 -- 172.21.15.99:0/3010710761 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0021590 con 0x7f12d0153d20 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.058+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d0154c20 con 0x7f12d0153d20 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.058+0000 7f12c7e00700 1 -- 172.21.15.99:0/3010710761 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0021940 con 0x7f12d0153d20 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.058+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d01551e0 con 0x7f12d0153d20 2024-03-29T09:12:27.060 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.059+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d0155690 con 0x7f12d0153d20 2024-03-29T09:12:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.060+0000 7f12c7e00700 1 -- 172.21.15.99:0/3010710761 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f12c0015070 con 0x7f12d0153d20 2024-03-29T09:12:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.061+0000 7f12c7e00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12b006ac90 0x7f12b006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.061+0000 7f12c7e00700 1 -- 172.21.15.99:0/3010710761 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f12c008acb0 con 0x7f12d0153d20 2024-03-29T09:12:27.062 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.061+0000 7f12cf400700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12b006ac90 0x7f12b006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.063 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.062+0000 7f12cf400700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12b006ac90 0x7f12b006d150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f12d00ce2c0 tx=0x7f12bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.065+0000 7f12c7e00700 1 -- 172.21.15.99:0/3010710761 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f12c0055e90 con 0x7f12d0153d20 2024-03-29T09:12:27.268 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.269+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12d0155690 con 0x7f12d0153d20 2024-03-29T09:12:27.269 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.270+0000 7f12c7e00700 1 -- 172.21.15.99:0/3010710761 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16302 (secure 0 0 0) 0x7f12c00594b0 con 0x7f12d0153d20 2024-03-29T09:12:27.269 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:27.269 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":55,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:11:44.655318+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T09:11:41.952804+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T09:11:42.633561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12b006ac90 msgr2=0x7f12b006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12b006ac90 0x7f12b006d150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f12d00ce2c0 tx=0x7f12bc008040 comp rx=0 tx=0).stop 2024-03-29T09:12:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 msgr2=0x7f12d00cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.272 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d00cacb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f12c0019040 tx=0x7f12c0004750 comp rx=0 tx=0).stop 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 shutdown_connections 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f12b006ac90 0x7f12b006d150 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12d015d330 0x7f12d01543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f12d0153d20 0x7f12d00cacb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 --2- 172.21.15.99:0/3010710761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12d0159d40 0x7f12d0144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 >> 172.21.15.99:0/3010710761 conn(0x7f12d0066540 msgr2=0x7f12d015c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 shutdown_connections 2024-03-29T09:12:27.273 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.273+0000 7f12d6e00700 1 -- 172.21.15.99:0/3010710761 wait complete. 2024-03-29T09:12:27.291 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T09:12:27.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.483+0000 7f5999000700 1 Processor -- start 2024-03-29T09:12:27.482 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.483+0000 7f5999000700 1 -- start start 2024-03-29T09:12:27.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.484+0000 7f5999000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f599415e7a0 0x7f59940d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.484+0000 7f5999000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59940cdf80 con 0x7f59940cfad0 2024-03-29T09:12:27.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.484+0000 7f5999000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599415eb80 con 0x7f59940d12f0 2024-03-29T09:12:27.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.484+0000 7f5999000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59940d2140 con 0x7f599415e7a0 2024-03-29T09:12:27.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.484+0000 7f5991e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f599415e7a0 0x7f59940d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.483 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.484+0000 7f5992800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f59940cfad0 0x7f59940cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:41020/0 (socket says 172.21.15.99:41020) 2024-03-29T09:12:27.484 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.484+0000 7f5992800700 1 -- 172.21.15.99:0/1046455936 learned_addr learned my addr 172.21.15.99:0/1046455936 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:27.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.485+0000 7f5990a00700 1 -- 172.21.15.99:0/1046455936 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769589367 0 0) 0x7f59940cdf80 con 0x7f59940cfad0 2024-03-29T09:12:27.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.485+0000 7f5990a00700 1 -- 172.21.15.99:0/1046455936 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59780035b0 con 0x7f59940cfad0 2024-03-29T09:12:27.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.485+0000 7f5991e00700 1 -- 172.21.15.99:0/1046455936 >> v1:172.21.15.133:6789/0 conn(0x7f59940d12f0 legacy=0x7f59940cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.485+0000 7f5991e00700 1 -- 172.21.15.99:0/1046455936 >> v1:172.21.15.99:6789/0 conn(0x7f59940cfad0 legacy=0x7f59940cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.485+0000 7f5991e00700 1 -- 172.21.15.99:0/1046455936 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59940ce1f0 con 0x7f599415e7a0 2024-03-29T09:12:27.485 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.485+0000 7f5991e00700 1 --2- 172.21.15.99:0/1046455936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f599415e7a0 0x7f59940d1a40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5984009840 tx=0x7f5984009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940597ca264ca1c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.485+0000 7f5990a00700 1 -- 172.21.15.99:0/1046455936 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984011830 con 0x7f599415e7a0 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5990a00700 1 -- 172.21.15.99:0/1046455936 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5984011990 con 0x7f599415e7a0 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5990a00700 1 -- 172.21.15.99:0/1046455936 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984011d20 con 0x7f599415e7a0 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5999000700 1 -- 172.21.15.99:0/1046455936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f599415e7a0 msgr2=0x7f59940d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5999000700 1 --2- 172.21.15.99:0/1046455936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f599415e7a0 0x7f59940d1a40 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5984009840 tx=0x7f5984009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5999000700 1 -- 172.21.15.99:0/1046455936 shutdown_connections 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5999000700 1 --2- 172.21.15.99:0/1046455936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f599415e7a0 0x7f59940d1a40 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5999000700 1 -- 172.21.15.99:0/1046455936 >> 172.21.15.99:0/1046455936 conn(0x7f599409c480 msgr2=0x7f59940d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5999000700 1 -- 172.21.15.99:0/1046455936 shutdown_connections 2024-03-29T09:12:27.486 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.486+0000 7f5999000700 1 -- 172.21.15.99:0/1046455936 wait complete. 2024-03-29T09:12:27.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.488+0000 7f5999000700 1 Processor -- start 2024-03-29T09:12:27.487 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.488+0000 7f5999000700 1 -- start start 2024-03-29T09:12:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5999000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59940cfad0 0x7f59940b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5999000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59940d12f0 0x7f59940b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5999000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f599415e7a0 0x7f59940c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5999000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59940b7400 con 0x7f59940cfad0 2024-03-29T09:12:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5999000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59940b6780 con 0x7f599415e7a0 2024-03-29T09:12:27.488 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5999000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59940b6a60 con 0x7f59940d12f0 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5991e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59940cfad0 0x7f59940b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5991e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59940cfad0 0x7f59940b7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:46906/0 (socket says 172.21.15.99:46906) 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5991e00700 1 -- 172.21.15.99:0/3997145429 learned_addr learned my addr 172.21.15.99:0/3997145429 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5991400700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59940d12f0 0x7f59940b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.489+0000 7f5992800700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f599415e7a0 0x7f59940c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5991e00700 1 -- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59940d12f0 msgr2=0x7f59940b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5991e00700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59940d12f0 0x7f59940b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5991e00700 1 -- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f599415e7a0 msgr2=0x7f59940c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5991e00700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f599415e7a0 0x7f59940c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.489 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5991e00700 1 -- 172.21.15.99:0/3997145429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59840094a0 con 0x7f59940cfad0 2024-03-29T09:12:27.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5991e00700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59940cfad0 0x7f59940b7820 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5984009b90 tx=0x7f5984004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f598a000700 1 -- 172.21.15.99:0/3997145429 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59840038a0 con 0x7f59940cfad0 2024-03-29T09:12:27.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f598a000700 1 -- 172.21.15.99:0/3997145429 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598401e070 con 0x7f59940cfad0 2024-03-29T09:12:27.490 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59940b6d40 con 0x7f59940cfad0 2024-03-29T09:12:27.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f598a000700 1 -- 172.21.15.99:0/3997145429 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984003c40 con 0x7f59940cfad0 2024-03-29T09:12:27.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.490+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599415a290 con 0x7f59940cfad0 2024-03-29T09:12:27.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.491+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59940b7670 con 0x7f59940cfad0 2024-03-29T09:12:27.491 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.492+0000 7f598a000700 1 -- 172.21.15.99:0/3997145429 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f598402d450 con 0x7f59940cfad0 2024-03-29T09:12:27.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.493+0000 7f598a000700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f597406ac90 0x7f597406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.493+0000 7f598a000700 1 -- 172.21.15.99:0/3997145429 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f598408a480 con 0x7f59940cfad0 2024-03-29T09:12:27.492 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.494+0000 7f5991400700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f597406ac90 0x7f597406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.493 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.494+0000 7f5991400700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f597406ac90 0x7f597406d150 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f599415ab10 tx=0x7f5980009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.496 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.498+0000 7f598a000700 1 -- 172.21.15.99:0/3997145429 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f5984055510 con 0x7f59940cfad0 2024-03-29T09:12:27.741 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.742+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f59940d0650 con 0x7f59940cfad0 2024-03-29T09:12:27.742 INFO:tasks.ceph.mon.a.smithi099.stderr:2024-03-29T09:12:27.743+0000 7fe659e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T09:12:27.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.750+0000 7f598a000700 1 -- 172.21.15.99:0/3997145429 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5984058b30 con 0x7f59940cfad0 2024-03-29T09:12:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f597406ac90 msgr2=0x7f597406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f597406ac90 0x7f597406d150 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f599415ab10 tx=0x7f5980009040 comp rx=0 tx=0).stop 2024-03-29T09:12:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59940cfad0 msgr2=0x7f59940b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59940cfad0 0x7f59940b7820 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f5984009b90 tx=0x7f5984004750 comp rx=0 tx=0).stop 2024-03-29T09:12:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 shutdown_connections 2024-03-29T09:12:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f597406ac90 0x7f597406d150 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f599415e7a0 0x7f59940c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f59940cfad0 0x7f59940b7820 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 --2- 172.21.15.99:0/3997145429 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59940d12f0 0x7f59940b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 >> 172.21.15.99:0/3997145429 conn(0x7f599409c480 msgr2=0x7f59940b5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:27.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 shutdown_connections 2024-03-29T09:12:27.753 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.753+0000 7f5999000700 1 -- 172.21.15.99:0/3997145429 wait complete. 2024-03-29T09:12:27.753 INFO:teuthology.orchestra.run.smithi099.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T09:12:27.756 INFO:tasks.ceph.mds.c.smithi099.stderr: -3024> 2024-03-29T09:11:49.139+0000 7f385a400700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T09:12:27.771 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T09:12:27.805 INFO:tasks.ceph.mds.c.smithi099.stdout:starting mds.c at 2024-03-29T09:12:27.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.966+0000 7f4eed000700 1 Processor -- start 2024-03-29T09:12:27.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.966+0000 7f4eed000700 1 -- start start 2024-03-29T09:12:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.967+0000 7f4eed000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4ee80b3cc0 0x7f4ee80d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.967+0000 7f4eed000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee80d3810 con 0x7f4ee8160770 2024-03-29T09:12:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.967+0000 7f4eed000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee80d1520 con 0x7f4ee80d3380 2024-03-29T09:12:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.967+0000 7f4eed000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee80b4140 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.967+0000 7f4ee5400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4ee80b3cc0 0x7f4ee80d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.967+0000 7f4ee5e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f4ee8160770 0x7f4ee80d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40414/0 (socket says 172.21.15.99:40414) 2024-03-29T09:12:27.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.967+0000 7f4ee5e00700 1 -- 172.21.15.99:0/2356346693 learned_addr learned my addr 172.21.15.99:0/2356346693 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:27.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee5400700 1 -- 172.21.15.99:0/2356346693 >> v1:172.21.15.133:6789/0 conn(0x7f4ee80d3380 legacy=0x7f4ee80ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee5400700 1 -- 172.21.15.99:0/2356346693 >> v1:172.21.15.99:6789/0 conn(0x7f4ee8160770 legacy=0x7f4ee80d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee5400700 1 -- 172.21.15.99:0/2356346693 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80d1800 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee4a00700 1 -- 172.21.15.99:0/2356346693 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309664636 0 0) 0x7f4ee80d3810 con 0x7f4ee8160770 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee5400700 1 --2- 172.21.15.99:0/2356346693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee80d2e40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009840 tx=0x7f4ed4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0e30b1c9069fea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee4a00700 1 -- 172.21.15.99:0/2356346693 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4011830 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee4a00700 1 -- 172.21.15.99:0/2356346693 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed4011990 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.968+0000 7f4ee4a00700 1 -- 172.21.15.99:0/2356346693 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4011d20 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.969+0000 7f4eed000700 1 -- 172.21.15.99:0/2356346693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 msgr2=0x7f4ee80d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.969+0000 7f4eed000700 1 --2- 172.21.15.99:0/2356346693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee80d2e40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009840 tx=0x7f4ed4009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.969+0000 7f4eed000700 1 -- 172.21.15.99:0/2356346693 shutdown_connections 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.969+0000 7f4eed000700 1 --2- 172.21.15.99:0/2356346693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee80d2e40 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.969+0000 7f4eed000700 1 -- 172.21.15.99:0/2356346693 >> 172.21.15.99:0/2356346693 conn(0x7f4ee808afd0 msgr2=0x7f4ee809c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:27.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.969+0000 7f4eed000700 1 -- 172.21.15.99:0/2356346693 shutdown_connections 2024-03-29T09:12:27.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.969+0000 7f4eed000700 1 -- 172.21.15.99:0/2356346693 wait complete. 2024-03-29T09:12:27.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.971+0000 7f4eed000700 1 Processor -- start 2024-03-29T09:12:27.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.971+0000 7f4eed000700 1 -- start start 2024-03-29T09:12:27.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4eed000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee81540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4eed000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ee80d3380 0x7f4ee80b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4eed000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ee8160770 0x7f4ee806bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4eed000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee80b6110 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4eed000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee81f68f0 con 0x7f4ee8160770 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4eed000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee815acf0 con 0x7f4ee80d3380 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee81540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee81540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59874/0 (socket says 172.21.15.99:59874) 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 -- 172.21.15.99:0/4033465616 learned_addr learned my addr 172.21.15.99:0/4033465616 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5400700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ee80d3380 0x7f4ee80b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee6800700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ee8160770 0x7f4ee806bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 -- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ee80d3380 msgr2=0x7f4ee80b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ee80d3380 0x7f4ee80b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.973 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 -- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ee8160770 msgr2=0x7f4ee806bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ee8160770 0x7f4ee806bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee5e00700 1 -- 172.21.15.99:0/4033465616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40094a0 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.972+0000 7f4ee6800700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ee8160770 0x7f4ee806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.973+0000 7f4ee5e00700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee81540d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c520 tx=0x7f4ed800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.973+0000 7f4ede000700 1 -- 172.21.15.99:0/4033465616 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8004c80 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.973+0000 7f4ede000700 1 -- 172.21.15.99:0/4033465616 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed8004de0 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.973+0000 7f4ede000700 1 -- 172.21.15.99:0/4033465616 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8007310 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.973+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee81f62a0 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.973+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee815afd0 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.974+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee815f710 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.975+0000 7f4ede000700 1 -- 172.21.15.99:0/4033465616 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f4ed8007470 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.976+0000 7f4ede000700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ec806ac90 0x7f4ec806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:27.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.976+0000 7f4ede000700 1 -- 172.21.15.99:0/4033465616 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f4ed80891f0 con 0x7f4ee80b3cc0 2024-03-29T09:12:27.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.976+0000 7f4ee5400700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ec806ac90 0x7f4ec806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:27.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.977+0000 7f4ee5400700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ec806ac90 0x7f4ec806d150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40090b0 tx=0x7f4ed401c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:27.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:27.981+0000 7f4ede000700 1 -- 172.21.15.99:0/4033465616 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f4ed8054190 con 0x7f4ee80b3cc0 2024-03-29T09:12:28.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.234+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4ee80678b0 con 0x7f4ee80b3cc0 2024-03-29T09:12:28.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.752+0000 7f4ede000700 1 -- 172.21.15.99:0/4033465616 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4ed80577b0 con 0x7f4ee80b3cc0 2024-03-29T09:12:28.754 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ec806ac90 msgr2=0x7f4ec806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ec806ac90 0x7f4ec806d150 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40090b0 tx=0x7f4ed401c040 comp rx=0 tx=0).stop 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 msgr2=0x7f4ee81540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee81540d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c520 tx=0x7f4ed800c830 comp rx=0 tx=0).stop 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 shutdown_connections 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f4ec806ac90 0x7f4ec806d150 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ee8160770 0x7f4ee806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee80b3cc0 0x7f4ee81540d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 --2- 172.21.15.99:0/4033465616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ee80d3380 0x7f4ee80b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 >> 172.21.15.99:0/4033465616 conn(0x7f4ee808afd0 msgr2=0x7f4ee806ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 shutdown_connections 2024-03-29T09:12:28.755 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:28.755+0000 7f4eed000700 1 -- 172.21.15.99:0/4033465616 wait complete. 2024-03-29T09:12:28.775 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T09:12:29.009 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.010+0000 7f088f800700 1 Processor -- start 2024-03-29T09:12:29.010 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.011+0000 7f088f800700 1 -- start start 2024-03-29T09:12:29.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.012+0000 7f088f800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f08880b3ba0 0x7f08880ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.012+0000 7f088f800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880d1fa0 con 0x7f0888160b30 2024-03-29T09:12:29.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.012+0000 7f088f800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0888160f10 con 0x7f08880d0260 2024-03-29T09:12:29.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.012+0000 7f088f800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880cf440 con 0x7f08880b3ba0 2024-03-29T09:12:29.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.012+0000 7f088ce00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f0888160b30 0x7f08880cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40424/0 (socket says 172.21.15.99:40424) 2024-03-29T09:12:29.011 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.012+0000 7f088ce00700 1 -- 172.21.15.99:0/3346091719 learned_addr learned my addr 172.21.15.99:0/3346091719 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.012+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515486806 0 0) 0x7f08880d1fa0 con 0x7f0888160b30 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08700035b0 con 0x7f0888160b30 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f088d800700 1 --2- 172.21.15.99:0/3346091719 >> v2:172.21.15.133:3300/0 conn(0x7f08880b3ba0 0x7f08880ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4126218263 0 0) 0x7f08700035b0 con 0x7f0888160b30 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 >> v2:172.21.15.133:3300/0 conn(0x7f08880b3ba0 msgr2=0x7f08880ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 --2- 172.21.15.99:0/3346091719 >> v2:172.21.15.133:3300/0 conn(0x7f08880b3ba0 0x7f08880ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 >> v1:172.21.15.133:6789/0 conn(0x7f08880d0260 legacy=0x7f08880cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880d1670 con 0x7f0888160b30 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f087c001670 con 0x7f0888160b30 2024-03-29T09:12:29.014 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 >> v1:172.21.15.99:6789/0 conn(0x7f0888160b30 legacy=0x7f08880cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0870003a50 0x7f0870003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0870004850 0x7f0870004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870005210 0x7f087000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087000c0e0 con 0x7f0870004850 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087000c250 con 0x7f0870003a50 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087000c410 con 0x7f0870005210 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f0887e00700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0870004850 0x7f0870004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f088d800700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870005210 0x7f087000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.013+0000 7f088ce00700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0870003a50 0x7f0870003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887e00700 1 -- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870005210 msgr2=0x7f087000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887e00700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870005210 0x7f087000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887e00700 1 -- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0870003a50 msgr2=0x7f0870003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887e00700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0870003a50 0x7f0870003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:29.015 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887e00700 1 -- 172.21.15.99:0/3346091719 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700035b0 con 0x7f0870004850 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f088d800700 1 -- 172.21.15.99:0/3346091719 reap_dead start 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887e00700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0870004850 0x7f0870004cd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f08740017c0 tx=0x7f0870005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ffb714589abe91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740052d0 con 0x7f0870004850 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0874005430 con 0x7f0870004850 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.014+0000 7f0887400700 1 -- 172.21.15.99:0/3346091719 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08740038c0 con 0x7f0870004850 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.015+0000 7f088f800700 1 -- 172.21.15.99:0/3346091719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0870004850 msgr2=0x7f0870004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.015+0000 7f088f800700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0870004850 0x7f0870004cd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f08740017c0 tx=0x7f0870005880 comp rx=0 tx=0).stop 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.015+0000 7f088f800700 1 -- 172.21.15.99:0/3346091719 shutdown_connections 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.015+0000 7f088f800700 1 --2- 172.21.15.99:0/3346091719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0870004850 0x7f0870004cd0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.015+0000 7f088f800700 1 -- 172.21.15.99:0/3346091719 >> 172.21.15.99:0/3346091719 conn(0x7f088809c4d0 msgr2=0x7f08880d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.015+0000 7f088f800700 1 -- 172.21.15.99:0/3346091719 shutdown_connections 2024-03-29T09:12:29.016 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.015+0000 7f088f800700 1 -- 172.21.15.99:0/3346091719 wait complete. 2024-03-29T09:12:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.017+0000 7f088f800700 1 Processor -- start 2024-03-29T09:12:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.017+0000 7f088f800700 1 -- start start 2024-03-29T09:12:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.017+0000 7f088f800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870004850 0x7f088814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088f800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08881f2660 0x7f0888150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.018 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088f800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088806c5b0 0x7f088806ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088f800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088806cf70 con 0x7f08881f2660 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088f800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088814e0c0 con 0x7f088806c5b0 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088f800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0888150120 con 0x7f0870004850 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08881f2660 0x7f0888150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08881f2660 0x7f0888150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59882/0 (socket says 172.21.15.99:59882) 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 -- 172.21.15.99:0/2496045202 learned_addr learned my addr 172.21.15.99:0/2496045202 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088ce00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870004850 0x7f088814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088d800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088806c5b0 0x7f088806ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 -- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870004850 msgr2=0x7f088814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870004850 0x7f088814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 -- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088806c5b0 msgr2=0x7f088806ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088806c5b0 0x7f088806ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:29.019 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 -- 172.21.15.99:0/2496045202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088814d4b0 con 0x7f08881f2660 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088ce00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870004850 0x7f088814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f088d800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088806c5b0 0x7f088806ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.018+0000 7f0887e00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08881f2660 0x7f0888150630 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f087000c220 tx=0x7f0874001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.019+0000 7f0884c00700 1 -- 172.21.15.99:0/2496045202 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874001290 con 0x7f08881f2660 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.019+0000 7f0884c00700 1 -- 172.21.15.99:0/2496045202 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087401d480 con 0x7f08881f2660 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.019+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088814d790 con 0x7f08881f2660 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.019+0000 7f0884c00700 1 -- 172.21.15.99:0/2496045202 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087401d830 con 0x7f08881f2660 2024-03-29T09:12:29.020 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.019+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088814dd50 con 0x7f08881f2660 2024-03-29T09:12:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.020+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08880678b0 con 0x7f08881f2660 2024-03-29T09:12:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.021+0000 7f0884c00700 1 -- 172.21.15.99:0/2496045202 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f0874019040 con 0x7f08881f2660 2024-03-29T09:12:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.021+0000 7f0884c00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f086806ac90 0x7f086806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.022+0000 7f0884c00700 1 -- 172.21.15.99:0/2496045202 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f087408a0e0 con 0x7f08881f2660 2024-03-29T09:12:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.022+0000 7f088ce00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f086806ac90 0x7f086806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:29.021 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.022+0000 7f088ce00700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f086806ac90 0x7f086806d150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f087c001eb0 tx=0x7f087c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:29.024 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.026+0000 7f0884c00700 1 -- 172.21.15.99:0/2496045202 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f0874055230 con 0x7f08881f2660 2024-03-29T09:12:29.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:29.230+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f08880d3be0 con 0x7f08881f2660 2024-03-29T09:12:30.082 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.083+0000 7f0884c00700 1 -- 172.21.15.99:0/2496045202 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 181+0+0 (secure 0 0 0) 0x7f08740170d0 con 0x7f08881f2660 2024-03-29T09:12:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f086806ac90 msgr2=0x7f086806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f086806ac90 0x7f086806d150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f087c001eb0 tx=0x7f087c009040 comp rx=0 tx=0).stop 2024-03-29T09:12:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08881f2660 msgr2=0x7f0888150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:30.086 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08881f2660 0x7f0888150630 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f087000c220 tx=0x7f0874001120 comp rx=0 tx=0).stop 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 shutdown_connections 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f086806ac90 0x7f086806d150 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f088806c5b0 0x7f088806ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0870004850 0x7f088814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 --2- 172.21.15.99:0/2496045202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08881f2660 0x7f0888150630 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.087+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 >> 172.21.15.99:0/2496045202 conn(0x7f088809c4d0 msgr2=0x7f08880c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.088+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 shutdown_connections 2024-03-29T09:12:30.087 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.088+0000 7f088f800700 1 -- 172.21.15.99:0/2496045202 wait complete. 2024-03-29T09:12:30.088 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_metadata' removed 2024-03-29T09:12:30.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T09:12:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.308+0000 7f9471600700 1 Processor -- start 2024-03-29T09:12:30.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.308+0000 7f9471600700 1 -- start start 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f9471600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f946c160b30 0x7f946c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f9471600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f946c0d0090 0x7f946c0ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f9471600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0d1fa0 con 0x7f946c0b1f50 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f9471600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0cf440 con 0x7f946c160b30 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f9471600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0d0500 con 0x7f946c0d0090 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f946aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f946c160b30 0x7f946c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f946aa00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f946c160b30 0x7f946c0cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59888/0 (socket says 172.21.15.99:59888) 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.309+0000 7f946aa00700 1 -- 172.21.15.99:0/2239593326 learned_addr learned my addr 172.21.15.99:0/2239593326 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f946aa00700 1 -- 172.21.15.99:0/2239593326 >> v2:172.21.15.133:3300/0 conn(0x7f946c0d0090 msgr2=0x7f946c0ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:30.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f946aa00700 1 --2- 172.21.15.99:0/2239593326 >> v2:172.21.15.133:3300/0 conn(0x7f946c0d0090 0x7f946c0ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f946aa00700 1 -- 172.21.15.99:0/2239593326 >> v1:172.21.15.133:6789/0 conn(0x7f946c0b1f50 legacy=0x7f946c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f946aa00700 1 -- 172.21.15.99:0/2239593326 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c160f10 con 0x7f946c160b30 2024-03-29T09:12:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f946aa00700 1 --2- 172.21.15.99:0/2239593326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c160b30 0x7f946c0cfb50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f946c08c010 tx=0x7f945c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5df081f93a43588 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f9469600700 1 -- 172.21.15.99:0/2239593326 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c004230 con 0x7f946c160b30 2024-03-29T09:12:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f9469600700 1 -- 172.21.15.99:0/2239593326 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c004390 con 0x7f946c160b30 2024-03-29T09:12:30.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.310+0000 7f9469600700 1 -- 172.21.15.99:0/2239593326 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0036a0 con 0x7f946c160b30 2024-03-29T09:12:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 -- 172.21.15.99:0/2239593326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c160b30 msgr2=0x7f946c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 --2- 172.21.15.99:0/2239593326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c160b30 0x7f946c0cfb50 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f946c08c010 tx=0x7f945c009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:30.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 -- 172.21.15.99:0/2239593326 shutdown_connections 2024-03-29T09:12:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 --2- 172.21.15.99:0/2239593326 >> v2:172.21.15.133:3300/0 conn(0x7f946c0d0090 0x7f946c0ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 --2- 172.21.15.99:0/2239593326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c160b30 0x7f946c0cfb50 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 -- 172.21.15.99:0/2239593326 >> 172.21.15.99:0/2239593326 conn(0x7f946c09c4d0 msgr2=0x7f946c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 -- 172.21.15.99:0/2239593326 shutdown_connections 2024-03-29T09:12:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.311+0000 7f9471600700 1 -- 172.21.15.99:0/2239593326 wait complete. 2024-03-29T09:12:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.313+0000 7f9471600700 1 Processor -- start 2024-03-29T09:12:30.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.313+0000 7f9471600700 1 -- start start 2024-03-29T09:12:30.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f9471600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f946c0b1f50 0x7f946c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:30.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f9471600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c0d0090 0x7f946c0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:30.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f9471600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c160b30 0x7f946c0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f9471600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c14c130 con 0x7f946c0d0090 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f9471600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0cbe30 con 0x7f946c160b30 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f9471600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0cc110 con 0x7f946c0b1f50 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c0d0090 0x7f946c0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c0d0090 0x7f946c0cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59902/0 (socket says 172.21.15.99:59902) 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 -- 172.21.15.99:0/1915725765 learned_addr learned my addr 172.21.15.99:0/1915725765 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946aa00700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f946c0b1f50 0x7f946c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946b400700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c160b30 0x7f946c0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 -- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f946c0b1f50 msgr2=0x7f946c14dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f946c0b1f50 0x7f946c14dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 -- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c160b30 msgr2=0x7f946c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c160b30 0x7f946c0cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:30.315 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.314+0000 7f946a000700 1 -- 172.21.15.99:0/1915725765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c009550 con 0x7f946c0d0090 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.315+0000 7f946a000700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c0d0090 0x7f946c0cb2c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f945800c4a0 tx=0x7f945800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.315+0000 7f9462a00700 1 -- 172.21.15.99:0/1915725765 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580054e0 con 0x7f946c0d0090 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.315+0000 7f9462a00700 1 -- 172.21.15.99:0/1915725765 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458003680 con 0x7f946c0d0090 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.315+0000 7f9462a00700 1 -- 172.21.15.99:0/1915725765 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458003a30 con 0x7f946c0d0090 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.315+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0cc3f0 con 0x7f946c0d0090 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.315+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c0cc9b0 con 0x7f946c0d0090 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.316+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0678b0 con 0x7f946c0d0090 2024-03-29T09:12:30.316 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.317+0000 7f9462a00700 1 -- 172.21.15.99:0/1915725765 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9458003b90 con 0x7f946c0d0090 2024-03-29T09:12:30.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.318+0000 7f9462a00700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f944c06ac90 0x7f944c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:30.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.319+0000 7f9462a00700 1 -- 172.21.15.99:0/1915725765 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5677+0+0 (secure 0 0 0) 0x7f9458088220 con 0x7f946c0d0090 2024-03-29T09:12:30.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.319+0000 7f946aa00700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f944c06ac90 0x7f944c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:30.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.319+0000 7f946aa00700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f944c06ac90 0x7f944c06d150 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f945c009870 tx=0x7f945c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:30.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.323+0000 7f9462a00700 1 -- 172.21.15.99:0/1915725765 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9458053420 con 0x7f946c0d0090 2024-03-29T09:12:30.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:30.528+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f946c0cf7c0 con 0x7f946c0d0090 2024-03-29T09:12:31.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.097+0000 7f9462a00700 1 -- 172.21.15.99:0/1915725765 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9458056a40 con 0x7f946c0d0090 2024-03-29T09:12:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f944c06ac90 msgr2=0x7f944c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f944c06ac90 0x7f944c06d150 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f945c009870 tx=0x7f945c008030 comp rx=0 tx=0).stop 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c0d0090 msgr2=0x7f946c0cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c0d0090 0x7f946c0cb2c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f945800c4a0 tx=0x7f945800c7b0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 shutdown_connections 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f944c06ac90 0x7f944c06d150 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c160b30 0x7f946c0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f946c0b1f50 0x7f946c14dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 --2- 172.21.15.99:0/1915725765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f946c0d0090 0x7f946c0cb2c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 >> 172.21.15.99:0/1915725765 conn(0x7f946c09c4d0 msgr2=0x7f946c0b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 shutdown_connections 2024-03-29T09:12:31.099 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.099+0000 7f9471600700 1 -- 172.21.15.99:0/1915725765 wait complete. 2024-03-29T09:12:31.100 INFO:teuthology.orchestra.run.smithi099.stderr:pool 'cephfs_data' removed 2024-03-29T09:12:31.128 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T09:12:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.321+0000 7f1104800700 1 Processor -- start 2024-03-29T09:12:31.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.321+0000 7f1104800700 1 -- start start 2024-03-29T09:12:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1104800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f10fc0cf090 0x7f10fc167f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1104800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc0d1520 con 0x7f10fc160770 2024-03-29T09:12:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1104800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10fc0d1800 con 0x7f10fc0b3cc0 2024-03-29T09:12:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1104800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10fc0b4140 con 0x7f10fc0cf090 2024-03-29T09:12:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1101e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f10fc160770 0x7f10fc0d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40436/0 (socket says 172.21.15.99:40436) 2024-03-29T09:12:31.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1101e00700 1 -- 172.21.15.99:0/184168820 learned_addr learned my addr 172.21.15.99:0/184168820 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1102800700 1 --2- 172.21.15.99:0/184168820 >> v2:172.21.15.133:3300/0 conn(0x7f10fc0cf090 0x7f10fc167f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.322+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186476693 0 0) 0x7f10fc0d1520 con 0x7f10fc160770 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e40035b0 con 0x7f10fc160770 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1658352715 0 0) 0x7f10e40035b0 con 0x7f10fc160770 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 >> v2:172.21.15.133:3300/0 conn(0x7f10fc0cf090 msgr2=0x7f10fc167f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 --2- 172.21.15.99:0/184168820 >> v2:172.21.15.133:3300/0 conn(0x7f10fc0cf090 0x7f10fc167f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 >> v1:172.21.15.133:6789/0 conn(0x7f10fc0b3cc0 legacy=0x7f10fc0d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc0d1ae0 con 0x7f10fc160770 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f10f0001670 con 0x7f10fc160770 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 >> v1:172.21.15.99:6789/0 conn(0x7f10fc160770 legacy=0x7f10fc0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.324 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e4003a00 0x7f10e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4004850 0x7f10e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10e4005210 0x7f10e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e400c080 con 0x7f10e4003a00 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e400c1f0 con 0x7f10e4004850 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e400c3b0 con 0x7f10e4005210 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1101e00700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e4003a00 0x7f10e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.323+0000 7f1101400700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4004850 0x7f10e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1102800700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10e4005210 0x7f10e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1101e00700 1 -- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10e4005210 msgr2=0x7f10e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1101e00700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10e4005210 0x7f10e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1101e00700 1 -- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4004850 msgr2=0x7f10e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1101e00700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4004850 0x7f10e4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.325 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1101e00700 1 -- 172.21.15.99:0/184168820 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40035b0 con 0x7f10e4003a00 2024-03-29T09:12:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1102800700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10e4005210 0x7f10e400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1102800700 1 -- 172.21.15.99:0/184168820 reap_dead start 2024-03-29T09:12:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1101e00700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e4003a00 0x7f10e4003e20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f10f000c240 tx=0x7f10e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d2783dd86aaf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0007c10 con 0x7f10e4003a00 2024-03-29T09:12:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f000ce00 con 0x7f10e4003a00 2024-03-29T09:12:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.324+0000 7f1100a00700 1 -- 172.21.15.99:0/184168820 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0005ba0 con 0x7f10e4003a00 2024-03-29T09:12:31.326 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.325+0000 7f1104800700 1 -- 172.21.15.99:0/184168820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e4003a00 msgr2=0x7f10e4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.325+0000 7f1104800700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e4003a00 0x7f10e4003e20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f10f000c240 tx=0x7f10e4005880 comp rx=0 tx=0).stop 2024-03-29T09:12:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.325+0000 7f1104800700 1 -- 172.21.15.99:0/184168820 shutdown_connections 2024-03-29T09:12:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.325+0000 7f1104800700 1 --2- 172.21.15.99:0/184168820 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10e4003a00 0x7f10e4003e20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.325+0000 7f1104800700 1 -- 172.21.15.99:0/184168820 >> 172.21.15.99:0/184168820 conn(0x7f10fc08afd0 msgr2=0x7f10fc09c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.325+0000 7f1104800700 1 -- 172.21.15.99:0/184168820 shutdown_connections 2024-03-29T09:12:31.327 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.325+0000 7f1104800700 1 -- 172.21.15.99:0/184168820 wait complete. 2024-03-29T09:12:31.332 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.327+0000 7f1104800700 1 Processor -- start 2024-03-29T09:12:31.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.327+0000 7f1104800700 1 -- start start 2024-03-29T09:12:31.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.327+0000 7f1104800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4003a00 0x7f10fc15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.327+0000 7f1104800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc06bd40 0x7f10fc06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1104800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10fc0c8250 0x7f10fc0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1104800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc06c700 con 0x7f10fc06bd40 2024-03-29T09:12:31.333 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1104800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc1f5f70 con 0x7f10e4003a00 2024-03-29T09:12:31.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1104800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10fc1f6250 con 0x7f10fc0c8250 2024-03-29T09:12:31.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc06bd40 0x7f10fc06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc06bd40 0x7f10fc06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59932/0 (socket says 172.21.15.99:59932) 2024-03-29T09:12:31.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 -- 172.21.15.99:0/1758303256 learned_addr learned my addr 172.21.15.99:0/1758303256 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:31.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1102800700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10fc0c8250 0x7f10fc0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101e00700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4003a00 0x7f10fc15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 -- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10fc0c8250 msgr2=0x7f10fc0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10fc0c8250 0x7f10fc0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 -- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4003a00 msgr2=0x7f10fc15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4003a00 0x7f10fc15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 -- 172.21.15.99:0/1758303256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc1f6530 con 0x7f10fc06bd40 2024-03-29T09:12:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.328+0000 7f1101400700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc06bd40 0x7f10fc06c1c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00f820 tx=0x7f10ec00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:31.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.329+0000 7f10f6000700 1 -- 172.21.15.99:0/1758303256 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0052e0 con 0x7f10fc06bd40 2024-03-29T09:12:31.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.329+0000 7f10f6000700 1 -- 172.21.15.99:0/1758303256 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec005460 con 0x7f10fc06bd40 2024-03-29T09:12:31.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.329+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10fc1f6810 con 0x7f10fc06bd40 2024-03-29T09:12:31.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.329+0000 7f10f6000700 1 -- 172.21.15.99:0/1758303256 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec0038c0 con 0x7f10fc06bd40 2024-03-29T09:12:31.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.329+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10fc0c6cc0 con 0x7f10fc06bd40 2024-03-29T09:12:31.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.330+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10fc1f6d30 con 0x7f10fc06bd40 2024-03-29T09:12:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.331+0000 7f10f6000700 1 -- 172.21.15.99:0/1758303256 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f10ec003a20 con 0x7f10fc06bd40 2024-03-29T09:12:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.332+0000 7f10f6000700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.332+0000 7f10f6000700 1 -- 172.21.15.99:0/1758303256 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f10ec088f70 con 0x7f10fc06bd40 2024-03-29T09:12:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.332+0000 7f1101e00700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.332+0000 7f1101e00700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f10f0000c00 tx=0x7f10f0007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:31.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.336+0000 7f10f6000700 1 -- 172.21.15.99:0/1758303256 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f10fc1f6d30 con 0x7f10fc06bd40 2024-03-29T09:12:31.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.541+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f10fc1f6d30 con 0x7f10fc06bd40 2024-03-29T09:12:31.544 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.545+0000 7f10f6000700 1 -- 172.21.15.99:0/1758303256 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f10fc1f6d30 con 0x7f10fc06bd40 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 msgr2=0x7f10e006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f10f0000c00 tx=0x7f10f0007000 comp rx=0 tx=0).stop 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc06bd40 msgr2=0x7f10fc06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc06bd40 0x7f10fc06c1c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00f820 tx=0x7f10ec00fbe0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 shutdown_connections 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f10e006ac90 0x7f10e006d150 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10fc0c8250 0x7f10fc0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10e4003a00 0x7f10fc15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 --2- 172.21.15.99:0/1758303256 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f10fc06bd40 0x7f10fc06c1c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 >> 172.21.15.99:0/1758303256 conn(0x7f10fc08afd0 msgr2=0x7f10fc15e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:31.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 shutdown_connections 2024-03-29T09:12:31.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.548+0000 7f1104800700 1 -- 172.21.15.99:0/1758303256 wait complete. 2024-03-29T09:12:31.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-03-29T09:12:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.758+0000 7fe267400700 1 Processor -- start 2024-03-29T09:12:31.757 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.758+0000 7fe267400700 1 -- start start 2024-03-29T09:12:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.759+0000 7fe267400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe26015d690 0x7fe2600cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.759+0000 7fe267400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe26015da70 0x7fe2600cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.759+0000 7fe267400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2600d1740 0x7fe2600d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.759+0000 7fe267400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2600d2110 con 0x7fe26015da70 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.759+0000 7fe267400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2600afde0 con 0x7fe26015d690 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.759+0000 7fe267400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600cf660 con 0x7fe2600d1740 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.759+0000 7fe25fe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe26015da70 0x7fe2600cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe264a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe26015d690 0x7fe2600cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fe26015da70 0x7fe2600cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59944/0 (socket says 172.21.15.99:59944) 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 -- 172.21.15.99:0/720834516 learned_addr learned my addr 172.21.15.99:0/720834516 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe265400700 1 --2- 172.21.15.99:0/720834516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2600d1740 0x7fe2600d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 -- 172.21.15.99:0/720834516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2600d1740 msgr2=0x7fe2600d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 --2- 172.21.15.99:0/720834516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2600d1740 0x7fe2600d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 -- 172.21.15.99:0/720834516 >> v2:172.21.15.133:3300/0 conn(0x7fe26015d690 msgr2=0x7fe2600cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 --2- 172.21.15.99:0/720834516 >> v2:172.21.15.133:3300/0 conn(0x7fe26015d690 0x7fe2600cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 -- 172.21.15.99:0/720834516 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2600d01d0 con 0x7fe26015da70 2024-03-29T09:12:31.760 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25fe00700 1 --2- 172.21.15.99:0/720834516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe26015da70 0x7fe2600cf120 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe2600d30b0 tx=0x7fe24c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14c0ecbcb91c016 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:31.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.760+0000 7fe25f400700 1 -- 172.21.15.99:0/720834516 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c011830 con 0x7fe26015da70 2024-03-29T09:12:31.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe25f400700 1 -- 172.21.15.99:0/720834516 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c011990 con 0x7fe26015da70 2024-03-29T09:12:31.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe25f400700 1 -- 172.21.15.99:0/720834516 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c011d20 con 0x7fe26015da70 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 -- 172.21.15.99:0/720834516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe26015da70 msgr2=0x7fe2600cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 --2- 172.21.15.99:0/720834516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe26015da70 0x7fe2600cf120 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe2600d30b0 tx=0x7fe24c009bd0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 -- 172.21.15.99:0/720834516 shutdown_connections 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 --2- 172.21.15.99:0/720834516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2600d1740 0x7fe2600d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 --2- 172.21.15.99:0/720834516 >> v2:172.21.15.133:3300/0 conn(0x7fe26015d690 0x7fe2600cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 --2- 172.21.15.99:0/720834516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe26015da70 0x7fe2600cf120 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 -- 172.21.15.99:0/720834516 >> 172.21.15.99:0/720834516 conn(0x7fe26009c480 msgr2=0x7fe2600d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 -- 172.21.15.99:0/720834516 shutdown_connections 2024-03-29T09:12:31.762 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.761+0000 7fe267400700 1 -- 172.21.15.99:0/720834516 wait complete. 2024-03-29T09:12:31.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.763+0000 7fe267400700 1 Processor -- start 2024-03-29T09:12:31.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.763+0000 7fe267400700 1 -- start start 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe267400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2600d1740 0x7fe2600ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe267400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe26015d690 0x7fe26014d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe267400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe26015da70 0x7fe2600b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe267400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26014b710 con 0x7fe2600d1740 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe267400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26014ae20 con 0x7fe26015d690 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe267400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2600b8cf0 con 0x7fe26015da70 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe264a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2600d1740 0x7fe2600ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe264a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2600d1740 0x7fe2600ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59946/0 (socket says 172.21.15.99:59946) 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe264a00700 1 -- 172.21.15.99:0/2433112553 learned_addr learned my addr 172.21.15.99:0/2433112553 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:31.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe265400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe26015da70 0x7fe2600b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.764+0000 7fe25fe00700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe26015d690 0x7fe26014d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe264a00700 1 -- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe26015da70 msgr2=0x7fe2600b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe264a00700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe26015da70 0x7fe2600b87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe264a00700 1 -- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe26015d690 msgr2=0x7fe26014d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe264a00700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe26015d690 0x7fe26014d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe264a00700 1 -- 172.21.15.99:0/2433112553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c009550 con 0x7fe2600d1740 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe265400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe26015da70 0x7fe2600b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:31.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe264a00700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2600d1740 0x7fe2600ce960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe25400e9b0 tx=0x7fe25400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:31.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe25cc00700 1 -- 172.21.15.99:0/2433112553 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254009710 con 0x7fe2600d1740 2024-03-29T09:12:31.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe25cc00700 1 -- 172.21.15.99:0/2433112553 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25400c850 con 0x7fe2600d1740 2024-03-29T09:12:31.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe25cc00700 1 -- 172.21.15.99:0/2433112553 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25400cbc0 con 0x7fe2600d1740 2024-03-29T09:12:31.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26014b0c0 con 0x7fe2600d1740 2024-03-29T09:12:31.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.765+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2600b8f90 con 0x7fe2600d1740 2024-03-29T09:12:31.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.767+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26014b9f0 con 0x7fe2600d1740 2024-03-29T09:12:31.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.767+0000 7fe25cc00700 1 -- 172.21.15.99:0/2433112553 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fe25400cd20 con 0x7fe2600d1740 2024-03-29T09:12:31.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.768+0000 7fe25cc00700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe24006ac90 0x7fe24006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:31.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.768+0000 7fe25cc00700 1 -- 172.21.15.99:0/2433112553 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fe254089270 con 0x7fe2600d1740 2024-03-29T09:12:31.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.768+0000 7fe25fe00700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe24006ac90 0x7fe24006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:31.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.769+0000 7fe25fe00700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe24006ac90 0x7fe24006d150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009c10 tx=0x7fe24c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:31.771 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:31.773+0000 7fe25cc00700 1 -- 172.21.15.99:0/2433112553 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fe254054610 con 0x7fe2600d1740 2024-03-29T09:12:32.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.057+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7fe2600677a0 con 0x7fe2600d1740 2024-03-29T09:12:32.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.098+0000 7fe25cc00700 1 -- 172.21.15.99:0/2433112553 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v211) v1 ==== 134+0+0 (secure 0 0 0) 0x7fe254057c30 con 0x7fe2600d1740 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe24006ac90 msgr2=0x7fe24006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe24006ac90 0x7fe24006d150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009c10 tx=0x7fe24c008030 comp rx=0 tx=0).stop 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2600d1740 msgr2=0x7fe2600ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2600d1740 0x7fe2600ce960 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe25400e9b0 tx=0x7fe25400ed70 comp rx=0 tx=0).stop 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 shutdown_connections 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fe24006ac90 0x7fe24006d150 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe26015da70 0x7fe2600b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe2600d1740 0x7fe2600ce960 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 --2- 172.21.15.99:0/2433112553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe26015d690 0x7fe26014d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.100+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 >> 172.21.15.99:0/2433112553 conn(0x7fe26009c480 msgr2=0x7fe26006c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.101+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 shutdown_connections 2024-03-29T09:12:32.100 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:32.101+0000 7fe267400700 1 -- 172.21.15.99:0/2433112553 wait complete. 2024-03-29T09:12:32.119 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2024-03-29T09:12:32.119 INFO:tasks.cephfs_test_runner: 2024-03-29T09:12:32.119 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T09:12:32.119 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-03-29T09:12:32.119 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T09:12:32.119 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/mount.py", line 777, in run_python 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: p.wait() 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner:Ran 2 tests in 150.358s 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-29T09:12:32.120 INFO:tasks.cephfs_test_runner: 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs/mount.py", line 777, in run_python 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: p.wait() 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi099 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-03-29T09:12:32.121 INFO:tasks.cephfs_test_runner: 2024-03-29T09:12:32.122 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-03-29T09:12:32.388 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=ecf4333faa9447eeafbed6cde0cc1976 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-03-29T09:12:32.389 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-29T09:12:32.399 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-29T09:12:32.437 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-29T09:12:32.438 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:12:32.438 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:32.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345608k,nr_inodes=4086402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-03-29T09:12:32.484 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-03-29T09:12:32.484 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:32.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345608k,nr_inodes=4086402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275376k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2024-03-29T09:12:32.566 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:12:32.566 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:32.615 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:12:32.615 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-29T09:12:32.615 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T09:12:32.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345628k,nr_inodes=4086407,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275380k,mode=700,uid=1000,gid=1304 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2024-03-29T09:12:32.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:32.700 DEBUG:teuthology.orchestra.run.smithi099:> ip netns list 2024-03-29T09:12:32.742 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T09:12:32.742 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T09:12:32.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:32.743 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T09:12:32.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:32.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T09:12:32.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:32.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo ip link delete ceph-brx 2024-03-29T09:12:33.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:33.046 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-29T09:12:33.093 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T09:12:33.093 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T09:12:33.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:33.094 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T09:12:33.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:33.165 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T09:12:33.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T09:12:33.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-03-29T09:12:33.342 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-29T09:12:33.351 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T09:12:33.352 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:12:33.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.546+0000 7f8ee4e00700 1 Processor -- start 2024-03-29T09:12:33.545 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.546+0000 7f8ee4e00700 1 -- start start 2024-03-29T09:12:33.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.547+0000 7f8ee4e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee015d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8ee4e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee00ccf60 con 0x7f8ee00cd6b0 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8ee4e00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee015dcd0 con 0x7f8ee0153d20 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8ee4e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee0071750 con 0x7f8ee015d330 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8ede800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f8ee00cd6b0 0x7f8ee00cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40448/0 (socket says 172.21.15.99:40448) 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8ede800700 1 -- 172.21.15.99:0/3704521103 learned_addr learned my addr 172.21.15.99:0/3704521103 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edd400700 1 --2- 172.21.15.99:0/3704521103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edca00700 1 -- 172.21.15.99:0/3704521103 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064705399 0 0) 0x7f8ee00ccf60 con 0x7f8ee00cd6b0 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edca00700 1 -- 172.21.15.99:0/3704521103 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec40035b0 con 0x7f8ee00cd6b0 2024-03-29T09:12:33.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edca00700 1 -- 172.21.15.99:0/3704521103 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651262899 0 0) 0x7f8ee015dcd0 con 0x7f8ee0153d20 2024-03-29T09:12:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edca00700 1 -- 172.21.15.99:0/3704521103 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee00ccf60 con 0x7f8ee0153d20 2024-03-29T09:12:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edd400700 1 -- 172.21.15.99:0/3704521103 >> v1:172.21.15.133:6789/0 conn(0x7f8ee0153d20 legacy=0x7f8ee0070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edd400700 1 -- 172.21.15.99:0/3704521103 >> v1:172.21.15.99:6789/0 conn(0x7f8ee00cd6b0 legacy=0x7f8ee00cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.548+0000 7f8edd400700 1 -- 172.21.15.99:0/3704521103 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee0070c50 con 0x7f8ee015d330 2024-03-29T09:12:33.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.549+0000 7f8edd400700 1 --2- 172.21.15.99:0/3704521103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee015d790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00aa10 tx=0x7f8ecc00ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5796866f5ac03d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:33.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.549+0000 7f8edca00700 1 -- 172.21.15.99:0/3704521103 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00cb70 con 0x7f8ee015d330 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.549+0000 7f8edca00700 1 -- 172.21.15.99:0/3704521103 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc00ccd0 con 0x7f8ee015d330 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.549+0000 7f8edca00700 1 -- 172.21.15.99:0/3704521103 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc010640 con 0x7f8ee015d330 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.549+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3704521103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 msgr2=0x7f8ee015d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.549+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3704521103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee015d790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00aa10 tx=0x7f8ecc00ad20 comp rx=0 tx=0).stop 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.550+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3704521103 shutdown_connections 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.550+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3704521103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee015d790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.550+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3704521103 >> 172.21.15.99:0/3704521103 conn(0x7f8ee0066540 msgr2=0x7f8ee0066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:33.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.550+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3704521103 shutdown_connections 2024-03-29T09:12:33.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.550+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3704521103 wait complete. 2024-03-29T09:12:33.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.551+0000 7f8ee4e00700 1 Processor -- start 2024-03-29T09:12:33.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.552+0000 7f8ee4e00700 1 -- start start 2024-03-29T09:12:33.554 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.552+0000 7f8ee4e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee00cd6b0 0x7f8ee0148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.552+0000 7f8ee4e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee0153d20 0x7f8ee0152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.552+0000 7f8ee4e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee0154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.552+0000 7f8ee4e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee0154af0 con 0x7f8ee0153d20 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.552+0000 7f8ee4e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee0152ab0 con 0x7f8ee00cd6b0 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.552+0000 7f8ee4e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee01544a0 con 0x7f8ee015d330 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee0153d20 0x7f8ee0152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee0153d20 0x7f8ee0152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59962/0 (socket says 172.21.15.99:59962) 2024-03-29T09:12:33.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 -- 172.21.15.99:0/3408454131 learned_addr learned my addr 172.21.15.99:0/3408454131 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 -- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 msgr2=0x7f8ee0154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8ede800700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee0154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edde00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee00cd6b0 0x7f8ee0148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee0154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 -- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee00cd6b0 msgr2=0x7f8ee0148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee00cd6b0 0x7f8ee0148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 -- 172.21.15.99:0/3408454131 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc00a640 con 0x7f8ee0153d20 2024-03-29T09:12:33.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8ede800700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edde00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee00cd6b0 0x7f8ee0148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.553+0000 7f8edd400700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee0153d20 0x7f8ee0152140 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00a960 tx=0x7f8ecc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.554+0000 7f8ed6000700 1 -- 172.21.15.99:0/3408454131 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc010c00 con 0x7f8ee0153d20 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.554+0000 7f8ed6000700 1 -- 172.21.15.99:0/3408454131 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc010d60 con 0x7f8ee0153d20 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.554+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee0154780 con 0x7f8ee0153d20 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.554+0000 7f8ed6000700 1 -- 172.21.15.99:0/3408454131 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc019690 con 0x7f8ee0153d20 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.554+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee0148c40 con 0x7f8ee0153d20 2024-03-29T09:12:33.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.555+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee0154ca0 con 0x7f8ee0153d20 2024-03-29T09:12:33.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.556+0000 7f8ed6000700 1 -- 172.21.15.99:0/3408454131 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f8ecc0197f0 con 0x7f8ee0153d20 2024-03-29T09:12:33.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.557+0000 7f8ed6000700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ec006ac90 0x7f8ec006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.557+0000 7f8ed6000700 1 -- 172.21.15.99:0/3408454131 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f8ecc08a6c0 con 0x7f8ee0153d20 2024-03-29T09:12:33.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.557+0000 7f8edde00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ec006ac90 0x7f8ec006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.558+0000 7f8edde00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ec006ac90 0x7f8ec006d150 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00011e0 tx=0x7f8ed0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:33.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.561+0000 7f8ed6000700 1 -- 172.21.15.99:0/3408454131 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f8ee0154ca0 con 0x7f8ee0153d20 2024-03-29T09:12:33.759 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.760+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ee0071470 con 0x7f8ec006ac90 2024-03-29T09:12:33.761 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.763+0000 7f8ed6000700 1 -- 172.21.15.99:0/3408454131 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50638 (secure 0 0 0) 0x7f8ed0001c70 con 0x7f8ec006ac90 2024-03-29T09:12:33.762 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:33.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ec006ac90 msgr2=0x7f8ec006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ec006ac90 0x7f8ec006d150 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00011e0 tx=0x7f8ed0009040 comp rx=0 tx=0).stop 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee0153d20 msgr2=0x7f8ee0152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee0153d20 0x7f8ee0152140 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00a960 tx=0x7f8ecc004750 comp rx=0 tx=0).stop 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 shutdown_connections 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f8ec006ac90 0x7f8ec006d150 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8ee015d330 0x7f8ee0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee00cd6b0 0x7f8ee0148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 --2- 172.21.15.99:0/3408454131 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ee0153d20 0x7f8ee0152140 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.765 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 >> 172.21.15.99:0/3408454131 conn(0x7f8ee0066540 msgr2=0x7f8ee0066920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:33.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 shutdown_connections 2024-03-29T09:12:33.766 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.765+0000 7f8ee4e00700 1 -- 172.21.15.99:0/3408454131 wait complete. 2024-03-29T09:12:33.766 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:12:33.783 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-03-29T09:12:32.496260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222752,"kb_used_data":6580,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751968,"statfs":{"total":773094113280,"available":772866015232,"internally_reserved":0,"allocated":6737920,"data_stored":3654277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14228,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-64,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001152"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-29T09:12:30.082749+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:12:30.082749+0000","last_peered":"2024-03-29T09:12:30.082749+0000","last_clean":"2024-03-29T09:12:30.082749+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:12:30.082749+0000","last_undegraded":"2024-03-29T09:12:30.082749+0000","last_fullsized":"2024-03-29T09:12:30.082749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.71399999999999997},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":1.639,"15min":1.639},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":0.86399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77600000000000002,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.71699999999999997,"15min":0.76400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72499999999999998,"5min":1.992,"15min":1.992},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.75900000000000001,"15min":0.81000000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.752},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.81899999999999995,"15min":0.86899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85099999999999998,"5min":2.883,"15min":2.883},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.87,"5min":1.387,"15min":1.387},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68500000000000005,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89200000000000002,"5min":1.367,"15min":1.367},"last":0.87}]},{"osd":5,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.458,"15min":1.458},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.88700000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.92200000000000004,"5min":1.413,"15min":1.413},"last":0.84699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":300296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.438,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82299999999999995,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.47799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1.556}]},{"osd":1,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.53600000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75900000000000001,"15min":0.81200000000000006},"min":{"1min":0.49099999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.76}]},{"osd":2,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.8779999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.77600000000000002,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91000000000000003,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":2.1360000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":2.581},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":3.3130000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":2.8839999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":2.6680000000000001}]},{"osd":7,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78200000000000003,"15min":0.79800000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90100000000000002,"5min":1.026,"15min":1.026},"last":3.242},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94499999999999995,"5min":1.111,"15min":1.111},"last":2.508}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":896965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.55300000000000005,"15min":0.61599999999999999},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.755,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.52700000000000002,"15min":0.55100000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67200000000000004,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.59999999999999998,"15min":0.66600000000000004},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72199999999999998,"5min":1,"15min":1},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.96199999999999997,"15min":0.93400000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94799999999999995,"5min":12.356,"15min":12.356},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94999999999999996,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.018,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":0.92000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.034,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.80200000000000005,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91800000000000004,"5min":1.99,"15min":1.99},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":1.0109999999999999,"15min":0.97699999999999998},"min":{"1min":0.57799999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99199999999999999,"5min":12.669,"15min":12.669},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.97899999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84999999999999998,"15min":0.872},"min":{"1min":0.628,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97499999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.77800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":316008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57699999999999996,"5min":0.751,"15min":0.751},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59899999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.61799999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45300000000000001,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.54100000000000004,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.61499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.52100000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.022,"5min":1.093,"15min":1.093},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.147,"15min":1.147},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96699999999999997,"5min":1.069,"15min":1.069},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85099999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.755},"min":{"1min":0.58199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97899999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.45700000000000002,"15min":0.48199999999999998},"min":{"1min":0.25600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.59399999999999997,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.49099999999999999,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.252,"15min":1.252},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.51600000000000001,"15min":0.55500000000000005},"min":{"1min":0.31,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.53800000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.495},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58299999999999996,"15min":0.629},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.57799999999999996,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.56799999999999995,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55000000000000004,"5min":2.415,"15min":2.415},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.625,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.66600000000000004,"5min":2.056,"15min":2.056},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.77500000000000002,"15min":0.81699999999999995},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88900000000000001,"5min":2.073,"15min":2.073},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.58299999999999996,"5min":0.496,"15min":0.496},"max":{"1min":0.87,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80500000000000005},"min":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89600000000000002,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.92600000000000005}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.83599999999999997,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92000000000000004,"5min":2.387,"15min":2.387},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.873,"15min":0.93300000000000005},"min":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90200000000000002,"5min":2.996,"15min":2.996},"last":1.036}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.86099999999999999},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90700000000000003,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.81699999999999995,"15min":0.86299999999999999},"min":{"1min":0.53600000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.89000000000000001,"5min":2.165,"15min":2.165},"last":0.91100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.498,"15min":0.54000000000000004},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61499999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.502,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57499999999999996,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.57799999999999996,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55000000000000004,"15min":0.59299999999999997},"min":{"1min":0.311,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.65200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.439}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.58899999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93300000000000005,"5min":2.278,"15min":2.278},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.95299999999999996,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83099999999999996,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.754}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96599999999999997,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.83999999999999997,"15min":0.88700000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.014,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.84899999999999998,"15min":0.89600000000000002},"min":{"1min":0.503,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89000000000000001,"5min":2.649,"15min":2.649},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86199999999999999,"15min":0.90200000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.024,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.77800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":306597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.83099999999999996,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.67700000000000005,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.80400000000000005},"min":{"1min":0.60899999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0089999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95499999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.751,"15min":0.80500000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80100000000000005,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.056,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98699999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93600000000000005,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"min":{"1min":0.623,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0289999999999999,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.88700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":896965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.64700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.02,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.73299999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98499999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.495,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86099999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0780000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0329999999999999,"5min":1.05,"15min":1.05},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96899999999999997,"5min":1.079,"15min":1.079},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0580000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:12:33.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:12:33.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.976+0000 7f517f800700 1 Processor -- start 2024-03-29T09:12:33.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.976+0000 7f517f800700 1 -- start start 2024-03-29T09:12:33.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.977+0000 7f517f800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f517815da70 0x7f51780cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.977+0000 7f517f800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51780d1740 0x7f51780d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.977+0000 7f517f800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51780afde0 con 0x7f517815d690 2024-03-29T09:12:33.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.977+0000 7f517f800700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51780d2130 con 0x7f517815da70 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.977+0000 7f517f800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51780d01d0 con 0x7f51780d1740 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f517815da70 0x7f51780cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f517815da70 0x7f51780cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59968/0 (socket says 172.21.15.99:59968) 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 -- 172.21.15.99:0/2726593034 learned_addr learned my addr 172.21.15.99:0/2726593034 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f517d800700 1 --2- 172.21.15.99:0/2726593034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51780d1740 0x7f51780d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 -- 172.21.15.99:0/2726593034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51780d1740 msgr2=0x7f51780d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 --2- 172.21.15.99:0/2726593034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51780d1740 0x7f51780d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 -- 172.21.15.99:0/2726593034 >> v1:172.21.15.133:6789/0 conn(0x7f517815d690 legacy=0x7f51780cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 -- 172.21.15.99:0/2726593034 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51780d04b0 con 0x7f517815da70 2024-03-29T09:12:33.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177400700 1 -- 172.21.15.99:0/2726593034 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758726146 0 0) 0x7f51780afde0 con 0x7f517815d690 2024-03-29T09:12:33.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.978+0000 7f5177e00700 1 --2- 172.21.15.99:0/2726593034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517815da70 0x7f51780cfb60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f51640098f0 tx=0x7f5164009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d553df02155fe7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:33.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f5177400700 1 -- 172.21.15.99:0/2726593034 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164011830 con 0x7f517815da70 2024-03-29T09:12:33.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f5177400700 1 -- 172.21.15.99:0/2726593034 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5164011990 con 0x7f517815da70 2024-03-29T09:12:33.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f5177400700 1 -- 172.21.15.99:0/2726593034 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5164011d20 con 0x7f517815da70 2024-03-29T09:12:33.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 -- 172.21.15.99:0/2726593034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517815da70 msgr2=0x7f51780cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 --2- 172.21.15.99:0/2726593034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517815da70 0x7f51780cfb60 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f51640098f0 tx=0x7f5164009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:33.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 -- 172.21.15.99:0/2726593034 shutdown_connections 2024-03-29T09:12:33.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 --2- 172.21.15.99:0/2726593034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51780d1740 0x7f51780d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 --2- 172.21.15.99:0/2726593034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f517815da70 0x7f51780cfb60 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 -- 172.21.15.99:0/2726593034 >> 172.21.15.99:0/2726593034 conn(0x7f517809c480 msgr2=0x7f51780d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:33.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 -- 172.21.15.99:0/2726593034 shutdown_connections 2024-03-29T09:12:33.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.979+0000 7f517f800700 1 -- 172.21.15.99:0/2726593034 wait complete. 2024-03-29T09:12:33.985 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.981+0000 7f517f800700 1 Processor -- start 2024-03-29T09:12:33.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.981+0000 7f517f800700 1 -- start start 2024-03-29T09:12:33.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517f800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51780d1740 0x7f517815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517f800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f517815d690 0x7f51780c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517f800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f517815da70 0x7f517806b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517f800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51780c8960 con 0x7f51780d1740 2024-03-29T09:12:33.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517f800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51780c8c40 con 0x7f517815da70 2024-03-29T09:12:33.986 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517f800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51780c8f20 con 0x7f517815d690 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51780d1740 0x7f517815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51780d1740 0x7f517815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59972/0 (socket says 172.21.15.99:59972) 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f517ce00700 1 -- 172.21.15.99:0/942944059 learned_addr learned my addr 172.21.15.99:0/942944059 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.982+0000 7f5177e00700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f517815d690 0x7f51780c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517d800700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f517815da70 0x7f517806b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517ce00700 1 -- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f517815d690 msgr2=0x7f51780c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517ce00700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f517815d690 0x7f51780c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517ce00700 1 -- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f517815da70 msgr2=0x7f517806b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517ce00700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f517815da70 0x7f517806b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517ce00700 1 -- 172.21.15.99:0/942944059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164009550 con 0x7f51780d1740 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517ce00700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51780d1740 0x7f517815a7f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f516c0011e0 tx=0x7f516c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f5174c00700 1 -- 172.21.15.99:0/942944059 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c0054e0 con 0x7f51780d1740 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f5174c00700 1 -- 172.21.15.99:0/942944059 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c003680 con 0x7f51780d1740 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f5174c00700 1 -- 172.21.15.99:0/942944059 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c003a30 con 0x7f51780d1740 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51780c9200 con 0x7f51780d1740 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.983+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517815ad30 con 0x7f51780d1740 2024-03-29T09:12:33.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.984+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51780c9720 con 0x7f51780d1740 2024-03-29T09:12:33.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.985+0000 7f5174c00700 1 -- 172.21.15.99:0/942944059 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f516c003b90 con 0x7f51780d1740 2024-03-29T09:12:33.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.987+0000 7f5174c00700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f515806ac90 0x7f515806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:33.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.987+0000 7f5174c00700 1 -- 172.21.15.99:0/942944059 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f516c0885b0 con 0x7f51780d1740 2024-03-29T09:12:33.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.987+0000 7f5177e00700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f515806ac90 0x7f515806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:33.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.987+0000 7f5177e00700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f515806ac90 0x7f515806d150 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5164014040 tx=0x7f5164008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:33.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:33.991+0000 7f5174c00700 1 -- 172.21.15.99:0/942944059 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f51780c9720 con 0x7f51780d1740 2024-03-29T09:12:34.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.189+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51780c8db0 con 0x7f515806ac90 2024-03-29T09:12:34.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.192+0000 7f5174c00700 1 -- 172.21.15.99:0/942944059 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50638 (secure 0 0 0) 0x7f51780c8db0 con 0x7f515806ac90 2024-03-29T09:12:34.191 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:34.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f515806ac90 msgr2=0x7f515806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f515806ac90 0x7f515806d150 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5164014040 tx=0x7f5164008030 comp rx=0 tx=0).stop 2024-03-29T09:12:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51780d1740 msgr2=0x7f517815a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51780d1740 0x7f517815a7f0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f516c0011e0 tx=0x7f516c00ca30 comp rx=0 tx=0).stop 2024-03-29T09:12:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 shutdown_connections 2024-03-29T09:12:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f515806ac90 0x7f515806d150 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f517815da70 0x7f517806b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.194 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f51780d1740 0x7f517815a7f0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 --2- 172.21.15.99:0/942944059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f517815d690 0x7f51780c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.194+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 >> 172.21.15.99:0/942944059 conn(0x7f517809c480 msgr2=0x7f51780b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:34.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.195+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 shutdown_connections 2024-03-29T09:12:34.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.195+0000 7f517f800700 1 -- 172.21.15.99:0/942944059 wait complete. 2024-03-29T09:12:34.195 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:12:34.212 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-03-29T09:12:32.496260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222752,"kb_used_data":6580,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751968,"statfs":{"total":773094113280,"available":772866015232,"internally_reserved":0,"allocated":6737920,"data_stored":3654277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14228,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-64,"num_write_kb":-35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001152"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-29T09:12:30.082749+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:12:30.082749+0000","last_peered":"2024-03-29T09:12:30.082749+0000","last_clean":"2024-03-29T09:12:30.082749+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:12:30.082749+0000","last_undegraded":"2024-03-29T09:12:30.082749+0000","last_fullsized":"2024-03-29T09:12:30.082749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.71399999999999997},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":1.639,"15min":1.639},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":0.86399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77600000000000002,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.71699999999999997,"15min":0.76400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72499999999999998,"5min":1.992,"15min":1.992},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.75900000000000001,"15min":0.81000000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.752},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.81899999999999995,"15min":0.86899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85099999999999998,"5min":2.883,"15min":2.883},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.87,"5min":1.387,"15min":1.387},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68500000000000005,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89200000000000002,"5min":1.367,"15min":1.367},"last":0.87}]},{"osd":5,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.458,"15min":1.458},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.88700000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.92200000000000004,"5min":1.413,"15min":1.413},"last":0.84699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":300296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.438,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82299999999999995,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.47799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1.556}]},{"osd":1,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.53600000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75900000000000001,"15min":0.81200000000000006},"min":{"1min":0.49099999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.76}]},{"osd":2,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.8779999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.77600000000000002,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91000000000000003,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":2.1360000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":2.581},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":3.3130000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":2.8839999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":2.6680000000000001}]},{"osd":7,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78200000000000003,"15min":0.79800000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90100000000000002,"5min":1.026,"15min":1.026},"last":3.242},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94499999999999995,"5min":1.111,"15min":1.111},"last":2.508}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":896965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.55300000000000005,"15min":0.61599999999999999},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.755,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.52700000000000002,"15min":0.55100000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67200000000000004,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.59999999999999998,"15min":0.66600000000000004},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72199999999999998,"5min":1,"15min":1},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.96199999999999997,"15min":0.93400000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94799999999999995,"5min":12.356,"15min":12.356},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94999999999999996,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.018,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":0.92000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.034,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.80200000000000005,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91800000000000004,"5min":1.99,"15min":1.99},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":1.0109999999999999,"15min":0.97699999999999998},"min":{"1min":0.57799999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99199999999999999,"5min":12.669,"15min":12.669},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.97899999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84999999999999998,"15min":0.872},"min":{"1min":0.628,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97499999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.77800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":316008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57699999999999996,"5min":0.751,"15min":0.751},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59899999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.61799999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45300000000000001,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.54100000000000004,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.61499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.52100000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.022,"5min":1.093,"15min":1.093},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.147,"15min":1.147},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96699999999999997,"5min":1.069,"15min":1.069},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85099999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.755},"min":{"1min":0.58199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97899999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.45700000000000002,"15min":0.48199999999999998},"min":{"1min":0.25600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.59399999999999997,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.49099999999999999,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.252,"15min":1.252},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.51600000000000001,"15min":0.55500000000000005},"min":{"1min":0.31,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.53800000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.495},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58299999999999996,"15min":0.629},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.57799999999999996,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.56799999999999995,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55000000000000004,"5min":2.415,"15min":2.415},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.625,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.66600000000000004,"5min":2.056,"15min":2.056},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.77500000000000002,"15min":0.81699999999999995},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88900000000000001,"5min":2.073,"15min":2.073},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.58299999999999996,"5min":0.496,"15min":0.496},"max":{"1min":0.87,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80500000000000005},"min":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89600000000000002,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.92600000000000005}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.83599999999999997,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92000000000000004,"5min":2.387,"15min":2.387},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.873,"15min":0.93300000000000005},"min":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90200000000000002,"5min":2.996,"15min":2.996},"last":1.036}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.86099999999999999},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90700000000000003,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.81699999999999995,"15min":0.86299999999999999},"min":{"1min":0.53600000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.89000000000000001,"5min":2.165,"15min":2.165},"last":0.91100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.498,"15min":0.54000000000000004},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61499999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.502,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57499999999999996,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.57799999999999996,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55000000000000004,"15min":0.59299999999999997},"min":{"1min":0.311,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.65200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.439}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.58899999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93300000000000005,"5min":2.278,"15min":2.278},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.95299999999999996,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83099999999999996,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.754}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96599999999999997,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.83999999999999997,"15min":0.88700000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.014,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.84899999999999998,"15min":0.89600000000000002},"min":{"1min":0.503,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89000000000000001,"5min":2.649,"15min":2.649},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86199999999999999,"15min":0.90200000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.024,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.77800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":306597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.83099999999999996,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.67700000000000005,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.80400000000000005},"min":{"1min":0.60899999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0089999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95499999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.751,"15min":0.80500000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80100000000000005,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.056,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98699999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93600000000000005,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"min":{"1min":0.623,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0289999999999999,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.88700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":896965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.64700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.02,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.73299999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98499999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.495,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86099999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0780000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0329999999999999,"5min":1.05,"15min":1.05},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96899999999999997,"5min":1.079,"15min":1.079},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0580000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:12:34.214 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T09:12:34.214 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:12:34.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.405+0000 7f1e65c00700 1 Processor -- start 2024-03-29T09:12:34.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.405+0000 7f1e65c00700 1 -- start start 2024-03-29T09:12:34.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e65c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e601608e0 0x7f1e600d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e65c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60160cc0 0x7f1e600ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e65c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e600d3a70 0x7f1e600d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e65c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e600cddc0 con 0x7f1e600d3a70 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e65c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e600ce0a0 con 0x7f1e601608e0 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e65c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600ce380 con 0x7f1e60160cc0 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e5f400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e600d3a70 0x7f1e600d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e5f400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f1e600d3a70 0x7f1e600d1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59976/0 (socket says 172.21.15.99:59976) 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e5f400700 1 -- 172.21.15.99:0/1354034104 learned_addr learned my addr 172.21.15.99:0/1354034104 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5e000700 1 --2- 172.21.15.99:0/1354034104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60160cc0 0x7f1e600ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.406+0000 7f1e5ea00700 1 --2- 172.21.15.99:0/1354034104 >> v2:172.21.15.133:3300/0 conn(0x7f1e601608e0 0x7f1e600d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5f400700 1 -- 172.21.15.99:0/1354034104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60160cc0 msgr2=0x7f1e600ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5f400700 1 --2- 172.21.15.99:0/1354034104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60160cc0 0x7f1e600ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5f400700 1 -- 172.21.15.99:0/1354034104 >> v2:172.21.15.133:3300/0 conn(0x7f1e601608e0 msgr2=0x7f1e600d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5f400700 1 --2- 172.21.15.99:0/1354034104 >> v2:172.21.15.133:3300/0 conn(0x7f1e601608e0 0x7f1e600d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5f400700 1 -- 172.21.15.99:0/1354034104 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e600ce660 con 0x7f1e600d3a70 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5e000700 1 --2- 172.21.15.99:0/1354034104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60160cc0 0x7f1e600ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5f400700 1 --2- 172.21.15.99:0/1354034104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d3a70 0x7f1e600d1410 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1e6008d790 tx=0x7f1e5800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c41617f7fd53f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5d600700 1 -- 172.21.15.99:0/1354034104 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800e070 con 0x7f1e600d3a70 2024-03-29T09:12:34.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5d600700 1 -- 172.21.15.99:0/1354034104 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5800b910 con 0x7f1e600d3a70 2024-03-29T09:12:34.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.407+0000 7f1e5d600700 1 -- 172.21.15.99:0/1354034104 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800bcc0 con 0x7f1e600d3a70 2024-03-29T09:12:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 -- 172.21.15.99:0/1354034104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d3a70 msgr2=0x7f1e600d1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 --2- 172.21.15.99:0/1354034104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d3a70 0x7f1e600d1410 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1e6008d790 tx=0x7f1e5800b330 comp rx=0 tx=0).stop 2024-03-29T09:12:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 -- 172.21.15.99:0/1354034104 shutdown_connections 2024-03-29T09:12:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 --2- 172.21.15.99:0/1354034104 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e600d3a70 0x7f1e600d1410 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.412 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 --2- 172.21.15.99:0/1354034104 >> v2:172.21.15.133:3300/0 conn(0x7f1e601608e0 0x7f1e600d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 --2- 172.21.15.99:0/1354034104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e60160cc0 0x7f1e600ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 -- 172.21.15.99:0/1354034104 >> 172.21.15.99:0/1354034104 conn(0x7f1e6009c490 msgr2=0x7f1e600d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 -- 172.21.15.99:0/1354034104 shutdown_connections 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.408+0000 7f1e65c00700 1 -- 172.21.15.99:0/1354034104 wait complete. 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.410+0000 7f1e65c00700 1 Processor -- start 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.410+0000 7f1e65c00700 1 -- start start 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e65c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e600d3a70 0x7f1e601f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.413 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e65c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e601608e0 0x7f1e60159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e65c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e60160cc0 0x7f1e600ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e65c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e600ca590 con 0x7f1e60160cc0 2024-03-29T09:12:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e65c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6015b170 con 0x7f1e601608e0 2024-03-29T09:12:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e65c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6015a2c0 con 0x7f1e600d3a70 2024-03-29T09:12:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e60160cc0 0x7f1e600ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e60160cc0 0x7f1e600ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:59988/0 (socket says 172.21.15.99:59988) 2024-03-29T09:12:34.414 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 -- 172.21.15.99:0/320334055 learned_addr learned my addr 172.21.15.99:0/320334055 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5e000700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e601608e0 0x7f1e60159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5ea00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e600d3a70 0x7f1e601f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 -- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e600d3a70 msgr2=0x7f1e601f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e600d3a70 0x7f1e601f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 -- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e601608e0 msgr2=0x7f1e60159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e601608e0 0x7f1e60159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.411+0000 7f1e5f400700 1 -- 172.21.15.99:0/320334055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e58009b30 con 0x7f1e60160cc0 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.412+0000 7f1e5f400700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e60160cc0 0x7f1e600ccf70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1e6008c9a0 tx=0x7f1e58004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:34.415 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.412+0000 7f1e56a00700 1 -- 172.21.15.99:0/320334055 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580099b0 con 0x7f1e60160cc0 2024-03-29T09:12:34.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.412+0000 7f1e56a00700 1 -- 172.21.15.99:0/320334055 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e58004250 con 0x7f1e60160cc0 2024-03-29T09:12:34.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.412+0000 7f1e56a00700 1 -- 172.21.15.99:0/320334055 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e58004600 con 0x7f1e60160cc0 2024-03-29T09:12:34.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.412+0000 7f1e65c00700 1 -- 172.21.15.99:0/320334055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6015a560 con 0x7f1e60160cc0 2024-03-29T09:12:34.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.412+0000 7f1e65c00700 1 -- 172.21.15.99:0/320334055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6015ab20 con 0x7f1e60160cc0 2024-03-29T09:12:34.416 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.413+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e340052f0 con 0x7f1e60160cc0 2024-03-29T09:12:34.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.414+0000 7f1e56a00700 1 -- 172.21.15.99:0/320334055 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f1e58004760 con 0x7f1e60160cc0 2024-03-29T09:12:34.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.415+0000 7f1e56a00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e4006ac90 0x7f1e4006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.416+0000 7f1e56a00700 1 -- 172.21.15.99:0/320334055 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1e5808a6e0 con 0x7f1e60160cc0 2024-03-29T09:12:34.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.416+0000 7f1e5ea00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e4006ac90 0x7f1e4006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.416+0000 7f1e5ea00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e4006ac90 0x7f1e4006d150 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009af0 tx=0x7f1e50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:34.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.420+0000 7f1e56a00700 1 -- 172.21.15.99:0/320334055 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1e580559e0 con 0x7f1e60160cc0 2024-03-29T09:12:34.619 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.620+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e34000bc0 con 0x7f1e4006ac90 2024-03-29T09:12:34.621 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.622+0000 7f1e56a00700 1 -- 172.21.15.99:0/320334055 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50638 (secure 0 0 0) 0x7f1e34000bc0 con 0x7f1e4006ac90 2024-03-29T09:12:34.622 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:34.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e4006ac90 msgr2=0x7f1e4006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e4006ac90 0x7f1e4006d150 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009af0 tx=0x7f1e50008040 comp rx=0 tx=0).stop 2024-03-29T09:12:34.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e60160cc0 msgr2=0x7f1e600ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e60160cc0 0x7f1e600ccf70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1e6008c9a0 tx=0x7f1e58004ad0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 shutdown_connections 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f1e4006ac90 0x7f1e4006d150 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1e60160cc0 0x7f1e600ccf70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e600d3a70 0x7f1e601f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 --2- 172.21.15.99:0/320334055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e601608e0 0x7f1e60159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 >> 172.21.15.99:0/320334055 conn(0x7f1e6009c490 msgr2=0x7f1e600b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 shutdown_connections 2024-03-29T09:12:34.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.625+0000 7f1e4be00700 1 -- 172.21.15.99:0/320334055 wait complete. 2024-03-29T09:12:34.626 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:12:34.642 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-03-29T09:12:34.496818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222752,"kb_used_data":6580,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751968,"statfs":{"total":773094113280,"available":772866015232,"internally_reserved":0,"allocated":6737920,"data_stored":3654277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16532,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.751202"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-29T09:12:30.082749+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:12:30.082749+0000","last_peered":"2024-03-29T09:12:30.082749+0000","last_clean":"2024-03-29T09:12:30.082749+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:12:30.082749+0000","last_undegraded":"2024-03-29T09:12:30.082749+0000","last_fullsized":"2024-03-29T09:12:30.082749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.71399999999999997},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":1.639,"15min":1.639},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":0.86399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77600000000000002,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.71699999999999997,"15min":0.76400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72499999999999998,"5min":1.992,"15min":1.992},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.75900000000000001,"15min":0.81000000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.752},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.81899999999999995,"15min":0.86899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85099999999999998,"5min":2.883,"15min":2.883},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.87,"5min":1.387,"15min":1.387},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68500000000000005,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89200000000000002,"5min":1.367,"15min":1.367},"last":0.87}]},{"osd":5,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.458,"15min":1.458},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.88700000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.92200000000000004,"5min":1.413,"15min":1.413},"last":0.84699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":300296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.438,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82299999999999995,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.47799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":1.556}]},{"osd":1,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.53600000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75900000000000001,"15min":0.81200000000000006},"min":{"1min":0.49099999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.76}]},{"osd":2,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":1.8779999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.77600000000000002,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91000000000000003,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":2.1360000000000001}]},{"osd":4,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":2.581},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":3.3130000000000002}]},{"osd":5,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":2.8839999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":2.6680000000000001}]},{"osd":7,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78200000000000003,"15min":0.79800000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90100000000000002,"5min":1.026,"15min":1.026},"last":3.242},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94499999999999995,"5min":1.111,"15min":1.111},"last":2.508}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":896965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.55300000000000005,"15min":0.61599999999999999},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.755,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.52700000000000002,"15min":0.55100000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67200000000000004,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.59999999999999998,"15min":0.66600000000000004},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72199999999999998,"5min":1,"15min":1},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.96199999999999997,"15min":0.93400000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94799999999999995,"5min":12.356,"15min":12.356},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94999999999999996,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.018,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":0.92000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.034,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.80200000000000005,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91800000000000004,"5min":1.99,"15min":1.99},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":1.0109999999999999,"15min":0.97699999999999998},"min":{"1min":0.57799999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99199999999999999,"5min":12.669,"15min":12.669},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.97899999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84999999999999998,"15min":0.872},"min":{"1min":0.628,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97499999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.77800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27700,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":696320,"data_stored":316008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57699999999999996,"5min":0.751,"15min":0.751},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59899999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.61799999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.38400000000000001}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45300000000000001,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.54100000000000004,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.61499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.52100000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.022,"5min":1.093,"15min":1.093},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.147,"15min":1.147},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96699999999999997,"5min":1.069,"15min":1.069},"last":0.84399999999999997}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85099999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.755},"min":{"1min":0.58199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97899999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.45700000000000002,"15min":0.48199999999999998},"min":{"1min":0.25600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.59399999999999997,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.49099999999999999,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.252,"15min":1.252},"last":0.47299999999999998}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.51600000000000001,"15min":0.55500000000000005},"min":{"1min":0.31,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.53800000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.495},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58299999999999996,"15min":0.629},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.57799999999999996,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.56799999999999995,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55000000000000004,"5min":2.415,"15min":2.415},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.625,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.66600000000000004,"5min":2.056,"15min":2.056},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.77500000000000002,"15min":0.81699999999999995},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88900000000000001,"5min":2.073,"15min":2.073},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.58299999999999996,"5min":0.496,"15min":0.496},"max":{"1min":0.87,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80500000000000005},"min":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89600000000000002,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.92600000000000005}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.83599999999999997,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92000000000000004,"5min":2.387,"15min":2.387},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.873,"15min":0.93300000000000005},"min":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90200000000000002,"5min":2.996,"15min":2.996},"last":1.036}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.86099999999999999},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90700000000000003,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.81699999999999995,"15min":0.86299999999999999},"min":{"1min":0.53600000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.89000000000000001,"5min":2.165,"15min":2.165},"last":0.91100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.498,"15min":0.54000000000000004},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61499999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.502,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57499999999999996,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.57799999999999996,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55000000000000004,"15min":0.59299999999999997},"min":{"1min":0.311,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.65200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.439}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.58899999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93300000000000005,"5min":2.278,"15min":2.278},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.95299999999999996,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83099999999999996,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.754}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96599999999999997,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.83999999999999997,"15min":0.88700000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.014,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.84899999999999998,"15min":0.89600000000000002},"min":{"1min":0.503,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89000000000000001,"5min":2.649,"15min":2.649},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86199999999999999,"15min":0.90200000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.024,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.77800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27692,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":688128,"data_stored":306597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.83099999999999996,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.627,"5min":0.67700000000000005,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.80400000000000005},"min":{"1min":0.60899999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0089999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95499999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.751,"15min":0.80500000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80100000000000005,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.056,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98699999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93600000000000005,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"min":{"1min":0.623,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0289999999999999,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.88700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28272,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343568,"statfs":{"total":96636764160,"available":96607813632,"internally_reserved":0,"allocated":1282048,"data_stored":896965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.64700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.02,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.73299999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98499999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.495,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86099999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0780000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0329999999999999,"5min":1.05,"15min":1.05},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96899999999999997,"5min":1.079,"15min":1.079},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0580000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:12:34.644 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T09:12:34.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.833+0000 7f368ce00700 1 Processor -- start 2024-03-29T09:12:34.833 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.834+0000 7f368ce00700 1 -- start start 2024-03-29T09:12:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f368ce00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f368815da70 0x7f36880cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f368ce00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880d1740 0x7f36880d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f368ce00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36880afde0 con 0x7f368815d690 2024-03-29T09:12:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f368ce00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36880d2130 con 0x7f368815da70 2024-03-29T09:12:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f368ce00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36880d01d0 con 0x7f36880d1740 2024-03-29T09:12:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3685400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f368815da70 0x7f36880cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.836 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3685400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f368815da70 0x7f36880cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60002/0 (socket says 172.21.15.99:60002) 2024-03-29T09:12:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3685400700 1 -- 172.21.15.99:0/3334733501 learned_addr learned my addr 172.21.15.99:0/3334733501 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3686800700 1 --2- 172.21.15.99:0/3334733501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880d1740 0x7f36880d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3685400700 1 -- 172.21.15.99:0/3334733501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880d1740 msgr2=0x7f36880d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3685400700 1 --2- 172.21.15.99:0/3334733501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880d1740 0x7f36880d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.837 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3685400700 1 -- 172.21.15.99:0/3334733501 >> v1:172.21.15.133:6789/0 conn(0x7f368815d690 legacy=0x7f36880cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.835+0000 7f3685400700 1 -- 172.21.15.99:0/3334733501 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36880d04b0 con 0x7f368815da70 2024-03-29T09:12:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.836+0000 7f3685400700 1 --2- 172.21.15.99:0/3334733501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815da70 0x7f36880cfb60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f36740098f0 tx=0x7f3674009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440773b37a5ee11 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.836+0000 7f3684a00700 1 -- 172.21.15.99:0/3334733501 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011830 con 0x7f368815da70 2024-03-29T09:12:34.838 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.836+0000 7f3684a00700 1 -- 172.21.15.99:0/3334733501 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674011990 con 0x7f368815da70 2024-03-29T09:12:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.836+0000 7f3684a00700 1 -- 172.21.15.99:0/3334733501 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011d20 con 0x7f368815da70 2024-03-29T09:12:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.836+0000 7f368ce00700 1 -- 172.21.15.99:0/3334733501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815da70 msgr2=0x7f36880cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.836+0000 7f368ce00700 1 --2- 172.21.15.99:0/3334733501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815da70 0x7f36880cfb60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f36740098f0 tx=0x7f3674009c00 comp rx=0 tx=0).stop 2024-03-29T09:12:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.837+0000 7f368ce00700 1 -- 172.21.15.99:0/3334733501 shutdown_connections 2024-03-29T09:12:34.839 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.837+0000 7f368ce00700 1 --2- 172.21.15.99:0/3334733501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880d1740 0x7f36880d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.837+0000 7f368ce00700 1 --2- 172.21.15.99:0/3334733501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815da70 0x7f36880cfb60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.837+0000 7f368ce00700 1 -- 172.21.15.99:0/3334733501 >> 172.21.15.99:0/3334733501 conn(0x7f368809c480 msgr2=0x7f36880d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.837+0000 7f368ce00700 1 -- 172.21.15.99:0/3334733501 shutdown_connections 2024-03-29T09:12:34.840 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.837+0000 7f368ce00700 1 -- 172.21.15.99:0/3334733501 wait complete. 2024-03-29T09:12:34.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.838+0000 7f368ce00700 1 Processor -- start 2024-03-29T09:12:34.842 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.839+0000 7f368ce00700 1 -- start start 2024-03-29T09:12:34.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.839+0000 7f368ce00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36880d1740 0x7f368815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.839+0000 7f368ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815d690 0x7f36880cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f368ce00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f368815da70 0x7f36881f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.843 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f368ce00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368815fa60 con 0x7f368815d690 2024-03-29T09:12:34.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f368ce00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36880cce80 con 0x7f36880d1740 2024-03-29T09:12:34.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f368ce00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36880cc200 con 0x7f368815da70 2024-03-29T09:12:34.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815d690 0x7f36880cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.844 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815d690 0x7f36880cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60010/0 (socket says 172.21.15.99:60010) 2024-03-29T09:12:34.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 -- 172.21.15.99:0/265421090 learned_addr learned my addr 172.21.15.99:0/265421090 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:34.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685e00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36880d1740 0x7f368815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3686800700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f368815da70 0x7f36881f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.845 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 -- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f368815da70 msgr2=0x7f36881f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f368815da70 0x7f36881f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 -- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36880d1740 msgr2=0x7f368815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:34.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36880d1740 0x7f368815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:34.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 -- 172.21.15.99:0/265421090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674009550 con 0x7f368815d690 2024-03-29T09:12:34.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.840+0000 7f3685400700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815d690 0x7f36880cb890 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:34.846 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.841+0000 7f367e000700 1 -- 172.21.15.99:0/265421090 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674004530 con 0x7f368815d690 2024-03-29T09:12:34.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.841+0000 7f367e000700 1 -- 172.21.15.99:0/265421090 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367401f070 con 0x7f368815d690 2024-03-29T09:12:34.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.841+0000 7f367e000700 1 -- 172.21.15.99:0/265421090 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740038e0 con 0x7f368815d690 2024-03-29T09:12:34.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.841+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36880cc4e0 con 0x7f368815d690 2024-03-29T09:12:34.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.841+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36880ccaa0 con 0x7f368815d690 2024-03-29T09:12:34.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.842+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36880677a0 con 0x7f368815d690 2024-03-29T09:12:34.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.843+0000 7f367e000700 1 -- 172.21.15.99:0/265421090 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3674021030 con 0x7f368815d690 2024-03-29T09:12:34.847 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.844+0000 7f367e000700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f366806ac90 0x7f366806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:34.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.844+0000 7f367e000700 1 -- 172.21.15.99:0/265421090 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f367408a600 con 0x7f368815d690 2024-03-29T09:12:34.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.844+0000 7f3685e00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f366806ac90 0x7f366806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:34.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.845+0000 7f3685e00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f366806ac90 0x7f366806d150 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3678000f60 tx=0x7f3678009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:34.848 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:34.848+0000 7f367e000700 1 -- 172.21.15.99:0/265421090 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3674055990 con 0x7f368815d690 2024-03-29T09:12:35.051 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.053+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36880cd4f0 con 0x7f368815d690 2024-03-29T09:12:35.052 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.054+0000 7f367e000700 1 -- 172.21.15.99:0/265421090 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+12723 (secure 0 0 0) 0x7f3674058fb0 con 0x7f368815d690 2024-03-29T09:12:35.053 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:35.053 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":58,"fsid":"83e3f490-14b9-4a33-8e2d-bbf792b299cc","created":"2024-03-29T09:08:33.072821+0000","modified":"2024-03-29T09:12:31.085260+0000","last_up_change":"2024-03-29T09:08:45.867171+0000","last_in_change":"2024-03-29T09:08:36.744811+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T09:08:46.511608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f0f3c53f-e223-47cf-9087-542a9d6e72c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":42219},{"type":"v1","addr":"172.21.15.99:6825","nonce":42219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":42219},{"type":"v1","addr":"172.21.15.99:6827","nonce":42219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":42219},{"type":"v1","addr":"172.21.15.99:6831","nonce":42219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":42219},{"type":"v1","addr":"172.21.15.99:6829","nonce":42219}]},"public_addr":"172.21.15.99:6825/42219","cluster_addr":"172.21.15.99:6827/42219","heartbeat_back_addr":"172.21.15.99:6831/42219","heartbeat_front_addr":"172.21.15.99:6829/42219","state":["exists","up"]},{"osd":1,"uuid":"0e36e77e-2573-41e5-ae41-fe615e4076dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":42220},{"type":"v1","addr":"172.21.15.99:6809","nonce":42220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":42220},{"type":"v1","addr":"172.21.15.99:6811","nonce":42220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6815","nonce":42220},{"type":"v1","addr":"172.21.15.99:6817","nonce":42220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":42220},{"type":"v1","addr":"172.21.15.99:6813","nonce":42220}]},"public_addr":"172.21.15.99:6809/42220","cluster_addr":"172.21.15.99:6811/42220","heartbeat_back_addr":"172.21.15.99:6817/42220","heartbeat_front_addr":"172.21.15.99:6813/42220","state":["exists","up"]},{"osd":2,"uuid":"09245253-441b-44d5-ae4c-116226d0ecdf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":42218},{"type":"v1","addr":"172.21.15.99:6816","nonce":42218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":42218},{"type":"v1","addr":"172.21.15.99:6819","nonce":42218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":42218},{"type":"v1","addr":"172.21.15.99:6823","nonce":42218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":42218},{"type":"v1","addr":"172.21.15.99:6821","nonce":42218}]},"public_addr":"172.21.15.99:6816/42218","cluster_addr":"172.21.15.99:6819/42218","heartbeat_back_addr":"172.21.15.99:6823/42218","heartbeat_front_addr":"172.21.15.99:6821/42218","state":["exists","up"]},{"osd":3,"uuid":"3d441769-67e2-4cf9-8501-9a9dee172963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":42217},{"type":"v1","addr":"172.21.15.99:6801","nonce":42217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":42217},{"type":"v1","addr":"172.21.15.99:6803","nonce":42217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":42217},{"type":"v1","addr":"172.21.15.99:6807","nonce":42217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":42217},{"type":"v1","addr":"172.21.15.99:6805","nonce":42217}]},"public_addr":"172.21.15.99:6801/42217","cluster_addr":"172.21.15.99:6803/42217","heartbeat_back_addr":"172.21.15.99:6807/42217","heartbeat_front_addr":"172.21.15.99:6805/42217","state":["exists","up"]},{"osd":4,"uuid":"8f07961c-cadf-41ba-b375-d6f1a6ebfb1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":42489},{"type":"v1","addr":"172.21.15.133:6801","nonce":42489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":42489},{"type":"v1","addr":"172.21.15.133:6803","nonce":42489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":42489},{"type":"v1","addr":"172.21.15.133:6807","nonce":42489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":42489},{"type":"v1","addr":"172.21.15.133:6805","nonce":42489}]},"public_addr":"172.21.15.133:6801/42489","cluster_addr":"172.21.15.133:6803/42489","heartbeat_back_addr":"172.21.15.133:6807/42489","heartbeat_front_addr":"172.21.15.133:6805/42489","state":["exists","up"]},{"osd":5,"uuid":"73ad4bba-ab4f-481d-b7b4-5910fe9d2f82","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":42491},{"type":"v1","addr":"172.21.15.133:6817","nonce":42491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":42491},{"type":"v1","addr":"172.21.15.133:6819","nonce":42491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":42491},{"type":"v1","addr":"172.21.15.133:6823","nonce":42491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":42491},{"type":"v1","addr":"172.21.15.133:6821","nonce":42491}]},"public_addr":"172.21.15.133:6817/42491","cluster_addr":"172.21.15.133:6819/42491","heartbeat_back_addr":"172.21.15.133:6823/42491","heartbeat_front_addr":"172.21.15.133:6821/42491","state":["exists","up"]},{"osd":6,"uuid":"1eac9d41-854e-4bd1-9134-2c80a6c97bc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":42490},{"type":"v1","addr":"172.21.15.133:6809","nonce":42490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":42490},{"type":"v1","addr":"172.21.15.133:6811","nonce":42490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":42490},{"type":"v1","addr":"172.21.15.133:6815","nonce":42490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":42490},{"type":"v1","addr":"172.21.15.133:6813","nonce":42490}]},"public_addr":"172.21.15.133:6809/42490","cluster_addr":"172.21.15.133:6811/42490","heartbeat_back_addr":"172.21.15.133:6815/42490","heartbeat_front_addr":"172.21.15.133:6813/42490","state":["exists","up"]},{"osd":7,"uuid":"2fb07c4a-1ab7-454c-9a54-f127a9e53bff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":42500},{"type":"v1","addr":"172.21.15.133:6825","nonce":42500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":42500},{"type":"v1","addr":"172.21.15.133:6827","nonce":42500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":42500},{"type":"v1","addr":"172.21.15.133:6831","nonce":42500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":42500},{"type":"v1","addr":"172.21.15.133:6829","nonce":42500}]},"public_addr":"172.21.15.133:6825/42500","cluster_addr":"172.21.15.133:6827/42500","heartbeat_back_addr":"172.21.15.133:6831/42500","heartbeat_front_addr":"172.21.15.133:6829/42500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.104001+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.346985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.129660+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.279931+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.268198+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.321499+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.065042+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T09:08:42.112031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.99:6837/2396189132":"2024-03-30T09:12:27.743133+0000","172.21.15.99:6835/2396189132":"2024-03-30T09:12:27.743133+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T09:12:35.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f366806ac90 msgr2=0x7f366806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:35.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f366806ac90 0x7f366806d150 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3678000f60 tx=0x7f3678009040 comp rx=0 tx=0).stop 2024-03-29T09:12:35.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815d690 msgr2=0x7f36880cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:35.055 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815d690 0x7f36880cb890 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674004750 comp rx=0 tx=0).stop 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 shutdown_connections 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f366806ac90 0x7f366806d150 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f368815da70 0x7f36881f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36880d1740 0x7f368815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 --2- 172.21.15.99:0/265421090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f368815d690 0x7f36880cb890 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 >> 172.21.15.99:0/265421090 conn(0x7f368809c480 msgr2=0x7f368806be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 shutdown_connections 2024-03-29T09:12:35.056 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:35.056+0000 7f368ce00700 1 -- 172.21.15.99:0/265421090 wait complete. 2024-03-29T09:12:36.074 INFO:tasks.ceph:Scrubbing osd.0 2024-03-29T09:12:36.074 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.279+0000 7fc264600700 1 Processor -- start 2024-03-29T09:12:36.278 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.279+0000 7fc264600700 1 -- start start 2024-03-29T09:12:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.280+0000 7fc264600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc25c160b30 0x7fc25c0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc264600700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc25c0b1f50 0x7fc25c0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc264600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c0b2b90 0x7fc25c0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc264600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c0cf440 con 0x7fc25c160b30 2024-03-29T09:12:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc264600700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25c0d1fa0 con 0x7fc25c0b1f50 2024-03-29T09:12:36.281 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc264600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c160f10 con 0x7fc25c0b2b90 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc25c160b30 0x7fc25c0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fc25c160b30 0x7fc25c0cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60018/0 (socket says 172.21.15.99:60018) 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 -- 172.21.15.99:0/1732645029 learned_addr learned my addr 172.21.15.99:0/1732645029 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 -- 172.21.15.99:0/1732645029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c0b2b90 msgr2=0x7fc25c0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261200700 1 --2- 172.21.15.99:0/1732645029 >> v2:172.21.15.133:3300/0 conn(0x7fc25c0b1f50 0x7fc25c0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc262600700 1 --2- 172.21.15.99:0/1732645029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c0b2b90 0x7fc25c0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 --2- 172.21.15.99:0/1732645029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c0b2b90 0x7fc25c0b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.282 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 -- 172.21.15.99:0/1732645029 >> v2:172.21.15.133:3300/0 conn(0x7fc25c0b1f50 msgr2=0x7fc25c0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 --2- 172.21.15.99:0/1732645029 >> v2:172.21.15.133:3300/0 conn(0x7fc25c0b1f50 0x7fc25c0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261c00700 1 -- 172.21.15.99:0/1732645029 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0d1670 con 0x7fc25c160b30 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc261200700 1 --2- 172.21.15.99:0/1732645029 >> v2:172.21.15.133:3300/0 conn(0x7fc25c0b1f50 0x7fc25c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.281+0000 7fc262600700 1 --2- 172.21.15.99:0/1732645029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c0b2b90 0x7fc25c0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc261c00700 1 --2- 172.21.15.99:0/1732645029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c160b30 0x7fc25c0cfb50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0d42a0 tx=0x7fc250009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985096a0c4625552 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc25be00700 1 -- 172.21.15.99:0/1732645029 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000b870 con 0x7fc25c160b30 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc25be00700 1 -- 172.21.15.99:0/1732645029 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25000b9d0 con 0x7fc25c160b30 2024-03-29T09:12:36.283 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc25be00700 1 -- 172.21.15.99:0/1732645029 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25000bd60 con 0x7fc25c160b30 2024-03-29T09:12:36.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc264600700 1 -- 172.21.15.99:0/1732645029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c160b30 msgr2=0x7fc25c0cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc264600700 1 --2- 172.21.15.99:0/1732645029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c160b30 0x7fc25c0cfb50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0d42a0 tx=0x7fc250009bd0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.286 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc264600700 1 -- 172.21.15.99:0/1732645029 shutdown_connections 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc264600700 1 --2- 172.21.15.99:0/1732645029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c0b2b90 0x7fc25c0b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc264600700 1 --2- 172.21.15.99:0/1732645029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c160b30 0x7fc25c0cfb50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc264600700 1 --2- 172.21.15.99:0/1732645029 >> v2:172.21.15.133:3300/0 conn(0x7fc25c0b1f50 0x7fc25c0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.282+0000 7fc264600700 1 -- 172.21.15.99:0/1732645029 >> 172.21.15.99:0/1732645029 conn(0x7fc25c09c4d0 msgr2=0x7fc25c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.283+0000 7fc264600700 1 -- 172.21.15.99:0/1732645029 shutdown_connections 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.283+0000 7fc264600700 1 -- 172.21.15.99:0/1732645029 wait complete. 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.284+0000 7fc264600700 1 Processor -- start 2024-03-29T09:12:36.287 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.284+0000 7fc264600700 1 -- start start 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.285+0000 7fc264600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c0b1f50 0x7fc25c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.285+0000 7fc264600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc25c0b2b90 0x7fc25c1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.285+0000 7fc264600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c160b30 0x7fc25c1f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.285+0000 7fc264600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0b66c0 con 0x7fc25c0b1f50 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.285+0000 7fc264600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0b69a0 con 0x7fc25c0b2b90 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.285+0000 7fc264600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0b6c80 con 0x7fc25c160b30 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.285+0000 7fc261c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c0b1f50 0x7fc25c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.288 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c0b1f50 0x7fc25c0cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60030/0 (socket says 172.21.15.99:60030) 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 -- 172.21.15.99:0/3884139036 learned_addr learned my addr 172.21.15.99:0/3884139036 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc262600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c160b30 0x7fc25c1f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261200700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc25c0b2b90 0x7fc25c1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 -- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c160b30 msgr2=0x7fc25c1f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c160b30 0x7fc25c1f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 -- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc25c0b2b90 msgr2=0x7fc25c1f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc25c0b2b90 0x7fc25c1f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.289 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 -- 172.21.15.99:0/3884139036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc250009550 con 0x7fc25c0b1f50 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261200700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc25c0b2b90 0x7fc25c1f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc262600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc25c160b30 0x7fc25c1f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc261c00700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c0b1f50 0x7fc25c0cc6b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc250014040 tx=0x7fc250004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250004520 con 0x7fc25c0b1f50 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25001e070 con 0x7fc25c0b1f50 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.286+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500038e0 con 0x7fc25c0b1f50 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.287+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c0b6f60 con 0x7fc25c0b1f50 2024-03-29T09:12:36.290 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.287+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c0ccbf0 con 0x7fc25c0b1f50 2024-03-29T09:12:36.291 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.287+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc25c0b48e0 con 0x7fc25c0b1f50 2024-03-29T09:12:36.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.288+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fc250003a40 con 0x7fc25c0b1f50 2024-03-29T09:12:36.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.289+0000 7fc259600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc23c06ac90 0x7fc23c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.289+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fc250020070 con 0x7fc25c0b1f50 2024-03-29T09:12:36.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.290+0000 7fc261200700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc23c06ac90 0x7fc23c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.290+0000 7fc259600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7fc23c070f20 0x7fc23c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.290+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 --> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc23c073a50 con 0x7fc23c070f20 2024-03-29T09:12:36.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.290+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc25008cbf0 con 0x7fc25c0b1f50 2024-03-29T09:12:36.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.290+0000 7fc262600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7fc23c070f20 0x7fc23c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.290+0000 7fc261200700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc23c06ac90 0x7fc23c06d150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc25c160220 tx=0x7fc248008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.290+0000 7fc262600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7fc23c070f20 0x7fc23c073380 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.291+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== osd.0 v2:172.21.15.99:6824/42219 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc23c073a50 con 0x7fc23c070f20 2024-03-29T09:12:36.304 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 --> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc25c0d1c30 con 0x7fc23c070f20 2024-03-29T09:12:36.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc259600700 1 -- 172.21.15.99:0/3884139036 <== osd.0 v2:172.21.15.99:6824/42219 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc25400e5a0 con 0x7fc23c070f20 2024-03-29T09:12:36.305 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:36.305 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:36.305 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:36.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7fc23c070f20 msgr2=0x7fc23c073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc264600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6824/42219,v1:172.21.15.99:6825/42219] conn(0x7fc23c070f20 0x7fc23c073380 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc23c06ac90 msgr2=0x7fc23c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc264600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fc23c06ac90 0x7fc23c06d150 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc25c160220 tx=0x7fc248008040 comp rx=0 tx=0).stop 2024-03-29T09:12:36.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c0b1f50 msgr2=0x7fc25c0cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc264600700 1 --2- 172.21.15.99:0/3884139036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc25c0b1f50 0x7fc25c0cc6b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc250014040 tx=0x7fc250004750 comp rx=0 tx=0).stop 2024-03-29T09:12:36.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.306+0000 7fc262600700 1 -- 172.21.15.99:0/3884139036 reap_dead start 2024-03-29T09:12:36.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.307+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 shutdown_connections 2024-03-29T09:12:36.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.307+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 >> 172.21.15.99:0/3884139036 conn(0x7fc25c09c4d0 msgr2=0x7fc25c06b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:36.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.307+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 shutdown_connections 2024-03-29T09:12:36.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.307+0000 7fc264600700 1 -- 172.21.15.99:0/3884139036 wait complete. 2024-03-29T09:12:36.322 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-29T09:12:36.514 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.514+0000 7f09c4a00700 1 Processor -- start 2024-03-29T09:12:36.515 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.514+0000 7f09c4a00700 1 -- start start 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.515+0000 7f09c4a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 0x7f09c01615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.515+0000 7f09c4a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00d1be0 con 0x7f09c00cdcb0 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09c4a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00b4980 con 0x7f09c00d0150 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09c4a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00b46a0 con 0x7f09c00cfd70 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09be800700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f09c00cdcb0 0x7f09c00ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40458/0 (socket says 172.21.15.99:40458) 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09be800700 1 -- 172.21.15.99:0/4233498966 learned_addr learned my addr 172.21.15.99:0/4233498966 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bde00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 0x7f09c01615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575619992 0 0) 0x7f09c00d1be0 con 0x7f09c00cdcb0 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09a40035b0 con 0x7f09c00cdcb0 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789111049 0 0) 0x7f09c00b4980 con 0x7f09c00d0150 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c00d1be0 con 0x7f09c00d0150 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641553200 0 0) 0x7f09a40035b0 con 0x7f09c00cdcb0 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 msgr2=0x7f09c01615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 0x7f09c01615a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 >> v1:172.21.15.133:6789/0 conn(0x7f09c00d0150 legacy=0x7f09c0161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.520 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.516+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00d1590 con 0x7f09c00cdcb0 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f09b80019f0 con 0x7f09c00cdcb0 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bde00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 0x7f09c01615a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 >> v1:172.21.15.99:6789/0 conn(0x7f09c00cdcb0 legacy=0x7f09c00ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 0x7f09a4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09a4004150 0x7f09a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a4004d90 0x7f09a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00b46a0 con 0x7f09a4004d90 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a400bb20 con 0x7f09a4004150 2024-03-29T09:12:36.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a400bce0 con 0x7f09c00cfd70 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09be800700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a4004d90 0x7f09a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bd400700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09a4004150 0x7f09a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09bde00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 0x7f09a4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09be800700 1 -- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 msgr2=0x7f09a4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09be800700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c00cfd70 0x7f09a4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09be800700 1 -- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09a4004150 msgr2=0x7f09a4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09be800700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09a4004150 0x7f09a4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.517+0000 7f09be800700 1 -- 172.21.15.99:0/4233498966 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00094a0 con 0x7f09a4004d90 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09be800700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a4004d90 0x7f09a4005210 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f09b800a6e0 tx=0x7f09a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdd1008ed1d04e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80072d0 con 0x7f09a4004d90 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b8007430 con 0x7f09a4004d90 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09bca00700 1 -- 172.21.15.99:0/4233498966 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80077c0 con 0x7f09a4004d90 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09c4a00700 1 -- 172.21.15.99:0/4233498966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a4004d90 msgr2=0x7f09a4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09c4a00700 1 --2- 172.21.15.99:0/4233498966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09a4004d90 0x7f09a4005210 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f09b800a6e0 tx=0x7f09a4005880 comp rx=0 tx=0).stop 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09be800700 1 -- 172.21.15.99:0/4233498966 reap_dead start 2024-03-29T09:12:36.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09c4a00700 1 -- 172.21.15.99:0/4233498966 shutdown_connections 2024-03-29T09:12:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09c4a00700 1 -- 172.21.15.99:0/4233498966 >> 172.21.15.99:0/4233498966 conn(0x7f09c009c4a0 msgr2=0x7f09c00d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09c4a00700 1 -- 172.21.15.99:0/4233498966 shutdown_connections 2024-03-29T09:12:36.524 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.518+0000 7f09c4a00700 1 -- 172.21.15.99:0/4233498966 wait complete. 2024-03-29T09:12:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.521+0000 7f09c4a00700 1 Processor -- start 2024-03-29T09:12:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.521+0000 7f09c4a00700 1 -- start start 2024-03-29T09:12:36.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09c4a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c014ff40 0x7f09c0151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09c4a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c00cb900 0x7f09c006b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09c4a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09c006b9d0 0x7f09c00ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09c4a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00cd110 con 0x7f09c006b9d0 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09c4a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0150320 con 0x7f09c00cb900 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09c4a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c006be50 con 0x7f09c014ff40 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09be800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09c006b9d0 0x7f09c00ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09be800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09c006b9d0 0x7f09c00ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60054/0 (socket says 172.21.15.99:60054) 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.522+0000 7f09be800700 1 -- 172.21.15.99:0/417360727 learned_addr learned my addr 172.21.15.99:0/417360727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:36.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09bd400700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c00cb900 0x7f09c006b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09bde00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c014ff40 0x7f09c0151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09be800700 1 -- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c014ff40 msgr2=0x7f09c0151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09be800700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c014ff40 0x7f09c0151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09be800700 1 -- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c00cb900 msgr2=0x7f09c006b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09be800700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c00cb900 0x7f09c006b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09be800700 1 -- 172.21.15.99:0/417360727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0151850 con 0x7f09c006b9d0 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09be800700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09c006b9d0 0x7f09c00ca220 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f09a400baf0 tx=0x7f09b8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09b6000700 1 -- 172.21.15.99:0/417360727 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80079b0 con 0x7f09c006b9d0 2024-03-29T09:12:36.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09b6000700 1 -- 172.21.15.99:0/417360727 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b8007b10 con 0x7f09c006b9d0 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09b6000700 1 -- 172.21.15.99:0/417360727 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8012600 con 0x7f09c006b9d0 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.523+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c00cc150 con 0x7f09c006b9d0 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.524+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c01e63e0 con 0x7f09c006b9d0 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.524+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c01e6240 con 0x7f09c006b9d0 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.525+0000 7f09b6000700 1 -- 172.21.15.99:0/417360727 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f09b801a070 con 0x7f09c006b9d0 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.526+0000 7f09b6000700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09a006acc0 0x7f09a006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.526+0000 7f09b6000700 1 -- 172.21.15.99:0/417360727 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f09b808a8a0 con 0x7f09c006b9d0 2024-03-29T09:12:36.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.526+0000 7f09bde00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09a006acc0 0x7f09a006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.527+0000 7f09bde00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09a006acc0 0x7f09a006d180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009d30 tx=0x7f09b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.530+0000 7f09b6000700 1 -- 172.21.15.99:0/417360727 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f09c01e6240 con 0x7f09c006b9d0 2024-03-29T09:12:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.734+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f09c00b3960 con 0x7f09a006acc0 2024-03-29T09:12:36.735 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.735+0000 7f09b6000700 1 -- 172.21.15.99:0/417360727 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f09b0009e70 con 0x7f09a006acc0 2024-03-29T09:12:36.737 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09a006acc0 msgr2=0x7f09a006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09a006acc0 0x7f09a006d180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009d30 tx=0x7f09b0008040 comp rx=0 tx=0).stop 2024-03-29T09:12:36.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09c006b9d0 msgr2=0x7f09c00ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09c006b9d0 0x7f09c00ca220 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f09a400baf0 tx=0x7f09b8001cb0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 shutdown_connections 2024-03-29T09:12:36.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f09a006acc0 0x7f09a006d180 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.738 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f09c006b9d0 0x7f09c00ca220 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09c014ff40 0x7f09c0151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 --2- 172.21.15.99:0/417360727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c00cb900 0x7f09c006b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 >> 172.21.15.99:0/417360727 conn(0x7f09c009c4a0 msgr2=0x7f09c00aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 shutdown_connections 2024-03-29T09:12:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.737+0000 7f09c4a00700 1 -- 172.21.15.99:0/417360727 wait complete. 2024-03-29T09:12:36.739 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 0 to deep-scrub 2024-03-29T09:12:36.755 INFO:tasks.ceph:Scrubbing osd.1 2024-03-29T09:12:36.756 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:36.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.949+0000 7ff7b2400700 1 Processor -- start 2024-03-29T09:12:36.949 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.949+0000 7ff7b2400700 1 -- start start 2024-03-29T09:12:36.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.950+0000 7ff7b2400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff7ac160b30 0x7ff7ac0cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.951 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.950+0000 7ff7b2400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac0b1f50 0x7ff7ac0d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7b2400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff7ac0b2b90 0x7ff7ac0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7b2400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0cf440 con 0x7ff7ac0b2b90 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7b2400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7ac0d1fa0 con 0x7ff7ac160b30 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7b2400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac160f10 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff7ac0b2b90 0x7ff7ac0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7ff7ac0b2b90 0x7ff7ac0b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60068/0 (socket says 172.21.15.99:60068) 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 -- 172.21.15.99:0/2049420488 learned_addr learned my addr 172.21.15.99:0/2049420488 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:36.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7ab400700 1 --2- 172.21.15.99:0/2049420488 >> v2:172.21.15.133:3300/0 conn(0x7ff7ac160b30 0x7ff7ac0cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7aaa00700 1 --2- 172.21.15.99:0/2049420488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac0b1f50 0x7ff7ac0d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 -- 172.21.15.99:0/2049420488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac0b1f50 msgr2=0x7ff7ac0d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 --2- 172.21.15.99:0/2049420488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac0b1f50 0x7ff7ac0d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 -- 172.21.15.99:0/2049420488 >> v2:172.21.15.133:3300/0 conn(0x7ff7ac160b30 msgr2=0x7ff7ac0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 --2- 172.21.15.99:0/2049420488 >> v2:172.21.15.133:3300/0 conn(0x7ff7ac160b30 0x7ff7ac0cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.951+0000 7ff7abe00700 1 -- 172.21.15.99:0/2049420488 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0d1670 con 0x7ff7ac0b2b90 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7abe00700 1 --2- 172.21.15.99:0/2049420488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b2b90 0x7ff7ac0b3020 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac09cc00 tx=0x7ff7a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7637fcf452a1fdaf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7aa000700 1 -- 172.21.15.99:0/2049420488 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400e070 con 0x7ff7ac0b2b90 2024-03-29T09:12:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7aa000700 1 -- 172.21.15.99:0/2049420488 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a400b910 con 0x7ff7ac0b2b90 2024-03-29T09:12:36.954 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7aa000700 1 -- 172.21.15.99:0/2049420488 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400bcc0 con 0x7ff7ac0b2b90 2024-03-29T09:12:36.956 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 -- 172.21.15.99:0/2049420488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b2b90 msgr2=0x7ff7ac0b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 --2- 172.21.15.99:0/2049420488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b2b90 0x7ff7ac0b3020 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac09cc00 tx=0x7ff7a400b330 comp rx=0 tx=0).stop 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 -- 172.21.15.99:0/2049420488 shutdown_connections 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 --2- 172.21.15.99:0/2049420488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b2b90 0x7ff7ac0b3020 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 --2- 172.21.15.99:0/2049420488 >> v2:172.21.15.133:3300/0 conn(0x7ff7ac160b30 0x7ff7ac0cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 --2- 172.21.15.99:0/2049420488 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac0b1f50 0x7ff7ac0d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 -- 172.21.15.99:0/2049420488 >> 172.21.15.99:0/2049420488 conn(0x7ff7ac09c4d0 msgr2=0x7ff7ac0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.952+0000 7ff7b2400700 1 -- 172.21.15.99:0/2049420488 shutdown_connections 2024-03-29T09:12:36.957 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.953+0000 7ff7b2400700 1 -- 172.21.15.99:0/2049420488 wait complete. 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.954+0000 7ff7b2400700 1 Processor -- start 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.954+0000 7ff7b2400700 1 -- start start 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.955+0000 7ff7b2400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b1f50 0x7ff7ac15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.955+0000 7ff7b2400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0b2b90 0x7ff7ac09fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.955+0000 7ff7b2400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac160b30 0x7ff7ac152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.955+0000 7ff7b2400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac1f5fc0 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.955+0000 7ff7b2400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac1f62a0 con 0x7ff7ac0b2b90 2024-03-29T09:12:36.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.955+0000 7ff7b2400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac1f6580 con 0x7ff7ac160b30 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.955+0000 7ff7ab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b1f50 0x7ff7ac15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b1f50 0x7ff7ac15fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60072/0 (socket says 172.21.15.99:60072) 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 -- 172.21.15.99:0/3371265265 learned_addr learned my addr 172.21.15.99:0/3371265265 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7aaa00700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0b2b90 0x7ff7ac09fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7abe00700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac160b30 0x7ff7ac152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 -- 172.21.15.99:0/3371265265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac160b30 msgr2=0x7ff7ac152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff7ac160b30 0x7ff7ac152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 -- 172.21.15.99:0/3371265265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0b2b90 msgr2=0x7ff7ac09fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0b2b90 0x7ff7ac09fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 -- 172.21.15.99:0/3371265265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4009b30 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7aaa00700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7ac0b2b90 0x7ff7ac09fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7ab400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b1f50 0x7ff7ac15fb10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e870 tx=0x7ff79c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c009590 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79c013680 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.956+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac1f6860 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.957+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0139f0 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.960 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.957+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac09f990 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.957+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7ac0b48e0 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.958+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7ff79c013b50 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.959+0000 7ff7a3400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff78c06ac90 0x7ff78c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.959+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7ff79c089000 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.960+0000 7ff7aaa00700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff78c06ac90 0x7ff78c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.960+0000 7ff7a3400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7ff78c070f20 0x7ff78c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:36.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.960+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 --> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff78c073a50 con 0x7ff78c070f20 2024-03-29T09:12:36.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.960+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff79c08b6c0 con 0x7ff7ac0b1f50 2024-03-29T09:12:36.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.960+0000 7ff7abe00700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7ff78c070f20 0x7ff78c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:36.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.960+0000 7ff7aaa00700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff78c06ac90 0x7ff78c06d150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac068210 tx=0x7ff798008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.960+0000 7ff7abe00700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7ff78c070f20 0x7ff78c073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:36.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.961+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== osd.1 v2:172.21.15.99:6808/42220 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff78c073a50 con 0x7ff78c070f20 2024-03-29T09:12:36.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.976+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 --> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff7ac0d1c30 con 0x7ff78c070f20 2024-03-29T09:12:36.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.976+0000 7ff7a3400700 1 -- 172.21.15.99:0/3371265265 <== osd.1 v2:172.21.15.99:6808/42220 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff7a40099b0 con 0x7ff78c070f20 2024-03-29T09:12:36.976 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:36.977 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:36.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.976+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7ff78c070f20 msgr2=0x7ff78c073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.976+0000 7ff7b2400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6808/42220,v1:172.21.15.99:6809/42220] conn(0x7ff78c070f20 0x7ff78c073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:36.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff78c06ac90 msgr2=0x7ff78c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7ff78c06ac90 0x7ff78c06d150 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac068210 tx=0x7ff798008040 comp rx=0 tx=0).stop 2024-03-29T09:12:36.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b1f50 msgr2=0x7ff7ac15fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:36.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 --2- 172.21.15.99:0/3371265265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ac0b1f50 0x7ff7ac15fb10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00e870 tx=0x7ff79c00eb80 comp rx=0 tx=0).stop 2024-03-29T09:12:36.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7abe00700 1 -- 172.21.15.99:0/3371265265 reap_dead start 2024-03-29T09:12:36.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 shutdown_connections 2024-03-29T09:12:36.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 >> 172.21.15.99:0/3371265265 conn(0x7ff7ac09c4d0 msgr2=0x7ff7ac0afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:36.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 shutdown_connections 2024-03-29T09:12:36.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:36.977+0000 7ff7b2400700 1 -- 172.21.15.99:0/3371265265 wait complete. 2024-03-29T09:12:36.994 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-29T09:12:37.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.186+0000 7f9743400700 1 Processor -- start 2024-03-29T09:12:37.186 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.186+0000 7f9743400700 1 -- start start 2024-03-29T09:12:37.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9743400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f973c15d690 0x7f973c0cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9743400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15da70 0x7f973c0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9743400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0afde0 con 0x7f973c0d1740 2024-03-29T09:12:37.188 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9743400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0cf660 con 0x7f973c15d690 2024-03-29T09:12:37.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9743400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0d01d0 con 0x7f973c15da70 2024-03-29T09:12:37.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f973be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15da70 0x7f973c0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9740a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f973c15d690 0x7f973c0cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9741400700 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f973c0d1740 0x7f973c0cde10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:38714/0 (socket says 172.21.15.99:38714) 2024-03-29T09:12:37.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f9741400700 1 -- 172.21.15.99:0/3992994256 learned_addr learned my addr 172.21.15.99:0/3992994256 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:37.189 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.188+0000 7f973be00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15da70 0x7f973c0cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:57744/0 (socket says 172.21.15.99:57744) 2024-03-29T09:12:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973be00700 1 -- 172.21.15.99:0/3992994256 >> v2:172.21.15.133:3300/0 conn(0x7f973c15d690 msgr2=0x7f973c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973be00700 1 --2- 172.21.15.99:0/3992994256 >> v2:172.21.15.133:3300/0 conn(0x7f973c15d690 0x7f973c0cfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973b400700 1 -- 172.21.15.99:0/3992994256 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965194880 0 0) 0x7f973c0afde0 con 0x7f973c0d1740 2024-03-29T09:12:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973be00700 1 -- 172.21.15.99:0/3992994256 >> v1:172.21.15.133:6789/0 conn(0x7f973c0d1740 legacy=0x7f973c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973be00700 1 -- 172.21.15.99:0/3992994256 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0d04b0 con 0x7f973c15da70 2024-03-29T09:12:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973be00700 1 --2- 172.21.15.99:0/3992994256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15da70 0x7f973c0cf120 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9728009840 tx=0x7f9728009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0cecd5aeefbe53 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.191 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973b400700 1 -- 172.21.15.99:0/3992994256 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728011830 con 0x7f973c15da70 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973b400700 1 -- 172.21.15.99:0/3992994256 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9728011990 con 0x7f973c15da70 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.189+0000 7f973b400700 1 -- 172.21.15.99:0/3992994256 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728011d20 con 0x7f973c15da70 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 -- 172.21.15.99:0/3992994256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15da70 msgr2=0x7f973c0cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 --2- 172.21.15.99:0/3992994256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15da70 0x7f973c0cf120 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9728009840 tx=0x7f9728009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 -- 172.21.15.99:0/3992994256 shutdown_connections 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 --2- 172.21.15.99:0/3992994256 >> v2:172.21.15.133:3300/0 conn(0x7f973c15d690 0x7f973c0cfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 --2- 172.21.15.99:0/3992994256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15da70 0x7f973c0cf120 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.192 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 -- 172.21.15.99:0/3992994256 >> 172.21.15.99:0/3992994256 conn(0x7f973c09c480 msgr2=0x7f973c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:37.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 -- 172.21.15.99:0/3992994256 shutdown_connections 2024-03-29T09:12:37.193 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.190+0000 7f9743400700 1 -- 172.21.15.99:0/3992994256 wait complete. 2024-03-29T09:12:37.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.192+0000 7f9743400700 1 Processor -- start 2024-03-29T09:12:37.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.192+0000 7f9743400700 1 -- start start 2024-03-29T09:12:37.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9743400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f973c0d1740 0x7f973c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9743400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15d690 0x7f973c1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9743400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973c15da70 0x7f973c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.195 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9743400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c150f70 con 0x7f973c15da70 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9743400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1600f0 con 0x7f973c0d1740 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9743400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0cc5f0 con 0x7f973c15d690 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973c15da70 0x7f973c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973c15da70 0x7f973c0b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60076/0 (socket says 172.21.15.99:60076) 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 -- 172.21.15.99:0/2127845452 learned_addr learned my addr 172.21.15.99:0/2127845452 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f973be00700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15d690 0x7f973c1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9740a00700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f973c0d1740 0x7f973c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 -- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15d690 msgr2=0x7f973c1f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.196 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15d690 0x7f973c1f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 -- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f973c0d1740 msgr2=0x7f973c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f973c0d1740 0x7f973c06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.193+0000 7f9741400700 1 -- 172.21.15.99:0/2127845452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97280094a0 con 0x7f973c15da70 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.194+0000 7f9741400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973c15da70 0x7f973c0b4e50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f973400cba0 tx=0x7f973400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.194+0000 7f9738c00700 1 -- 172.21.15.99:0/2127845452 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400aaa0 con 0x7f973c15da70 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.194+0000 7f9738c00700 1 -- 172.21.15.99:0/2127845452 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400ac20 con 0x7f973c15da70 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.194+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c0cc8d0 con 0x7f973c15da70 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.194+0000 7f9738c00700 1 -- 172.21.15.99:0/2127845452 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340070d0 con 0x7f973c15da70 2024-03-29T09:12:37.197 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.194+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0cce90 con 0x7f973c15da70 2024-03-29T09:12:37.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.195+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c160280 con 0x7f973c15da70 2024-03-29T09:12:37.198 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.196+0000 7f9738c00700 1 -- 172.21.15.99:0/2127845452 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9734007230 con 0x7f973c15da70 2024-03-29T09:12:37.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.197+0000 7f9738c00700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f971c06ac90 0x7f971c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.197+0000 7f9738c00700 1 -- 172.21.15.99:0/2127845452 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9734088c10 con 0x7f973c15da70 2024-03-29T09:12:37.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.197+0000 7f9740a00700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f971c06ac90 0x7f971c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.199 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.198+0000 7f9740a00700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f971c06ac90 0x7f971c06d150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9730009b40 tx=0x7f9730009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.200 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.201+0000 7f9738c00700 1 -- 172.21.15.99:0/2127845452 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f9734053fa0 con 0x7f973c15da70 2024-03-29T09:12:37.403 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.403+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f973c0b0310 con 0x7f971c06ac90 2024-03-29T09:12:37.404 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.403+0000 7f9738c00700 1 -- 172.21.15.99:0/2127845452 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9730004020 con 0x7f971c06ac90 2024-03-29T09:12:37.406 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f971c06ac90 msgr2=0x7f971c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f971c06ac90 0x7f971c06d150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9730009b40 tx=0x7f9730009210 comp rx=0 tx=0).stop 2024-03-29T09:12:37.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973c15da70 msgr2=0x7f973c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973c15da70 0x7f973c0b4e50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f973400cba0 tx=0x7f973400ceb0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 shutdown_connections 2024-03-29T09:12:37.407 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f971c06ac90 0x7f971c06d150 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f973c15da70 0x7f973c0b4e50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f973c0d1740 0x7f973c06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 --2- 172.21.15.99:0/2127845452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f973c15d690 0x7f973c1f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 >> 172.21.15.99:0/2127845452 conn(0x7f973c09c480 msgr2=0x7f973c15b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:37.408 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 shutdown_connections 2024-03-29T09:12:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.406+0000 7f9743400700 1 -- 172.21.15.99:0/2127845452 wait complete. 2024-03-29T09:12:37.409 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 1 to deep-scrub 2024-03-29T09:12:37.424 INFO:tasks.ceph:Scrubbing osd.2 2024-03-29T09:12:37.425 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:37.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.618+0000 7fcf06000700 1 Processor -- start 2024-03-29T09:12:37.618 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.618+0000 7fcf06000700 1 -- start start 2024-03-29T09:12:37.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.619+0000 7fcf06000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf000cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.619+0000 7fcf06000700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000d1fa0 con 0x7fcf000d0090 2024-03-29T09:12:37.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.619+0000 7fcf06000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf00160f10 con 0x7fcf000b1f50 2024-03-29T09:12:37.622 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.619+0000 7fcf06000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000cf440 con 0x7fcf00160b30 2024-03-29T09:12:37.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.619+0000 7fceffe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fcf000d0090 0x7fcf000b32a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:40474/0 (socket says 172.21.15.99:40474) 2024-03-29T09:12:37.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.619+0000 7fceffe00700 1 -- 172.21.15.99:0/1647436435 learned_addr learned my addr 172.21.15.99:0/1647436435 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:37.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.619+0000 7fceff400700 1 --2- 172.21.15.99:0/1647436435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf000cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fcefe000700 1 -- 172.21.15.99:0/1647436435 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630447898 0 0) 0x7fcf000d1fa0 con 0x7fcf000d0090 2024-03-29T09:12:37.623 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fcefe000700 1 -- 172.21.15.99:0/1647436435 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee40035b0 con 0x7fcf000d0090 2024-03-29T09:12:37.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fcefe000700 1 -- 172.21.15.99:0/1647436435 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564551477 0 0) 0x7fcf00160f10 con 0x7fcf000b1f50 2024-03-29T09:12:37.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fcefe000700 1 -- 172.21.15.99:0/1647436435 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf000d1fa0 con 0x7fcf000b1f50 2024-03-29T09:12:37.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fceff400700 1 -- 172.21.15.99:0/1647436435 >> v1:172.21.15.133:6789/0 conn(0x7fcf000b1f50 legacy=0x7fcf000b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fceff400700 1 -- 172.21.15.99:0/1647436435 >> v1:172.21.15.99:6789/0 conn(0x7fcf000d0090 legacy=0x7fcf000b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fceff400700 1 -- 172.21.15.99:0/1647436435 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000d1670 con 0x7fcf00160b30 2024-03-29T09:12:37.624 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fceff400700 1 --2- 172.21.15.99:0/1647436435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf000cfb50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009870 tx=0x7fcef0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143550849f9d6a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fcefe000700 1 -- 172.21.15.99:0/1647436435 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0011830 con 0x7fcf00160b30 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fcefe000700 1 -- 172.21.15.99:0/1647436435 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0011990 con 0x7fcf00160b30 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.620+0000 7fcefe000700 1 -- 172.21.15.99:0/1647436435 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0011d20 con 0x7fcf00160b30 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.621+0000 7fcf06000700 1 -- 172.21.15.99:0/1647436435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 msgr2=0x7fcf000cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.621+0000 7fcf06000700 1 --2- 172.21.15.99:0/1647436435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf000cfb50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009870 tx=0x7fcef0009b80 comp rx=0 tx=0).stop 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.621+0000 7fcf06000700 1 -- 172.21.15.99:0/1647436435 shutdown_connections 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.621+0000 7fcf06000700 1 --2- 172.21.15.99:0/1647436435 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf000cfb50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.621+0000 7fcf06000700 1 -- 172.21.15.99:0/1647436435 >> 172.21.15.99:0/1647436435 conn(0x7fcf0009c4d0 msgr2=0x7fcf000d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:37.625 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.621+0000 7fcf06000700 1 -- 172.21.15.99:0/1647436435 shutdown_connections 2024-03-29T09:12:37.626 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.621+0000 7fcf06000700 1 -- 172.21.15.99:0/1647436435 wait complete. 2024-03-29T09:12:37.628 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.623+0000 7fcf06000700 1 Processor -- start 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.623+0000 7fcf06000700 1 -- start start 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcf06000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf000b1f50 0x7fcf000ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcf06000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf000d0090 0x7fcf000b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcf06000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf0006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcf06000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b6e10 con 0x7fcf000d0090 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcf06000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b6190 con 0x7fcf000b1f50 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcf06000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b6470 con 0x7fcf00160b30 2024-03-29T09:12:37.629 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcefea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf000d0090 0x7fcf000b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcefea00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf000d0090 0x7fcf000b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:60080/0 (socket says 172.21.15.99:60080) 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcefea00700 1 -- 172.21.15.99:0/1997013952 learned_addr learned my addr 172.21.15.99:0/1997013952 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fceff400700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf000b1f50 0x7fcf000ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fceffe00700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf0006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcefea00700 1 -- 172.21.15.99:0/1997013952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 msgr2=0x7fcf0006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.624+0000 7fcefea00700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcf00160b30 0x7fcf0006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcefea00700 1 -- 172.21.15.99:0/1997013952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf000b1f50 msgr2=0x7fcf000ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.630 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcefea00700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf000b1f50 0x7fcf000ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcefea00700 1 -- 172.21.15.99:0/1997013952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00094a0 con 0x7fcf000d0090 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcefea00700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf000d0090 0x7fcf000b5a80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fceec00c520 tx=0x7fceec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0052d0 con 0x7fcf000d0090 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec005430 con 0x7fcf000d0090 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0038c0 con 0x7fcf000d0090 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf000b6750 con 0x7fcf000d0090 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.625+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0006bcf0 con 0x7fcf000d0090 2024-03-29T09:12:37.631 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.626+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf000b48e0 con 0x7fcf000d0090 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.627+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fceec003a20 con 0x7fcf000d0090 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.628+0000 7fcef7400700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcee006ac90 0x7fcee006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.628+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fceec0885d0 con 0x7fcf000d0090 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.628+0000 7fceff400700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcee006ac90 0x7fcee006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.628+0000 7fcef7400700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fcee0070f20 0x7fcee0073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.628+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 --> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcee0073a50 con 0x7fcee0070f20 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.628+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceec08abb0 con 0x7fcf000d0090 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.628+0000 7fceffe00700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fcee0070f20 0x7fcee0073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.629+0000 7fceff400700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcee006ac90 0x7fcee006d150 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcef00090b0 tx=0x7fcef001c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.629+0000 7fceffe00700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fcee0070f20 0x7fcee0073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.630+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== osd.2 v2:172.21.15.99:6814/42218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fcee0073a50 con 0x7fcee0070f20 2024-03-29T09:12:37.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.644+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 --> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcf000d1c30 con 0x7fcee0070f20 2024-03-29T09:12:37.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcef7400700 1 -- 172.21.15.99:0/1997013952 <== osd.2 v2:172.21.15.99:6814/42218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcef8001fa0 con 0x7fcee0070f20 2024-03-29T09:12:37.645 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:37.646 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:37.646 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fcee0070f20 msgr2=0x7fcee0073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6814/42218,v1:172.21.15.99:6816/42218] conn(0x7fcee0070f20 0x7fcee0073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcee006ac90 msgr2=0x7fcee006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fcee006ac90 0x7fcee006d150 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcef00090b0 tx=0x7fcef001c040 comp rx=0 tx=0).stop 2024-03-29T09:12:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf000d0090 msgr2=0x7fcf000b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 --2- 172.21.15.99:0/1997013952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf000d0090 0x7fcf000b5a80 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fceec00c520 tx=0x7fceec00c830 comp rx=0 tx=0).stop 2024-03-29T09:12:37.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fceffe00700 1 -- 172.21.15.99:0/1997013952 reap_dead start 2024-03-29T09:12:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 shutdown_connections 2024-03-29T09:12:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 >> 172.21.15.99:0/1997013952 conn(0x7fcf0009c4d0 msgr2=0x7fcf000b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 shutdown_connections 2024-03-29T09:12:37.650 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.645+0000 7fcf06000700 1 -- 172.21.15.99:0/1997013952 wait complete. 2024-03-29T09:12:37.662 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-29T09:12:37.852 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.852+0000 7f721c400700 1 Processor -- start 2024-03-29T09:12:37.853 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.853+0000 7f721c400700 1 -- start start 2024-03-29T09:12:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.854+0000 7f721c400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 0x7f7214161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.854+0000 7f721c400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140d1be0 con 0x7f72140cfd70 2024-03-29T09:12:37.856 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.854+0000 7f721c400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140b4980 con 0x7f72140cdcb0 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.854+0000 7f721c400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72140b46a0 con 0x7f72140d0150 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.854+0000 7f7219a00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f72140cfd70 0x7f72141615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42496/0 (socket says 172.21.15.99:42496) 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.854+0000 7f7219a00700 1 -- 172.21.15.99:0/1888219872 learned_addr learned my addr 172.21.15.99:0/1888219872 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992798774 0 0) 0x7f72140d1be0 con 0x7f72140cfd70 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7219000700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 0x7f7214161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71fc0035b0 con 0x7f72140cfd70 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4166197284 0 0) 0x7f71fc0035b0 con 0x7f72140cfd70 2024-03-29T09:12:37.857 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 msgr2=0x7f7214161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 0x7f7214161cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 >> v1:172.21.15.133:6789/0 conn(0x7f72140cdcb0 legacy=0x7f72140ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140d1590 con 0x7f72140cfd70 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f7208001670 con 0x7f72140cfd70 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 >> v1:172.21.15.99:6789/0 conn(0x7f72140cfd70 legacy=0x7f72141615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71fc003a30 0x7f71fc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 0x7f71fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.858 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f71fc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140b46a0 con 0x7f71fc004d90 2024-03-29T09:12:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc00baf0 con 0x7f71fc003a30 2024-03-29T09:12:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.855+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc00bcb0 con 0x7f72140d0150 2024-03-29T09:12:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f721a400700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f71fc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f7219000700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 0x7f71fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f7219a00700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71fc003a30 0x7f71fc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.859 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f721a400700 1 -- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 msgr2=0x7f71fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f721a400700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140d0150 0x7f71fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f721a400700 1 -- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71fc003a30 msgr2=0x7f71fc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f721a400700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71fc003a30 0x7f71fc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f721a400700 1 -- 172.21.15.99:0/1888219872 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0035b0 con 0x7f71fc004d90 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f721a400700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f71fc005210 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c160 tx=0x7f71fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a16f75a6b377e22 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.856+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c001db0 con 0x7f71fc004d90 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c00cca0 con 0x7f71fc004d90 2024-03-29T09:12:37.860 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f7213e00700 1 -- 172.21.15.99:0/1888219872 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c003c40 con 0x7f71fc004d90 2024-03-29T09:12:37.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f721c400700 1 -- 172.21.15.99:0/1888219872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 msgr2=0x7f71fc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f721c400700 1 --2- 172.21.15.99:0/1888219872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f71fc005210 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c160 tx=0x7f71fc005880 comp rx=0 tx=0).stop 2024-03-29T09:12:37.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f721a400700 1 -- 172.21.15.99:0/1888219872 reap_dead start 2024-03-29T09:12:37.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f721c400700 1 -- 172.21.15.99:0/1888219872 shutdown_connections 2024-03-29T09:12:37.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f721c400700 1 -- 172.21.15.99:0/1888219872 >> 172.21.15.99:0/1888219872 conn(0x7f721409c4a0 msgr2=0x7f72140d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:37.864 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f721c400700 1 -- 172.21.15.99:0/1888219872 shutdown_connections 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.857+0000 7f721c400700 1 -- 172.21.15.99:0/1888219872 wait complete. 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.859+0000 7f721c400700 1 Processor -- start 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.859+0000 7f721c400700 1 -- start start 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721c400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f72140b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721c400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140c9800 0x7f72140cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721c400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72140cb8e0 0x7f72140cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721c400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140c7060 con 0x7f71fc004d90 2024-03-29T09:12:37.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721c400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140b8dc0 con 0x7f72140cb8e0 2024-03-29T09:12:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721c400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72140c72b0 con 0x7f72140c9800 2024-03-29T09:12:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f72140b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f72140b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44910/0 (socket says 172.21.15.99:44910) 2024-03-29T09:12:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 -- 172.21.15.99:0/957704382 learned_addr learned my addr 172.21.15.99:0/957704382 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219000700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140c9800 0x7f72140cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.866 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721a400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72140cb8e0 0x7f72140cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 -- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140c9800 msgr2=0x7f72140cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140c9800 0x7f72140cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 -- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72140cb8e0 msgr2=0x7f72140cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72140cb8e0 0x7f72140cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f7219a00700 1 -- 172.21.15.99:0/957704382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140c7590 con 0x7f71fc004d90 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.860+0000 7f721a400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72140cb8e0 0x7f72140cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.861+0000 7f7219a00700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f72140b6140 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f72080072c0 tx=0x7f7208007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.867 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.861+0000 7f7211600700 1 -- 172.21.15.99:0/957704382 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72080014a0 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.861+0000 7f7211600700 1 -- 172.21.15.99:0/957704382 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72080077a0 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.861+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72140c7870 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.861+0000 7f7211600700 1 -- 172.21.15.99:0/957704382 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720800fb10 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.861+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72140b7960 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.862+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72140c7ab0 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.863+0000 7f7211600700 1 -- 172.21.15.99:0/957704382 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f720800fc70 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.864+0000 7f7211600700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71f406ac90 0x7f71f406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.864+0000 7f7211600700 1 -- 172.21.15.99:0/957704382 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7208089120 con 0x7f71fc004d90 2024-03-29T09:12:37.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.864+0000 7f7219000700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71f406ac90 0x7f71f406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:37.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.865+0000 7f7219000700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71f406ac90 0x7f71f406d150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f72140a1090 tx=0x7f7200009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:37.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:37.868+0000 7f7211600700 1 -- 172.21.15.99:0/957704382 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f72140c7ab0 con 0x7f71fc004d90 2024-03-29T09:12:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.076+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f72140b3960 con 0x7f71f406ac90 2024-03-29T09:12:38.076 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.077+0000 7f7211600700 1 -- 172.21.15.99:0/957704382 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7200019070 con 0x7f71f406ac90 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71f406ac90 msgr2=0x7f71f406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71f406ac90 0x7f71f406d150 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f72140a1090 tx=0x7f7200009210 comp rx=0 tx=0).stop 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 msgr2=0x7f72140b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f72140b6140 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f72080072c0 tx=0x7f7208007680 comp rx=0 tx=0).stop 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 shutdown_connections 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f71f406ac90 0x7f71f406d150 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72140cb8e0 0x7f72140cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.080 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71fc004d90 0x7f72140b6140 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 --2- 172.21.15.99:0/957704382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72140c9800 0x7f72140cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.079+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 >> 172.21.15.99:0/957704382 conn(0x7f721409c4a0 msgr2=0x7f72140afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:38.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.080+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 shutdown_connections 2024-03-29T09:12:38.081 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.080+0000 7f721c400700 1 -- 172.21.15.99:0/957704382 wait complete. 2024-03-29T09:12:38.081 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 2 to deep-scrub 2024-03-29T09:12:38.097 INFO:tasks.ceph:Scrubbing osd.3 2024-03-29T09:12:38.098 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:38.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.292+0000 7f7052c00700 1 Processor -- start 2024-03-29T09:12:38.292 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.292+0000 7f7052c00700 1 -- start start 2024-03-29T09:12:38.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.293+0000 7f7052c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 0x7f704c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.293+0000 7f7052c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c0d1fa0 con 0x7f704c160b30 2024-03-29T09:12:38.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.293+0000 7f7052c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c160f10 con 0x7f704c0b3ba0 2024-03-29T09:12:38.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.293+0000 7f7052c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0cf440 con 0x7f704c0d0260 2024-03-29T09:12:38.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.293+0000 7f704be00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f704c160b30 0x7f704c0cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42500/0 (socket says 172.21.15.99:42500) 2024-03-29T09:12:38.293 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.293+0000 7f704be00700 1 -- 172.21.15.99:0/511945471 learned_addr learned my addr 172.21.15.99:0/511945471 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:38.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610817940 0 0) 0x7f704c0d1fa0 con 0x7f704c160b30 2024-03-29T09:12:38.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70300035b0 con 0x7f704c160b30 2024-03-29T09:12:38.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704b400700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 0x7f704c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329180544 0 0) 0x7f70300035b0 con 0x7f704c160b30 2024-03-29T09:12:38.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 msgr2=0x7f704c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 0x7f704c0cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.297 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 >> v1:172.21.15.133:6789/0 conn(0x7f704c0b3ba0 legacy=0x7f704c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0d1670 con 0x7f704c160b30 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f703c001670 con 0x7f704c160b30 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 >> v1:172.21.15.99:6789/0 conn(0x7f704c160b30 legacy=0x7f704c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030003a30 0x7f7030003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7030004850 0x7f7030004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 0x7f7030005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703000bac0 con 0x7f7030004850 2024-03-29T09:12:38.298 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703000bc00 con 0x7f7030003a30 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703000bdc0 con 0x7f704c0d0260 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704b400700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7030004850 0x7f7030004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f704be00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030003a30 0x7f7030003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.294+0000 7f7050c00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 0x7f7030005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704b400700 1 -- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 msgr2=0x7f7030005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704b400700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 0x7f7030005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704b400700 1 -- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030003a30 msgr2=0x7f7030003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.299 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704b400700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7030003a30 0x7f7030003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704b400700 1 -- 172.21.15.99:0/511945471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70300035b0 con 0x7f7030004850 2024-03-29T09:12:38.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f7050c00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f704c0d0260 0x7f7030005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:38.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704b400700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7030004850 0x7f7030004cd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7038009b30 tx=0x7f7030005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2bd48dc8738c5a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703800e6c0 con 0x7f7030004850 2024-03-29T09:12:38.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703800e820 con 0x7f7030004850 2024-03-29T09:12:38.300 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.295+0000 7f704aa00700 1 -- 172.21.15.99:0/511945471 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703800eb90 con 0x7f7030004850 2024-03-29T09:12:38.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.296+0000 7f7052c00700 1 -- 172.21.15.99:0/511945471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7030004850 msgr2=0x7f7030004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.296+0000 7f7052c00700 1 --2- 172.21.15.99:0/511945471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7030004850 0x7f7030004cd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7038009b30 tx=0x7f7030005880 comp rx=0 tx=0).stop 2024-03-29T09:12:38.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.296+0000 7f7050c00700 1 -- 172.21.15.99:0/511945471 reap_dead start 2024-03-29T09:12:38.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.296+0000 7f7052c00700 1 -- 172.21.15.99:0/511945471 shutdown_connections 2024-03-29T09:12:38.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.296+0000 7f7052c00700 1 -- 172.21.15.99:0/511945471 >> 172.21.15.99:0/511945471 conn(0x7f704c09c4d0 msgr2=0x7f704c0d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:38.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.296+0000 7f7052c00700 1 -- 172.21.15.99:0/511945471 shutdown_connections 2024-03-29T09:12:38.301 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.296+0000 7f7052c00700 1 -- 172.21.15.99:0/511945471 wait complete. 2024-03-29T09:12:38.305 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.298+0000 7f7052c00700 1 Processor -- start 2024-03-29T09:12:38.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.298+0000 7f7052c00700 1 -- start start 2024-03-29T09:12:38.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7052c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7030004850 0x7f704c14d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7052c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704c15b9f0 0x7f704c0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7052c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c15be70 0x7f704c0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.306 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7052c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c14b000 con 0x7f704c15be70 2024-03-29T09:12:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7052c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0b6770 con 0x7f704c15b9f0 2024-03-29T09:12:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7052c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0b69c0 con 0x7f7030004850 2024-03-29T09:12:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c15be70 0x7f704c0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c15be70 0x7f704c0b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44926/0 (socket says 172.21.15.99:44926) 2024-03-29T09:12:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 -- 172.21.15.99:0/1585645777 learned_addr learned my addr 172.21.15.99:0/1585645777 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f704b400700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704c15b9f0 0x7f704c0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f704be00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7030004850 0x7f704c14d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 -- 172.21.15.99:0/1585645777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7030004850 msgr2=0x7f704c14d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7030004850 0x7f704c14d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 -- 172.21.15.99:0/1585645777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704c15b9f0 msgr2=0x7f704c0afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704c15b9f0 0x7f704c0afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 -- 172.21.15.99:0/1585645777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0b6ca0 con 0x7f704c15be70 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.299+0000 7f7050c00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c15be70 0x7f704c0b03e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f704c06da00 tx=0x7f704400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.300+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044003c10 con 0x7f704c15be70 2024-03-29T09:12:38.308 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.300+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7044003d90 con 0x7f704c15be70 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.300+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044005310 con 0x7f704c15be70 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.300+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c0b6f80 con 0x7f704c15be70 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.300+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c0b7540 con 0x7f704c15be70 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.301+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f704c0b48e0 con 0x7f704c15be70 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.302+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7044005470 con 0x7f704c15be70 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f7043e00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f702c06ac90 0x7f702c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7044089240 con 0x7f704c15be70 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f704be00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f702c06ac90 0x7f702c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f7043e00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f702c070f20 0x7f702c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 --> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f702c073a50 con 0x7f702c070f20 2024-03-29T09:12:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f704408b820 con 0x7f704c15be70 2024-03-29T09:12:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f704b400700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f702c070f20 0x7f702c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.303+0000 7f704be00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f702c06ac90 0x7f702c06d150 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.304+0000 7f704b400700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f702c070f20 0x7f702c073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.304+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== osd.3 v2:172.21.15.99:6800/42217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f702c073a50 con 0x7f702c070f20 2024-03-29T09:12:38.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.319+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 --> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f704c0d1c30 con 0x7f702c070f20 2024-03-29T09:12:38.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.319+0000 7f7043e00700 1 -- 172.21.15.99:0/1585645777 <== osd.3 v2:172.21.15.99:6800/42217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f703800e990 con 0x7f702c070f20 2024-03-29T09:12:38.319 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:38.320 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:38.320 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f702c070f20 msgr2=0x7f702c073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6800/42217,v1:172.21.15.99:6801/42217] conn(0x7f702c070f20 0x7f702c073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f702c06ac90 msgr2=0x7f702c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f702c06ac90 0x7f702c06d150 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c009040 comp rx=0 tx=0).stop 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c15be70 msgr2=0x7f704c0b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 --2- 172.21.15.99:0/1585645777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704c15be70 0x7f704c0b03e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f704c06da00 tx=0x7f704400ee70 comp rx=0 tx=0).stop 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7050c00700 1 -- 172.21.15.99:0/1585645777 reap_dead start 2024-03-29T09:12:38.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 shutdown_connections 2024-03-29T09:12:38.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 >> 172.21.15.99:0/1585645777 conn(0x7f704c09c4d0 msgr2=0x7f704c0cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:38.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 shutdown_connections 2024-03-29T09:12:38.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.320+0000 7f7052c00700 1 -- 172.21.15.99:0/1585645777 wait complete. 2024-03-29T09:12:38.337 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-29T09:12:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.527+0000 7fb465a00700 1 Processor -- start 2024-03-29T09:12:38.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.527+0000 7fb465a00700 1 -- start start 2024-03-29T09:12:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.528+0000 7fb465a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb460153d20 0x7fb460070540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.528+0000 7fb465a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb46015d330 0x7fb46015d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.528+0000 7fb465a00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb460159d40 0x7fb46015a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.528+0000 7fb465a00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4600ccf60 con 0x7fb460159d40 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.528+0000 7fb465a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46015dcd0 con 0x7fb460153d20 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.528+0000 7fb465a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb460071750 con 0x7fb46015d330 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb460159d40 0x7fb46015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fb460159d40 0x7fb46015a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44934/0 (socket says 172.21.15.99:44934) 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 -- 172.21.15.99:0/2087471196 learned_addr learned my addr 172.21.15.99:0/2087471196 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45e000700 1 --2- 172.21.15.99:0/2087471196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb46015d330 0x7fb46015d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45ea00700 1 --2- 172.21.15.99:0/2087471196 >> v2:172.21.15.133:3300/0 conn(0x7fb460153d20 0x7fb460070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 -- 172.21.15.99:0/2087471196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb46015d330 msgr2=0x7fb46015d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 --2- 172.21.15.99:0/2087471196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb46015d330 0x7fb46015d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 -- 172.21.15.99:0/2087471196 >> v2:172.21.15.133:3300/0 conn(0x7fb460153d20 msgr2=0x7fb460070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 --2- 172.21.15.99:0/2087471196 >> v2:172.21.15.133:3300/0 conn(0x7fb460153d20 0x7fb460070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 -- 172.21.15.99:0/2087471196 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460070eb0 con 0x7fb460159d40 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45ea00700 1 --2- 172.21.15.99:0/2087471196 >> v2:172.21.15.133:3300/0 conn(0x7fb460153d20 0x7fb460070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45f400700 1 --2- 172.21.15.99:0/2087471196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460159d40 0x7fb46015a170 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb46006c160 tx=0x7fb45800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b324eea9aee7513 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45d600700 1 -- 172.21.15.99:0/2087471196 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458010070 con 0x7fb460159d40 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45d600700 1 -- 172.21.15.99:0/2087471196 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800dad0 con 0x7fb460159d40 2024-03-29T09:12:38.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.529+0000 7fb45d600700 1 -- 172.21.15.99:0/2087471196 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800de60 con 0x7fb460159d40 2024-03-29T09:12:38.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 -- 172.21.15.99:0/2087471196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460159d40 msgr2=0x7fb46015a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 --2- 172.21.15.99:0/2087471196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460159d40 0x7fb46015a170 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb46006c160 tx=0x7fb45800d430 comp rx=0 tx=0).stop 2024-03-29T09:12:38.534 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 -- 172.21.15.99:0/2087471196 shutdown_connections 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 --2- 172.21.15.99:0/2087471196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb460159d40 0x7fb46015a170 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 --2- 172.21.15.99:0/2087471196 >> v2:172.21.15.133:3300/0 conn(0x7fb460153d20 0x7fb460070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 --2- 172.21.15.99:0/2087471196 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb46015d330 0x7fb46015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 -- 172.21.15.99:0/2087471196 >> 172.21.15.99:0/2087471196 conn(0x7fb460066540 msgr2=0x7fb460066950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 -- 172.21.15.99:0/2087471196 shutdown_connections 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.530+0000 7fb465a00700 1 -- 172.21.15.99:0/2087471196 wait complete. 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.532+0000 7fb465a00700 1 Processor -- start 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.532+0000 7fb465a00700 1 -- start start 2024-03-29T09:12:38.535 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb465a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb460153d20 0x7fb4601505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb465a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460159d40 0x7fb460156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb465a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46015d330 0x7fb460156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb465a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600aeac0 con 0x7fb46015d330 2024-03-29T09:12:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb465a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460150b20 con 0x7fb460159d40 2024-03-29T09:12:38.536 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb465a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb460157200 con 0x7fb460153d20 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb45f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46015d330 0x7fb460156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb45f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46015d330 0x7fb460156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44946/0 (socket says 172.21.15.99:44946) 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb45f400700 1 -- 172.21.15.99:0/2305953788 learned_addr learned my addr 172.21.15.99:0/2305953788 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb45e000700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460159d40 0x7fb460156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.533+0000 7fb45ea00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb460153d20 0x7fb4601505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb45f400700 1 -- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb460153d20 msgr2=0x7fb4601505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb45f400700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb460153d20 0x7fb4601505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb45f400700 1 -- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460159d40 msgr2=0x7fb460156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.537 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb45f400700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460159d40 0x7fb460156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb45f400700 1 -- 172.21.15.99:0/2305953788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45800bd40 con 0x7fb46015d330 2024-03-29T09:12:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb45f400700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46015d330 0x7fb460156cc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb46006ac80 tx=0x7fb458004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb456a00700 1 -- 172.21.15.99:0/2305953788 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800b4c0 con 0x7fb46015d330 2024-03-29T09:12:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb456a00700 1 -- 172.21.15.99:0/2305953788 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb458004140 con 0x7fb46015d330 2024-03-29T09:12:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4601551f0 con 0x7fb46015d330 2024-03-29T09:12:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb456a00700 1 -- 172.21.15.99:0/2305953788 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580044d0 con 0x7fb46015d330 2024-03-29T09:12:38.538 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.534+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb460154720 con 0x7fb46015d330 2024-03-29T09:12:38.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.535+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4600aec00 con 0x7fb46015d330 2024-03-29T09:12:38.539 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.536+0000 7fb456a00700 1 -- 172.21.15.99:0/2305953788 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb458004630 con 0x7fb46015d330 2024-03-29T09:12:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.537+0000 7fb456a00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb44006ac90 0x7fb44006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.537+0000 7fb456a00700 1 -- 172.21.15.99:0/2305953788 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb45808a660 con 0x7fb46015d330 2024-03-29T09:12:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.537+0000 7fb45ea00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb44006ac90 0x7fb44006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.540 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.538+0000 7fb45ea00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb44006ac90 0x7fb44006d150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb4500045f0 tx=0x7fb450008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.541 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.542+0000 7fb456a00700 1 -- 172.21.15.99:0/2305953788 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb4600aec00 con 0x7fb46015d330 2024-03-29T09:12:38.744 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.745+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4600cd190 con 0x7fb44006ac90 2024-03-29T09:12:38.745 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.745+0000 7fb456a00700 1 -- 172.21.15.99:0/2305953788 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb450009580 con 0x7fb44006ac90 2024-03-29T09:12:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.747+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb44006ac90 msgr2=0x7fb44006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.747+0000 7fb465a00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb44006ac90 0x7fb44006d150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb4500045f0 tx=0x7fb450008040 comp rx=0 tx=0).stop 2024-03-29T09:12:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.747+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46015d330 msgr2=0x7fb460156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.747+0000 7fb465a00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46015d330 0x7fb460156cc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb46006ac80 tx=0x7fb458004ad0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 shutdown_connections 2024-03-29T09:12:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb44006ac90 0x7fb44006d150 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.748 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb46015d330 0x7fb460156cc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb460153d20 0x7fb4601505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 --2- 172.21.15.99:0/2305953788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb460159d40 0x7fb460156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 >> 172.21.15.99:0/2305953788 conn(0x7fb460066540 msgr2=0x7fb4600669b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 shutdown_connections 2024-03-29T09:12:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.748+0000 7fb465a00700 1 -- 172.21.15.99:0/2305953788 wait complete. 2024-03-29T09:12:38.749 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 3 to deep-scrub 2024-03-29T09:12:38.766 INFO:tasks.ceph:Scrubbing osd.4 2024-03-29T09:12:38.766 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.961+0000 7f2d4cc00700 1 Processor -- start 2024-03-29T09:12:38.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.961+0000 7f2d4cc00700 1 -- start start 2024-03-29T09:12:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.962+0000 7f2d4cc00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2d48160b30 0x7f2d480cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.965 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d4cc00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d480d1fa0 con 0x7f2d480b1f50 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d4cc00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d48160f10 con 0x7f2d480d0090 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d4cc00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d480cf440 con 0x7f2d48160b30 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d45400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f2d480b1f50 0x7f2d480b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42510/0 (socket says 172.21.15.99:42510) 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d45400700 1 -- 172.21.15.99:0/982728997 learned_addr learned my addr 172.21.15.99:0/982728997 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d45e00700 1 --2- 172.21.15.99:0/982728997 >> v2:172.21.15.133:3300/0 conn(0x7f2d48160b30 0x7f2d480cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873831520 0 0) 0x7f2d480d1fa0 con 0x7f2d480b1f50 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d2c0035b0 con 0x7f2d480b1f50 2024-03-29T09:12:38.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601693522 0 0) 0x7f2d48160f10 con 0x7f2d480d0090 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d480d1fa0 con 0x7f2d480d0090 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 775667730 0 0) 0x7f2d2c0035b0 con 0x7f2d480b1f50 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.963+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 >> v2:172.21.15.133:3300/0 conn(0x7f2d48160b30 msgr2=0x7f2d480cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 --2- 172.21.15.99:0/982728997 >> v2:172.21.15.133:3300/0 conn(0x7f2d48160b30 0x7f2d480cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 >> v1:172.21.15.133:6789/0 conn(0x7f2d480d0090 legacy=0x7f2d480b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480d1670 con 0x7f2d480b1f50 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f2d34001670 con 0x7f2d480b1f50 2024-03-29T09:12:38.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45e00700 1 --2- 172.21.15.99:0/982728997 >> v2:172.21.15.133:3300/0 conn(0x7f2d48160b30 0x7f2d480cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 >> v1:172.21.15.99:6789/0 conn(0x7f2d480b1f50 legacy=0x7f2d480b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d2c003b90 0x7f2d2c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2c004850 0x7f2d2c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c005210 0x7f2d2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c00c060 con 0x7f2d2c003b90 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c00c1d0 con 0x7f2d2c004850 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c00c390 con 0x7f2d2c005210 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45e00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d2c003b90 0x7f2d2c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45400700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2c004850 0x7f2d2c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d46800700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c005210 0x7f2d2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45e00700 1 -- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c005210 msgr2=0x7f2d2c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45e00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c005210 0x7f2d2c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45e00700 1 -- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2c004850 msgr2=0x7f2d2c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45e00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2c004850 0x7f2d2c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45e00700 1 -- 172.21.15.99:0/982728997 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c0035b0 con 0x7f2d2c003b90 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d45400700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d2c004850 0x7f2d2c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:38.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.964+0000 7f2d46800700 1 -- 172.21.15.99:0/982728997 reap_dead start 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d45e00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d2c003b90 0x7f2d2c003f90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009d30 tx=0x7f2d2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2923d9403c8939ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38009e80 con 0x7f2d2c003b90 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3800e9a0 con 0x7f2d2c003b90 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d44a00700 1 -- 172.21.15.99:0/982728997 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800ed10 con 0x7f2d2c003b90 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d4cc00700 1 -- 172.21.15.99:0/982728997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d2c003b90 msgr2=0x7f2d2c003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d4cc00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d2c003b90 0x7f2d2c003f90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009d30 tx=0x7f2d2c005880 comp rx=0 tx=0).stop 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d4cc00700 1 -- 172.21.15.99:0/982728997 shutdown_connections 2024-03-29T09:12:38.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d4cc00700 1 --2- 172.21.15.99:0/982728997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d2c003b90 0x7f2d2c003f90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d4cc00700 1 -- 172.21.15.99:0/982728997 >> 172.21.15.99:0/982728997 conn(0x7f2d4809c4d0 msgr2=0x7f2d480d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:38.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.965+0000 7f2d4cc00700 1 -- 172.21.15.99:0/982728997 shutdown_connections 2024-03-29T09:12:38.971 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.966+0000 7f2d4cc00700 1 -- 172.21.15.99:0/982728997 wait complete. 2024-03-29T09:12:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.968+0000 7f2d4cc00700 1 Processor -- start 2024-03-29T09:12:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.968+0000 7f2d4cc00700 1 -- start start 2024-03-29T09:12:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.969+0000 7f2d4cc00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c003b90 0x7f2d480ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.969+0000 7f2d4cc00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d480b5880 0x7f2d480b7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.969+0000 7f2d4cc00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d480b2d70 0x7f2d4809dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.969+0000 7f2d4cc00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480b6ce0 con 0x7f2d480b2d70 2024-03-29T09:12:38.974 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.969+0000 7f2d4cc00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480b6060 con 0x7f2d480b5880 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.969+0000 7f2d4cc00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480b6340 con 0x7f2d2c003b90 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d480b2d70 0x7f2d4809dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d45e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c003b90 0x7f2d480ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d480b2d70 0x7f2d4809dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44966/0 (socket says 172.21.15.99:44966) 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 -- 172.21.15.99:0/3501601443 learned_addr learned my addr 172.21.15.99:0/3501601443 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d45400700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d480b5880 0x7f2d480b7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 -- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c003b90 msgr2=0x7f2d480ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2d2c003b90 0x7f2d480ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.975 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 -- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d480b5880 msgr2=0x7f2d480b7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2d480b5880 0x7f2d480b7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 -- 172.21.15.99:0/3501601443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480b6620 con 0x7f2d480b2d70 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d46800700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d480b2d70 0x7f2d4809dee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2d4808c600 tx=0x7f2d4000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40003c10 con 0x7f2d480b2d70 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d40003d90 con 0x7f2d480b2d70 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d480b6900 con 0x7f2d480b2d70 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.970+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40005310 con 0x7f2d480b2d70 2024-03-29T09:12:38.976 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.971+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d480b5d00 con 0x7f2d480b2d70 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.971+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2d480d1c30 con 0x7f2d480b2d70 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.973+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f2d40005470 con 0x7f2d480b2d70 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d3e000700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d2806ac90 0x7f2d2806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2d400890f0 con 0x7f2d480b2d70 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d45e00700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d2806ac90 0x7f2d2806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d3e000700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f2d28070f20 0x7f2d28073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 --> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d28073a50 con 0x7f2d28070f20 2024-03-29T09:12:38.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d4008b7b0 con 0x7f2d480b2d70 2024-03-29T09:12:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d45400700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f2d28070f20 0x7f2d28073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.974+0000 7f2d45e00700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d2806ac90 0x7f2d2806d150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000c00 tx=0x7f2d38008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.975+0000 7f2d45400700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f2d28070f20 0x7f2d28073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:38.979 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.976+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== osd.4 v2:172.21.15.133:6800/42489 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f2d28073a50 con 0x7f2d28070f20 2024-03-29T09:12:38.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.991+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 --> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2d480d0500 con 0x7f2d28070f20 2024-03-29T09:12:38.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.991+0000 7f2d3e000700 1 -- 172.21.15.99:0/3501601443 <== osd.4 v2:172.21.15.133:6800/42489 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2d34005ba0 con 0x7f2d28070f20 2024-03-29T09:12:38.992 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:38.992 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f2d28070f20 msgr2=0x7f2d28073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.133:6800/42489,v1:172.21.15.133:6801/42489] conn(0x7f2d28070f20 0x7f2d28073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d2806ac90 msgr2=0x7f2d2806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f2d2806ac90 0x7f2d2806d150 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000c00 tx=0x7f2d38008030 comp rx=0 tx=0).stop 2024-03-29T09:12:38.993 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d480b2d70 msgr2=0x7f2d4809dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 --2- 172.21.15.99:0/3501601443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d480b2d70 0x7f2d4809dee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2d4808c600 tx=0x7f2d4000eee0 comp rx=0 tx=0).stop 2024-03-29T09:12:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d46800700 1 -- 172.21.15.99:0/3501601443 reap_dead start 2024-03-29T09:12:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 shutdown_connections 2024-03-29T09:12:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 >> 172.21.15.99:0/3501601443 conn(0x7f2d4809c4d0 msgr2=0x7f2d480b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 shutdown_connections 2024-03-29T09:12:38.994 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:38.992+0000 7f2d4cc00700 1 -- 172.21.15.99:0/3501601443 wait complete. 2024-03-29T09:12:39.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-29T09:12:39.202 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.202+0000 7f1676c00700 1 Processor -- start 2024-03-29T09:12:39.203 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.203+0000 7f1676c00700 1 -- start start 2024-03-29T09:12:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1676c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f16701608e0 0x7f16700d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1676c00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f16700d1b20 0x7f16700ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1676c00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16700cddc0 con 0x7f1670160cc0 2024-03-29T09:12:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1676c00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16700ce0a0 con 0x7f16700d1b20 2024-03-29T09:12:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1676c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16700ce380 con 0x7f16701608e0 2024-03-29T09:12:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1667e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f1670160cc0 0x7f16700d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42526/0 (socket says 172.21.15.99:42526) 2024-03-29T09:12:39.205 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1667e00700 1 -- 172.21.15.99:0/2357301264 learned_addr learned my addr 172.21.15.99:0/2357301264 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f1674c00700 1 --2- 172.21.15.99:0/2357301264 >> v2:172.21.15.99:3300/0 conn(0x7f16700d1b20 0x7f16700ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f166fe00700 1 --2- 172.21.15.99:0/2357301264 >> v2:172.21.15.133:3300/0 conn(0x7f16701608e0 0x7f16700d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f166f400700 1 -- 172.21.15.99:0/2357301264 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130217612 0 0) 0x7f16700cddc0 con 0x7f1670160cc0 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.204+0000 7f166f400700 1 -- 172.21.15.99:0/2357301264 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16540035b0 con 0x7f1670160cc0 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f1674c00700 1 -- 172.21.15.99:0/2357301264 >> v2:172.21.15.133:3300/0 conn(0x7f16701608e0 msgr2=0x7f16700d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f1674c00700 1 --2- 172.21.15.99:0/2357301264 >> v2:172.21.15.133:3300/0 conn(0x7f16701608e0 0x7f16700d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f1674c00700 1 -- 172.21.15.99:0/2357301264 >> v1:172.21.15.99:6789/0 conn(0x7f1670160cc0 legacy=0x7f16700d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.206 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f1674c00700 1 -- 172.21.15.99:0/2357301264 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700ce660 con 0x7f16700d1b20 2024-03-29T09:12:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f1674c00700 1 --2- 172.21.15.99:0/2357301264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16700ceb80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f167008d790 tx=0x7f166800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb604c46f11b652 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f166f400700 1 -- 172.21.15.99:0/2357301264 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166800e070 con 0x7f16700d1b20 2024-03-29T09:12:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f166f400700 1 -- 172.21.15.99:0/2357301264 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166800bb20 con 0x7f16700d1b20 2024-03-29T09:12:39.207 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f166f400700 1 -- 172.21.15.99:0/2357301264 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1668012430 con 0x7f16700d1b20 2024-03-29T09:12:39.209 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f1676c00700 1 -- 172.21.15.99:0/2357301264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 msgr2=0x7f16700ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.205+0000 7f1676c00700 1 --2- 172.21.15.99:0/2357301264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16700ceb80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f167008d790 tx=0x7f166800b330 comp rx=0 tx=0).stop 2024-03-29T09:12:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.206+0000 7f1676c00700 1 -- 172.21.15.99:0/2357301264 shutdown_connections 2024-03-29T09:12:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.206+0000 7f1676c00700 1 --2- 172.21.15.99:0/2357301264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16700ceb80 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.206+0000 7f1676c00700 1 --2- 172.21.15.99:0/2357301264 >> v2:172.21.15.133:3300/0 conn(0x7f16701608e0 0x7f16700d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.206+0000 7f1676c00700 1 -- 172.21.15.99:0/2357301264 >> 172.21.15.99:0/2357301264 conn(0x7f167009c490 msgr2=0x7f16700d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.206+0000 7f1676c00700 1 -- 172.21.15.99:0/2357301264 shutdown_connections 2024-03-29T09:12:39.210 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.206+0000 7f1676c00700 1 -- 172.21.15.99:0/2357301264 wait complete. 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.207+0000 7f1676c00700 1 Processor -- start 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.208+0000 7f1676c00700 1 -- start start 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.208+0000 7f1676c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16701f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.208+0000 7f1676c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16701608e0 0x7f1670158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.208+0000 7f1676c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1670160cc0 0x7f16700b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.208+0000 7f1676c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16701f3c40 con 0x7f16700d1b20 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.208+0000 7f1676c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1670157650 con 0x7f16701608e0 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.208+0000 7f1676c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16701578f0 con 0x7f1670160cc0 2024-03-29T09:12:39.211 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16701f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16701f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:44982/0 (socket says 172.21.15.99:44982) 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 -- 172.21.15.99:0/2256224713 learned_addr learned my addr 172.21.15.99:0/2256224713 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f1667e00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16701608e0 0x7f1670158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f1674c00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1670160cc0 0x7f16700b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 -- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1670160cc0 msgr2=0x7f16700b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1670160cc0 0x7f16700b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 -- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16701608e0 msgr2=0x7f1670158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.212 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16701608e0 0x7f1670158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 -- 172.21.15.99:0/2256224713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668009a70 con 0x7f16700d1b20 2024-03-29T09:12:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.209+0000 7f166fe00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16701f3700 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f166000e9b0 tx=0x7f166000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.210+0000 7f166cc00700 1 -- 172.21.15.99:0/2256224713 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660009710 con 0x7f16700d1b20 2024-03-29T09:12:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.210+0000 7f166cc00700 1 -- 172.21.15.99:0/2256224713 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166000c850 con 0x7f16700d1b20 2024-03-29T09:12:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.210+0000 7f166cc00700 1 -- 172.21.15.99:0/2256224713 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166000cbc0 con 0x7f16700d1b20 2024-03-29T09:12:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.210+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1670157bd0 con 0x7f16700d1b20 2024-03-29T09:12:39.213 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.210+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16701f31e0 con 0x7f16700d1b20 2024-03-29T09:12:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.211+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16701f3550 con 0x7f16700d1b20 2024-03-29T09:12:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.212+0000 7f166cc00700 1 -- 172.21.15.99:0/2256224713 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f166000cd20 con 0x7f16700d1b20 2024-03-29T09:12:39.214 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.212+0000 7f166cc00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f164c06ac90 0x7f164c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.213+0000 7f166cc00700 1 -- 172.21.15.99:0/2256224713 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1660089280 con 0x7f16700d1b20 2024-03-29T09:12:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.213+0000 7f1667e00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f164c06ac90 0x7f164c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.215 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.213+0000 7f1667e00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f164c06ac90 0x7f164c06d150 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f167015ea80 tx=0x7f1658009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.217+0000 7f166cc00700 1 -- 172.21.15.99:0/2256224713 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f1660054610 con 0x7f16700d1b20 2024-03-29T09:12:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.417+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f16700b22a0 con 0x7f164c06ac90 2024-03-29T09:12:39.417 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.417+0000 7f166cc00700 1 -- 172.21.15.99:0/2256224713 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1658001460 con 0x7f164c06ac90 2024-03-29T09:12:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f164c06ac90 msgr2=0x7f164c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f164c06ac90 0x7f164c06d150 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f167015ea80 tx=0x7f1658009040 comp rx=0 tx=0).stop 2024-03-29T09:12:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 msgr2=0x7f16701f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16701f3700 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f166000e9b0 tx=0x7f166000ed70 comp rx=0 tx=0).stop 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 shutdown_connections 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f164c06ac90 0x7f164c06d150 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1670160cc0 0x7f16700b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16700d1b20 0x7f16701f3700 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 --2- 172.21.15.99:0/2256224713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16701608e0 0x7f1670158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 >> 172.21.15.99:0/2256224713 conn(0x7f167009c490 msgr2=0x7f16700b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 shutdown_connections 2024-03-29T09:12:39.421 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.420+0000 7f1676c00700 1 -- 172.21.15.99:0/2256224713 wait complete. 2024-03-29T09:12:39.422 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 4 to deep-scrub 2024-03-29T09:12:39.438 INFO:tasks.ceph:Scrubbing osd.5 2024-03-29T09:12:39.439 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:39.632 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.632+0000 7fb496800700 1 Processor -- start 2024-03-29T09:12:39.633 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.633+0000 7fb496800700 1 -- start start 2024-03-29T09:12:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.634+0000 7fb496800700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb4900b3fb0 0x7fb4900d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.634+0000 7fb496800700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900c6cd0 con 0x7fb490160980 2024-03-29T09:12:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.634+0000 7fb496800700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900d1f40 con 0x7fb4900d00d0 2024-03-29T09:12:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.634+0000 7fb496800700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4900d1330 con 0x7fb4900b3fb0 2024-03-29T09:12:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.634+0000 7fb48f400700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb490160980 0x7fb4900cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42538/0 (socket says 172.21.15.99:42538) 2024-03-29T09:12:39.634 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.634+0000 7fb48f400700 1 -- 172.21.15.99:0/3522566788 learned_addr learned my addr 172.21.15.99:0/3522566788 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.634+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097321800 0 0) 0x7fb4900c6cd0 con 0x7fb490160980 2024-03-29T09:12:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4740035b0 con 0x7fb490160980 2024-03-29T09:12:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48fe00700 1 --2- 172.21.15.99:0/3522566788 >> v2:172.21.15.133:3300/0 conn(0x7fb4900b3fb0 0x7fb4900d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1608868474 0 0) 0x7fb4740035b0 con 0x7fb490160980 2024-03-29T09:12:39.637 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 >> v2:172.21.15.133:3300/0 conn(0x7fb4900b3fb0 msgr2=0x7fb4900d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 --2- 172.21.15.99:0/3522566788 >> v2:172.21.15.133:3300/0 conn(0x7fb4900b3fb0 0x7fb4900d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 >> v1:172.21.15.133:6789/0 conn(0x7fb4900d00d0 legacy=0x7fb4900cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900d1610 con 0x7fb490160980 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7fb480001670 con 0x7fb490160980 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 >> v1:172.21.15.99:6789/0 conn(0x7fb490160980 legacy=0x7fb4900cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb474003a00 0x7fb474003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004850 0x7fb474004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.638 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb47400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900d1330 con 0x7fb474005330 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47400c200 con 0x7fb474004850 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47400c3c0 con 0x7fb474003a00 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48fe00700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb47400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48f400700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb474003a00 0x7fb474003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.635+0000 7fb48ea00700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004850 0x7fb474004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48fe00700 1 -- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb474003a00 msgr2=0x7fb474003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48fe00700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb474003a00 0x7fb474003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.639 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48fe00700 1 -- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004850 msgr2=0x7fb474004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48fe00700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb474004850 0x7fb474004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48fe00700 1 -- 172.21.15.99:0/3522566788 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740035b0 con 0x7fb474005330 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48fe00700 1 -- 172.21.15.99:0/3522566788 reap_dead start 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48fe00700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb47400ba10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb488007330 tx=0x7fb474005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a7df965df4231d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488007e50 con 0x7fb474005330 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb488003a20 con 0x7fb474005330 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.636+0000 7fb48e000700 1 -- 172.21.15.99:0/3522566788 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488003dd0 con 0x7fb474005330 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.637+0000 7fb496800700 1 -- 172.21.15.99:0/3522566788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 msgr2=0x7fb47400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.640 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.637+0000 7fb496800700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb47400ba10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb488007330 tx=0x7fb474005880 comp rx=0 tx=0).stop 2024-03-29T09:12:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.637+0000 7fb496800700 1 -- 172.21.15.99:0/3522566788 shutdown_connections 2024-03-29T09:12:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.637+0000 7fb496800700 1 --2- 172.21.15.99:0/3522566788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb47400ba10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.637+0000 7fb496800700 1 -- 172.21.15.99:0/3522566788 >> 172.21.15.99:0/3522566788 conn(0x7fb49009c4e0 msgr2=0x7fb4900d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.637+0000 7fb496800700 1 -- 172.21.15.99:0/3522566788 shutdown_connections 2024-03-29T09:12:39.641 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.637+0000 7fb496800700 1 -- 172.21.15.99:0/3522566788 wait complete. 2024-03-29T09:12:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.639+0000 7fb496800700 1 Processor -- start 2024-03-29T09:12:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.639+0000 7fb496800700 1 -- start start 2024-03-29T09:12:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb496800700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb490151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb496800700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4900cac60 0x7fb490152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb496800700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4900b7bd0 0x7fb4900b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb496800700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900b8590 con 0x7fb474005330 2024-03-29T09:12:39.644 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb496800700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900c6a00 con 0x7fb4900cac60 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb496800700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900c6ce0 con 0x7fb4900b7bd0 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb490151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb490151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45002/0 (socket says 172.21.15.99:45002) 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 -- 172.21.15.99:0/1622253659 learned_addr learned my addr 172.21.15.99:0/1622253659 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48fe00700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4900b7bd0 0x7fb4900b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48ea00700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4900cac60 0x7fb490152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 -- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4900b7bd0 msgr2=0x7fb4900b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.645 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4900b7bd0 0x7fb4900b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 -- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4900cac60 msgr2=0x7fb490152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4900cac60 0x7fb490152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.640+0000 7fb48f400700 1 -- 172.21.15.99:0/1622253659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900c6fc0 con 0x7fb474005330 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.641+0000 7fb48f400700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb490151600 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb480007290 tx=0x7fb4800075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.641+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480011580 con 0x7fb474005330 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.641+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4800116e0 con 0x7fb474005330 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.641+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4900c72a0 con 0x7fb474005330 2024-03-29T09:12:39.646 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.641+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480011a90 con 0x7fb474005330 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.641+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4900c7860 con 0x7fb474005330 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.642+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb490160d60 con 0x7fb474005330 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.643+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb480011bf0 con 0x7fb474005330 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.644+0000 7fb487400700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb47006ac90 0x7fb47006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.644+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb4800891d0 con 0x7fb474005330 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.644+0000 7fb48ea00700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb47006ac90 0x7fb47006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.644+0000 7fb487400700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7fb470070f20 0x7fb470073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.644+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 --> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb470073a50 con 0x7fb470070f20 2024-03-29T09:12:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.644+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb48008b7b0 con 0x7fb474005330 2024-03-29T09:12:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.644+0000 7fb48ea00700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb47006ac90 0x7fb47006d150 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb4901f6590 tx=0x7fb47c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.645+0000 7fb48fe00700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7fb470070f20 0x7fb470073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.645+0000 7fb48fe00700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7fb470070f20 0x7fb470073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.646+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== osd.5 v2:172.21.15.133:6816/42491 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fb470073a50 con 0x7fb470070f20 2024-03-29T09:12:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.661+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 --> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb4900d1bd0 con 0x7fb470070f20 2024-03-29T09:12:39.661 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb487400700 1 -- 172.21.15.99:0/1622253659 <== osd.5 v2:172.21.15.133:6816/42491 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb488007e70 con 0x7fb470070f20 2024-03-29T09:12:39.662 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:39.662 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:39.662 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7fb470070f20 msgr2=0x7fb470073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.133:6816/42491,v1:172.21.15.133:6817/42491] conn(0x7fb470070f20 0x7fb470073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb47006ac90 msgr2=0x7fb47006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.663 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb47006ac90 0x7fb47006d150 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb4901f6590 tx=0x7fb47c009040 comp rx=0 tx=0).stop 2024-03-29T09:12:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 msgr2=0x7fb490151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 --2- 172.21.15.99:0/1622253659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb474005330 0x7fb490151600 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb480007290 tx=0x7fb4800075a0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb48fe00700 1 -- 172.21.15.99:0/1622253659 reap_dead start 2024-03-29T09:12:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 shutdown_connections 2024-03-29T09:12:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.662+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 >> 172.21.15.99:0/1622253659 conn(0x7fb49009c4e0 msgr2=0x7fb490161770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.663+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 shutdown_connections 2024-03-29T09:12:39.664 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.663+0000 7fb496800700 1 -- 172.21.15.99:0/1622253659 wait complete. 2024-03-29T09:12:39.679 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-29T09:12:39.870 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.870+0000 7fa3c9400700 1 Processor -- start 2024-03-29T09:12:39.871 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.870+0000 7fa3c9400700 1 -- start start 2024-03-29T09:12:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.871+0000 7fa3c9400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c41608e0 0x7fa3c40d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.871+0000 7fa3c9400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa3c4160cc0 0x7fa3c40ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.871+0000 7fa3c9400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c40cddc0 con 0x7fa3c40d3a70 2024-03-29T09:12:39.872 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c9400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c40ce0a0 con 0x7fa3c4160cc0 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c9400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40ce380 con 0x7fa3c41608e0 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c1e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa3c4160cc0 0x7fa3c40ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c3200700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fa3c40d3a70 0x7fa3c40d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42554/0 (socket says 172.21.15.99:42554) 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c3200700 1 -- 172.21.15.99:0/2410580983 learned_addr learned my addr 172.21.15.99:0/2410580983 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c2800700 1 --2- 172.21.15.99:0/2410580983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c41608e0 0x7fa3c40d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c1e00700 1 -- 172.21.15.99:0/2410580983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c41608e0 msgr2=0x7fa3c40d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c1e00700 1 --2- 172.21.15.99:0/2410580983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c41608e0 0x7fa3c40d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c1e00700 1 -- 172.21.15.99:0/2410580983 >> v1:172.21.15.99:6789/0 conn(0x7fa3c40d3a70 legacy=0x7fa3c40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.873 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c1e00700 1 -- 172.21.15.99:0/2410580983 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40ce660 con 0x7fa3c4160cc0 2024-03-29T09:12:39.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.872+0000 7fa3c1400700 1 -- 172.21.15.99:0/2410580983 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682361854 0 0) 0x7fa3c40cddc0 con 0x7fa3c40d3a70 2024-03-29T09:12:39.874 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c1e00700 1 --2- 172.21.15.99:0/2410580983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c4160cc0 0x7fa3c40ceb80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009840 tx=0x7fa3b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d286245d823dcc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c1400700 1 -- 172.21.15.99:0/2410580983 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0011830 con 0x7fa3c4160cc0 2024-03-29T09:12:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c1400700 1 -- 172.21.15.99:0/2410580983 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b0011990 con 0x7fa3c4160cc0 2024-03-29T09:12:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c1400700 1 -- 172.21.15.99:0/2410580983 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0011d20 con 0x7fa3c4160cc0 2024-03-29T09:12:39.875 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 -- 172.21.15.99:0/2410580983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c4160cc0 msgr2=0x7fa3c40ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 --2- 172.21.15.99:0/2410580983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c4160cc0 0x7fa3c40ceb80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009840 tx=0x7fa3b0009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 -- 172.21.15.99:0/2410580983 shutdown_connections 2024-03-29T09:12:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 --2- 172.21.15.99:0/2410580983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c41608e0 0x7fa3c40d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 --2- 172.21.15.99:0/2410580983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c4160cc0 0x7fa3c40ceb80 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 -- 172.21.15.99:0/2410580983 >> 172.21.15.99:0/2410580983 conn(0x7fa3c409c490 msgr2=0x7fa3c40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 -- 172.21.15.99:0/2410580983 shutdown_connections 2024-03-29T09:12:39.876 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.873+0000 7fa3c9400700 1 -- 172.21.15.99:0/2410580983 wait complete. 2024-03-29T09:12:39.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.875+0000 7fa3c9400700 1 Processor -- start 2024-03-29T09:12:39.877 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.875+0000 7fa3c9400700 1 -- start start 2024-03-29T09:12:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c9400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c40d3a70 0x7fa3c4159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c9400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c41608e0 0x7fa3c415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c9400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c4160cc0 0x7fa3c415f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c9400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c415a2b0 con 0x7fa3c41608e0 2024-03-29T09:12:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c9400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40b7a80 con 0x7fa3c40d3a70 2024-03-29T09:12:39.881 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c9400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40b7d60 con 0x7fa3c4160cc0 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c41608e0 0x7fa3c415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c41608e0 0x7fa3c415f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45018/0 (socket says 172.21.15.99:45018) 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c1e00700 1 -- 172.21.15.99:0/3824414327 learned_addr learned my addr 172.21.15.99:0/3824414327 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.876+0000 7fa3c2800700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c40d3a70 0x7fa3c4159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c3200700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c4160cc0 0x7fa3c415f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c1e00700 1 -- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c4160cc0 msgr2=0x7fa3c415f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c1e00700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c4160cc0 0x7fa3c415f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.882 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c1e00700 1 -- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c40d3a70 msgr2=0x7fa3c4159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c1e00700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c40d3a70 0x7fa3c4159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c1e00700 1 -- 172.21.15.99:0/3824414327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b00094a0 con 0x7fa3c41608e0 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c1e00700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c41608e0 0x7fa3c415f4a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009fd0 tx=0x7fa3b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3baa00700 1 -- 172.21.15.99:0/3824414327 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00036a0 con 0x7fa3c41608e0 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3baa00700 1 -- 172.21.15.99:0/3824414327 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b001f070 con 0x7fa3c41608e0 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3baa00700 1 -- 172.21.15.99:0/3824414327 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0003a60 con 0x7fa3c41608e0 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c40b8040 con 0x7fa3c41608e0 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.877+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c40b5e10 con 0x7fa3c41608e0 2024-03-29T09:12:39.883 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.878+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c40677a0 con 0x7fa3c41608e0 2024-03-29T09:12:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.879+0000 7fa3baa00700 1 -- 172.21.15.99:0/3824414327 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa3b0003bc0 con 0x7fa3c41608e0 2024-03-29T09:12:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.880+0000 7fa3baa00700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa3a406ac90 0x7fa3a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.880+0000 7fa3baa00700 1 -- 172.21.15.99:0/3824414327 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa3b0021070 con 0x7fa3c41608e0 2024-03-29T09:12:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.880+0000 7fa3c2800700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa3a406ac90 0x7fa3a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.881+0000 7fa3c2800700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa3a406ac90 0x7fa3a406d150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009b70 tx=0x7fa3b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:39.884 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:39.884+0000 7fa3baa00700 1 -- 172.21.15.99:0/3824414327 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa3b0055ed0 con 0x7fa3c41608e0 2024-03-29T09:12:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.091+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3c40d0700 con 0x7fa3a406ac90 2024-03-29T09:12:40.091 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.091+0000 7fa3baa00700 1 -- 172.21.15.99:0/3824414327 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa3c40d0700 con 0x7fa3a406ac90 2024-03-29T09:12:40.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa3a406ac90 msgr2=0x7fa3a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa3a406ac90 0x7fa3a406d150 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009b70 tx=0x7fa3b4008040 comp rx=0 tx=0).stop 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c41608e0 msgr2=0x7fa3c415f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c41608e0 0x7fa3c415f4a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009fd0 tx=0x7fa3b0004750 comp rx=0 tx=0).stop 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 shutdown_connections 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa3a406ac90 0x7fa3a406d150 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3c4160cc0 0x7fa3c415f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3c40d3a70 0x7fa3c4159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 --2- 172.21.15.99:0/3824414327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3c41608e0 0x7fa3c415f4a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.095 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 >> 172.21.15.99:0/3824414327 conn(0x7fa3c409c490 msgr2=0x7fa3c415e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 shutdown_connections 2024-03-29T09:12:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.094+0000 7fa3c9400700 1 -- 172.21.15.99:0/3824414327 wait complete. 2024-03-29T09:12:40.096 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 5 to deep-scrub 2024-03-29T09:12:40.112 INFO:tasks.ceph:Scrubbing osd.6 2024-03-29T09:12:40.114 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:40.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.307+0000 7f9766e00700 1 Processor -- start 2024-03-29T09:12:40.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.307+0000 7f9766e00700 1 -- start start 2024-03-29T09:12:40.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.308+0000 7f9766e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f97600d0260 0x7f97600cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.308+0000 7f9766e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600b3ba0 0x7f97600ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.308+0000 7f9766e00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97600d1fa0 con 0x7f9760160b30 2024-03-29T09:12:40.310 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.308+0000 7f9766e00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97600cf440 con 0x7f97600d0260 2024-03-29T09:12:40.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.308+0000 7f9766e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9760160f10 con 0x7f97600b3ba0 2024-03-29T09:12:40.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975fe00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f9760160b30 0x7f97600cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42556/0 (socket says 172.21.15.99:42556) 2024-03-29T09:12:40.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975fe00700 1 -- 172.21.15.99:0/3388466165 learned_addr learned my addr 172.21.15.99:0/3388466165 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:40.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975f400700 1 --2- 172.21.15.99:0/3388466165 >> v2:172.21.15.99:3300/0 conn(0x7f97600d0260 0x7f97600cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f9764e00700 1 --2- 172.21.15.99:0/3388466165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600b3ba0 0x7f97600ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975ea00700 1 -- 172.21.15.99:0/3388466165 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62053829 0 0) 0x7f97600d1fa0 con 0x7f9760160b30 2024-03-29T09:12:40.311 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975ea00700 1 -- 172.21.15.99:0/3388466165 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97440035b0 con 0x7f9760160b30 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975f400700 1 -- 172.21.15.99:0/3388466165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600b3ba0 msgr2=0x7f97600ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975f400700 1 --2- 172.21.15.99:0/3388466165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600b3ba0 0x7f97600ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975f400700 1 -- 172.21.15.99:0/3388466165 >> v1:172.21.15.99:6789/0 conn(0x7f9760160b30 legacy=0x7f97600cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975f400700 1 -- 172.21.15.99:0/3388466165 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97600d1670 con 0x7f97600d0260 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f9764e00700 1 --2- 172.21.15.99:0/3388466165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600b3ba0 0x7f97600ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.309+0000 7f975f400700 1 --2- 172.21.15.99:0/3388466165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97600d0260 0x7f97600cdd90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f974c009840 tx=0x7f974c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd6901cd7c6eac7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f975ea00700 1 -- 172.21.15.99:0/3388466165 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011830 con 0x7f97600d0260 2024-03-29T09:12:40.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f975ea00700 1 -- 172.21.15.99:0/3388466165 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c011990 con 0x7f97600d0260 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f975ea00700 1 -- 172.21.15.99:0/3388466165 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c011d20 con 0x7f97600d0260 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 -- 172.21.15.99:0/3388466165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97600d0260 msgr2=0x7f97600cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 --2- 172.21.15.99:0/3388466165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97600d0260 0x7f97600cdd90 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f974c009840 tx=0x7f974c009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 -- 172.21.15.99:0/3388466165 shutdown_connections 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 --2- 172.21.15.99:0/3388466165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600b3ba0 0x7f97600ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 --2- 172.21.15.99:0/3388466165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f97600d0260 0x7f97600cdd90 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 -- 172.21.15.99:0/3388466165 >> 172.21.15.99:0/3388466165 conn(0x7f976009c4d0 msgr2=0x7f97600d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 -- 172.21.15.99:0/3388466165 shutdown_connections 2024-03-29T09:12:40.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.310+0000 7f9766e00700 1 -- 172.21.15.99:0/3388466165 wait complete. 2024-03-29T09:12:40.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.312+0000 7f9766e00700 1 Processor -- start 2024-03-29T09:12:40.317 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.312+0000 7f9766e00700 1 -- start start 2024-03-29T09:12:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9766e00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97600b3ba0 0x7f97600cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9766e00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600d0260 0x7f97600b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9766e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9760160b30 0x7f976015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9766e00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97601f3da0 con 0x7f9760160b30 2024-03-29T09:12:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9766e00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600b6dc0 con 0x7f97600b3ba0 2024-03-29T09:12:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9766e00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97600cb980 con 0x7f97600d0260 2024-03-29T09:12:40.318 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9764e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9760160b30 0x7f976015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9764e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9760160b30 0x7f976015ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45034/0 (socket says 172.21.15.99:45034) 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.313+0000 7f9764e00700 1 -- 172.21.15.99:0/1718306911 learned_addr learned my addr 172.21.15.99:0/1718306911 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f975f400700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600d0260 0x7f97600b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f975fe00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97600b3ba0 0x7f97600cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9764e00700 1 -- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600d0260 msgr2=0x7f97600b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9764e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600d0260 0x7f97600b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9764e00700 1 -- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97600b3ba0 msgr2=0x7f97600cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.319 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9764e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97600b3ba0 0x7f97600cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9764e00700 1 -- 172.21.15.99:0/1718306911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c0094a0 con 0x7f9760160b30 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f975f400700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97600d0260 0x7f97600b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f975fe00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97600b3ba0 0x7f97600cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9764e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9760160b30 0x7f976015ca80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9758007f20 tx=0x7f975800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800de90 con 0x7f9760160b30 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975800f040 con 0x7f9760160b30 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758003c60 con 0x7f9760160b30 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.314+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97600cbc60 con 0x7f9760160b30 2024-03-29T09:12:40.320 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.315+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97600cc220 con 0x7f9760160b30 2024-03-29T09:12:40.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.315+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97600b48e0 con 0x7f9760160b30 2024-03-29T09:12:40.321 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.316+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f9758004030 con 0x7f9760160b30 2024-03-29T09:12:40.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.317+0000 7f9757e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f974006ac90 0x7f974006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.317+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f9758089370 con 0x7f9760160b30 2024-03-29T09:12:40.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.317+0000 7f975fe00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f974006ac90 0x7f974006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.322 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.318+0000 7f9757e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f9740070f20 0x7f9740073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.318+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 --> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9740073a50 con 0x7f9740070f20 2024-03-29T09:12:40.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.318+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f975808b950 con 0x7f9760160b30 2024-03-29T09:12:40.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.318+0000 7f975fe00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f974006ac90 0x7f974006d150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f97500011e0 tx=0x7f9750009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.318+0000 7f975f400700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f9740070f20 0x7f9740073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.319+0000 7f975f400700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f9740070f20 0x7f9740073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.323 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.320+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== osd.6 v2:172.21.15.133:6808/42490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f9740073a50 con 0x7f9740070f20 2024-03-29T09:12:40.334 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.334+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 --> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f97600d1c30 con 0x7f9740070f20 2024-03-29T09:12:40.335 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9757e00700 1 -- 172.21.15.99:0/1718306911 <== osd.6 v2:172.21.15.133:6808/42490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f974c021040 con 0x7f9740070f20 2024-03-29T09:12:40.335 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:40.335 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:40.335 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:40.336 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f9740070f20 msgr2=0x7f9740073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.133:6808/42490,v1:172.21.15.133:6809/42490] conn(0x7f9740070f20 0x7f9740073380 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f974006ac90 msgr2=0x7f974006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f974006ac90 0x7f974006d150 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f97500011e0 tx=0x7f9750009040 comp rx=0 tx=0).stop 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9760160b30 msgr2=0x7f976015ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 --2- 172.21.15.99:0/1718306911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9760160b30 0x7f976015ca80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f9758007f20 tx=0x7f975800d3b0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9764e00700 1 -- 172.21.15.99:0/1718306911 reap_dead start 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 shutdown_connections 2024-03-29T09:12:40.337 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.335+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 >> 172.21.15.99:0/1718306911 conn(0x7f976009c4d0 msgr2=0x7f976006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:40.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.336+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 shutdown_connections 2024-03-29T09:12:40.338 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.336+0000 7f9766e00700 1 -- 172.21.15.99:0/1718306911 wait complete. 2024-03-29T09:12:40.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-29T09:12:40.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.547+0000 7fa53a600700 1 Processor -- start 2024-03-29T09:12:40.547 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.547+0000 7fa53a600700 1 -- start start 2024-03-29T09:12:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa53a600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa534160b80 0x7fa5340b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.548 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa53a600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5340d1a90 0x7fa5340ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa53a600700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa534160f60 con 0x7fa5340b45e0 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa53a600700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5340cea60 con 0x7fa5340d1a90 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa53a600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340cde50 con 0x7fa534160b80 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa533e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5340d1a90 0x7fa5340ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa533e00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa5340d1a90 0x7fa5340ceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45048/0 (socket says 172.21.15.99:45048) 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa533e00700 1 -- 172.21.15.99:0/814159346 learned_addr learned my addr 172.21.15.99:0/814159346 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa533400700 1 --2- 172.21.15.99:0/814159346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa534160b80 0x7fa5340b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.549 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.548+0000 7fa532a00700 1 --1- 172.21.15.99:0/814159346 >> v1:172.21.15.133:6789/0 conn(0x7fa5340b45e0 0x7fa5340d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.99:47324/0 (socket says 172.21.15.99:47324) 2024-03-29T09:12:40.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa533e00700 1 -- 172.21.15.99:0/814159346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa534160b80 msgr2=0x7fa5340b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:40.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa533e00700 1 --2- 172.21.15.99:0/814159346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa534160b80 0x7fa5340b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa533e00700 1 -- 172.21.15.99:0/814159346 >> v1:172.21.15.133:6789/0 conn(0x7fa5340b45e0 legacy=0x7fa5340d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.550 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa533e00700 1 -- 172.21.15.99:0/814159346 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5340ce130 con 0x7fa5340d1a90 2024-03-29T09:12:40.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa533e00700 1 --2- 172.21.15.99:0/814159346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ceef0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009e50 tx=0x7fa52c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47ee6ebfcdc3a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.551 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa532000700 1 -- 172.21.15.99:0/814159346 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00e070 con 0x7fa5340d1a90 2024-03-29T09:12:40.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa532000700 1 -- 172.21.15.99:0/814159346 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c00b910 con 0x7fa5340d1a90 2024-03-29T09:12:40.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.549+0000 7fa532000700 1 -- 172.21.15.99:0/814159346 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c00bcc0 con 0x7fa5340d1a90 2024-03-29T09:12:40.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 -- 172.21.15.99:0/814159346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 msgr2=0x7fa5340ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 --2- 172.21.15.99:0/814159346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ceef0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009e50 tx=0x7fa52c00b330 comp rx=0 tx=0).stop 2024-03-29T09:12:40.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 -- 172.21.15.99:0/814159346 shutdown_connections 2024-03-29T09:12:40.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 --2- 172.21.15.99:0/814159346 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ceef0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.552 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 --2- 172.21.15.99:0/814159346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa534160b80 0x7fa5340b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 -- 172.21.15.99:0/814159346 >> 172.21.15.99:0/814159346 conn(0x7fa53409c480 msgr2=0x7fa5340d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:40.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 -- 172.21.15.99:0/814159346 shutdown_connections 2024-03-29T09:12:40.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.550+0000 7fa53a600700 1 -- 172.21.15.99:0/814159346 wait complete. 2024-03-29T09:12:40.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.552+0000 7fa53a600700 1 Processor -- start 2024-03-29T09:12:40.553 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.552+0000 7fa53a600700 1 -- start start 2024-03-29T09:12:40.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa53a600700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5340b45e0 0x7fa5341536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa53a600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa53a600700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534160b80 0x7fa5341f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.555 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa53a600700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534153c30 con 0x7fa5340d1a90 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa53a600700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340cbcd0 con 0x7fa534160b80 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa53a600700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340cac30 con 0x7fa5340b45e0 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45064/0 (socket says 172.21.15.99:45064) 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 -- 172.21.15.99:0/1327891597 learned_addr learned my addr 172.21.15.99:0/1327891597 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa533e00700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534160b80 0x7fa5341f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa533400700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5340b45e0 0x7fa5341536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 -- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5340b45e0 msgr2=0x7fa5341536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.556 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5340b45e0 0x7fa5341536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 -- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534160b80 msgr2=0x7fa5341f3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534160b80 0x7fa5341f3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa532a00700 1 -- 172.21.15.99:0/1327891597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c009b30 con 0x7fa5340d1a90 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa533e00700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534160b80 0x7fa5341f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.553+0000 7fa533400700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5340b45e0 0x7fa5341536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.554+0000 7fa532a00700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ca220 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c4d0 tx=0x7fa52000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.554+0000 7fa52b400700 1 -- 172.21.15.99:0/1327891597 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520005cb0 con 0x7fa5340d1a90 2024-03-29T09:12:40.557 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.554+0000 7fa52b400700 1 -- 172.21.15.99:0/1327891597 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa520005e10 con 0x7fa5340d1a90 2024-03-29T09:12:40.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.554+0000 7fa52b400700 1 -- 172.21.15.99:0/1327891597 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520007310 con 0x7fa5340d1a90 2024-03-29T09:12:40.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.554+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5340caf10 con 0x7fa5340d1a90 2024-03-29T09:12:40.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.554+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5340cb4d0 con 0x7fa5340d1a90 2024-03-29T09:12:40.558 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.555+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5340b2550 con 0x7fa5340d1a90 2024-03-29T09:12:40.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.556+0000 7fa52b400700 1 -- 172.21.15.99:0/1327891597 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa520007470 con 0x7fa5340d1a90 2024-03-29T09:12:40.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.557+0000 7fa52b400700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa51406ac90 0x7fa51406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.557+0000 7fa52b400700 1 -- 172.21.15.99:0/1327891597 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa520088680 con 0x7fa5340d1a90 2024-03-29T09:12:40.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.557+0000 7fa533400700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa51406ac90 0x7fa51406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.559 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.557+0000 7fa533400700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa51406ac90 0x7fa51406d150 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa534157720 tx=0x7fa524008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.560 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.561+0000 7fa52b400700 1 -- 172.21.15.99:0/1327891597 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fa520053980 con 0x7fa5340d1a90 2024-03-29T09:12:40.763 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.764+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5340ce6f0 con 0x7fa51406ac90 2024-03-29T09:12:40.764 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.764+0000 7fa52b400700 1 -- 172.21.15.99:0/1327891597 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa524004020 con 0x7fa51406ac90 2024-03-29T09:12:40.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa51406ac90 msgr2=0x7fa51406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa51406ac90 0x7fa51406d150 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa534157720 tx=0x7fa524008040 comp rx=0 tx=0).stop 2024-03-29T09:12:40.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 msgr2=0x7fa5340ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ca220 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa52000c4d0 tx=0x7fa52000c890 comp rx=0 tx=0).stop 2024-03-29T09:12:40.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 shutdown_connections 2024-03-29T09:12:40.767 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa51406ac90 0x7fa51406d150 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa534160b80 0x7fa5341f3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa5340b45e0 0x7fa5341536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 --2- 172.21.15.99:0/1327891597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa5340d1a90 0x7fa5340ca220 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.766+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 >> 172.21.15.99:0/1327891597 conn(0x7fa53409c480 msgr2=0x7fa5340cfdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:40.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.767+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 shutdown_connections 2024-03-29T09:12:40.768 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.767+0000 7fa53a600700 1 -- 172.21.15.99:0/1327891597 wait complete. 2024-03-29T09:12:40.769 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 6 to deep-scrub 2024-03-29T09:12:40.784 INFO:tasks.ceph:Scrubbing osd.7 2024-03-29T09:12:40.785 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-29T09:12:40.977 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.977+0000 7fa77ce00700 1 Processor -- start 2024-03-29T09:12:40.978 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.977+0000 7fa77ce00700 1 -- start start 2024-03-29T09:12:40.980 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.978+0000 7fa77ce00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa778160b30 0x7fa7780cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.978+0000 7fa77ce00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa7780b1f50 0x7fa7780d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.978+0000 7fa77ce00700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7780b2b90 0x7fa7780b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.978+0000 7fa77ce00700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7780cf440 con 0x7fa7780b2b90 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.978+0000 7fa77ce00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7780d1fa0 con 0x7fa7780b1f50 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.978+0000 7fa77ce00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa778160f10 con 0x7fa778160b30 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7780b2b90 0x7fa7780b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7fa7780b2b90 0x7fa7780b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45076/0 (socket says 172.21.15.99:45076) 2024-03-29T09:12:40.981 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 -- 172.21.15.99:0/20349727 learned_addr learned my addr 172.21.15.99:0/20349727 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:40.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa775e00700 1 --2- 172.21.15.99:0/20349727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa778160b30 0x7fa7780cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa775400700 1 --2- 172.21.15.99:0/20349727 >> v2:172.21.15.133:3300/0 conn(0x7fa7780b1f50 0x7fa7780d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 -- 172.21.15.99:0/20349727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa778160b30 msgr2=0x7fa7780cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:40.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 --2- 172.21.15.99:0/20349727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa778160b30 0x7fa7780cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 -- 172.21.15.99:0/20349727 >> v2:172.21.15.133:3300/0 conn(0x7fa7780b1f50 msgr2=0x7fa7780d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:40.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 --2- 172.21.15.99:0/20349727 >> v2:172.21.15.133:3300/0 conn(0x7fa7780b1f50 0x7fa7780d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.982 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 -- 172.21.15.99:0/20349727 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7780d1670 con 0x7fa7780b2b90 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.979+0000 7fa776800700 1 --2- 172.21.15.99:0/20349727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b3020 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa77808bf20 tx=0x7fa77000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b393496a47ea21 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa774a00700 1 -- 172.21.15.99:0/20349727 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000e070 con 0x7fa7780b2b90 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa774a00700 1 -- 172.21.15.99:0/20349727 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77000b9c0 con 0x7fa7780b2b90 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa774a00700 1 -- 172.21.15.99:0/20349727 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77000bd70 con 0x7fa7780b2b90 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 -- 172.21.15.99:0/20349727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 msgr2=0x7fa7780b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 --2- 172.21.15.99:0/20349727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b3020 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fa77808bf20 tx=0x7fa77000b330 comp rx=0 tx=0).stop 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 -- 172.21.15.99:0/20349727 shutdown_connections 2024-03-29T09:12:40.983 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 --2- 172.21.15.99:0/20349727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa778160b30 0x7fa7780cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 --2- 172.21.15.99:0/20349727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b3020 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 --2- 172.21.15.99:0/20349727 >> v2:172.21.15.133:3300/0 conn(0x7fa7780b1f50 0x7fa7780d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 -- 172.21.15.99:0/20349727 >> 172.21.15.99:0/20349727 conn(0x7fa77809c4d0 msgr2=0x7fa7780d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:40.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 -- 172.21.15.99:0/20349727 shutdown_connections 2024-03-29T09:12:40.984 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.980+0000 7fa77ce00700 1 -- 172.21.15.99:0/20349727 wait complete. 2024-03-29T09:12:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.982+0000 7fa77ce00700 1 Processor -- start 2024-03-29T09:12:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.982+0000 7fa77ce00700 1 -- start start 2024-03-29T09:12:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.983+0000 7fa77ce00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7780b1f50 0x7fa77814aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.983+0000 7fa77ce00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.983+0000 7fa77ce00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa778160b30 0x7fa7780b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.983+0000 7fa77ce00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780b8bf0 con 0x7fa7780b2b90 2024-03-29T09:12:40.987 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.983+0000 7fa77ce00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780b86a0 con 0x7fa778160b30 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.983+0000 7fa77ce00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7780ca6d0 con 0x7fa7780b1f50 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.983+0000 7fa775400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45082/0 (socket says 172.21.15.99:45082) 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 -- 172.21.15.99:0/369465207 learned_addr learned my addr 172.21.15.99:0/369465207 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775e00700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7780b1f50 0x7fa77814aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa776800700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa778160b30 0x7fa7780b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 -- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7780b1f50 msgr2=0x7fa77814aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.988 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa7780b1f50 0x7fa77814aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 -- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa778160b30 msgr2=0x7fa7780b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa778160b30 0x7fa7780b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 -- 172.21.15.99:0/369465207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa770009b30 con 0x7fa7780b2b90 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa775400700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b1040 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa76400e840 tx=0x7fa76400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400c070 con 0x7fa7780b2b90 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764013610 con 0x7fa7780b2b90 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7780b8910 con 0x7fa7780b2b90 2024-03-29T09:12:40.989 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.984+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640139a0 con 0x7fa7780b2b90 2024-03-29T09:12:40.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.985+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7780b0b20 con 0x7fa7780b2b90 2024-03-29T09:12:40.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.985+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7780b48e0 con 0x7fa7780b2b90 2024-03-29T09:12:40.990 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.986+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fa764013b00 con 0x7fa7780b2b90 2024-03-29T09:12:40.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa76e000700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa75806acc0 0x7fa75806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fa764088da0 con 0x7fa7780b2b90 2024-03-29T09:12:40.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa775e00700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa75806acc0 0x7fa75806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.991 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa76e000700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7fa758070f50 0x7fa7580733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:40.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 --> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa758073a80 con 0x7fa758070f50 2024-03-29T09:12:40.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa76408b380 con 0x7fa7780b2b90 2024-03-29T09:12:40.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa775e00700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa75806acc0 0x7fa75806d180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa768006b20 tx=0x7fa768008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.988+0000 7fa776800700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7fa758070f50 0x7fa7580733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:40.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.989+0000 7fa776800700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7fa758070f50 0x7fa7580733b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:40.992 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:40.990+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== osd.7 v2:172.21.15.133:6824/42500 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa758073a80 con 0x7fa758070f50 2024-03-29T09:12:41.005 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.005+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 --> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa7780d1c30 con 0x7fa758070f50 2024-03-29T09:12:41.006 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa76e000700 1 -- 172.21.15.99:0/369465207 <== osd.7 v2:172.21.15.133:6824/42500 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa7700099b0 con 0x7fa758070f50 2024-03-29T09:12:41.006 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-03-29T09:12:41.006 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T09:12:41.006 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-03-29T09:12:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7fa758070f50 msgr2=0x7fa7580733b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.007 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa77ce00700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.133:6824/42500,v1:172.21.15.133:6825/42500] conn(0x7fa758070f50 0x7fa7580733b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa75806acc0 msgr2=0x7fa75806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa77ce00700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fa75806acc0 0x7fa75806d180 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa768006b20 tx=0x7fa768008040 comp rx=0 tx=0).stop 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 msgr2=0x7fa7780b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa77ce00700 1 --2- 172.21.15.99:0/369465207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7780b2b90 0x7fa7780b1040 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa76400e840 tx=0x7fa76400eb50 comp rx=0 tx=0).stop 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.006+0000 7fa776800700 1 -- 172.21.15.99:0/369465207 reap_dead start 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.007+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 shutdown_connections 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.007+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 >> 172.21.15.99:0/369465207 conn(0x7fa77809c4d0 msgr2=0x7fa77806c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.007+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 shutdown_connections 2024-03-29T09:12:41.008 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.007+0000 7fa77ce00700 1 -- 172.21.15.99:0/369465207 wait complete. 2024-03-29T09:12:41.023 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-29T09:12:41.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.216+0000 7f78b8a00700 1 Processor -- start 2024-03-29T09:12:41.216 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.216+0000 7f78b8a00700 1 -- start start 2024-03-29T09:12:41.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.217+0000 7f78b8a00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f78b4160cc0 0x7f78b40d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.217+0000 7f78b8a00700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b40cddc0 con 0x7f78b41608e0 2024-03-29T09:12:41.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.217+0000 7f78b8a00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b40ce0a0 con 0x7f78b40d1b20 2024-03-29T09:12:41.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.217+0000 7f78b8a00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b40ce380 con 0x7f78b4160cc0 2024-03-29T09:12:41.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b1e00700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7f78b41608e0 0x7f78b40d1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42562/0 (socket says 172.21.15.99:42562) 2024-03-29T09:12:41.218 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b1e00700 1 -- 172.21.15.99:0/4132319915 learned_addr learned my addr 172.21.15.99:0/4132319915 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:41.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427721413 0 0) 0x7f78b40cddc0 con 0x7f78b41608e0 2024-03-29T09:12:41.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b2800700 1 --2- 172.21.15.99:0/4132319915 >> v2:172.21.15.133:3300/0 conn(0x7f78b4160cc0 0x7f78b40d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78980035b0 con 0x7f78b41608e0 2024-03-29T09:12:41.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 <== mon.0 v1:172.21.15.99:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 643245729 0 0) 0x7f78980035b0 con 0x7f78b41608e0 2024-03-29T09:12:41.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 >> v2:172.21.15.133:3300/0 conn(0x7f78b4160cc0 msgr2=0x7f78b40d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:41.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 --2- 172.21.15.99:0/4132319915 >> v2:172.21.15.133:3300/0 conn(0x7f78b4160cc0 0x7f78b40d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 >> v1:172.21.15.133:6789/0 conn(0x7f78b40d1b20 legacy=0x7f78b40afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 --> v1:172.21.15.99:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40ce660 con 0x7f78b41608e0 2024-03-29T09:12:41.222 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 <== mon.0 v1:172.21.15.99:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2896864688 0 0) 0x7f78a4001670 con 0x7f78b41608e0 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 >> v1:172.21.15.99:6789/0 conn(0x7f78b41608e0 legacy=0x7f78b40d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.218+0000 7f78b0a00700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7898003a00 0x7f7898003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b0a00700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7898004850 0x7f7898004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b0a00700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f789800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40ce380 con 0x7f7898005210 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789800c0c0 con 0x7f7898004850 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789800c280 con 0x7f7898003a00 2024-03-29T09:12:41.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f789800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b1e00700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7898003a00 0x7f7898003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b1400700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7898004850 0x7f7898004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 -- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7898003a00 msgr2=0x7f7898003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7898003a00 0x7f7898003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 -- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7898004850 msgr2=0x7f7898004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7898004850 0x7f7898004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 -- 172.21.15.99:0/4132319915 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980035b0 con 0x7f7898005210 2024-03-29T09:12:41.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b1400700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7898004850 0x7f7898004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 -- 172.21.15.99:0/4132319915 reap_dead start 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.219+0000 7f78b2800700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f789800ba10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009fc0 tx=0x7f7898005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5a29e441816cd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003a20 con 0x7f7898005210 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac00f040 con 0x7f7898005210 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b0a00700 1 -- 172.21.15.99:0/4132319915 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac003da0 con 0x7f7898005210 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b8a00700 1 -- 172.21.15.99:0/4132319915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 msgr2=0x7f789800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b8a00700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f789800ba10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009fc0 tx=0x7f7898005880 comp rx=0 tx=0).stop 2024-03-29T09:12:41.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b8a00700 1 -- 172.21.15.99:0/4132319915 shutdown_connections 2024-03-29T09:12:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b8a00700 1 --2- 172.21.15.99:0/4132319915 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f789800ba10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b8a00700 1 -- 172.21.15.99:0/4132319915 >> 172.21.15.99:0/4132319915 conn(0x7f78b409c490 msgr2=0x7f78b40d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b8a00700 1 -- 172.21.15.99:0/4132319915 shutdown_connections 2024-03-29T09:12:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.220+0000 7f78b8a00700 1 -- 172.21.15.99:0/4132319915 wait complete. 2024-03-29T09:12:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.222+0000 7f78b8a00700 1 Processor -- start 2024-03-29T09:12:41.226 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.222+0000 7f78b8a00700 1 -- start start 2024-03-29T09:12:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b8a00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f78b4153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b8a00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78b4154a00 0x7f78b4154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b8a00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b40b3470 0x7f78b40b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.229 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b8a00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40b3e60 con 0x7f7898005210 2024-03-29T09:12:41.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b8a00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40cc060 con 0x7f78b40b3470 2024-03-29T09:12:41.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b8a00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b41554a0 con 0x7f78b4154a00 2024-03-29T09:12:41.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f78b4153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b1e00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f78b4153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45100/0 (socket says 172.21.15.99:45100) 2024-03-29T09:12:41.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.223+0000 7f78b1e00700 1 -- 172.21.15.99:0/971623590 learned_addr learned my addr 172.21.15.99:0/971623590 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:41.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b2800700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b40b3470 0x7f78b40b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.230 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b1400700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78b4154a00 0x7f78b4154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b1e00700 1 -- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78b4154a00 msgr2=0x7f78b4154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b1e00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78b4154a00 0x7f78b4154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b1e00700 1 -- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b40b3470 msgr2=0x7f78b40b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b1e00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b40b3470 0x7f78b40b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b1e00700 1 -- 172.21.15.99:0/971623590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40cb730 con 0x7f7898005210 2024-03-29T09:12:41.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b2800700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b40b3470 0x7f78b40b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:41.231 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b1e00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f78b4153580 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f78a4007290 tx=0x7f78a40075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78aa000700 1 -- 172.21.15.99:0/971623590 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400f660 con 0x7f7898005210 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78aa000700 1 -- 172.21.15.99:0/971623590 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a400f7c0 con 0x7f7898005210 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78aa000700 1 -- 172.21.15.99:0/971623590 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400fb70 con 0x7f7898005210 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b40cba10 con 0x7f7898005210 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.224+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b4157650 con 0x7f7898005210 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.225+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b40677a0 con 0x7f7898005210 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.226+0000 7f78aa000700 1 -- 172.21.15.99:0/971623590 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f78a400fcd0 con 0x7f7898005210 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.227+0000 7f78aa000700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f789406acc0 0x7f789406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.232 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.227+0000 7f78aa000700 1 -- 172.21.15.99:0/971623590 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f78a4089110 con 0x7f7898005210 2024-03-29T09:12:41.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.227+0000 7f78b1400700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f789406acc0 0x7f789406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.228+0000 7f78b1400700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f789406acc0 0x7f789406d180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f78a0000c00 tx=0x7f78a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:41.233 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.231+0000 7f78aa000700 1 -- 172.21.15.99:0/971623590 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f78a40544b0 con 0x7f7898005210 2024-03-29T09:12:41.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.435+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f78b40b22a0 con 0x7f789406acc0 2024-03-29T09:12:41.435 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.435+0000 7f78aa000700 1 -- 172.21.15.99:0/971623590 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f78a0005170 con 0x7f789406acc0 2024-03-29T09:12:41.437 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.437+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f789406acc0 msgr2=0x7f789406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.437+0000 7f78b8a00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f789406acc0 0x7f789406d180 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f78a0000c00 tx=0x7f78a0009040 comp rx=0 tx=0).stop 2024-03-29T09:12:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.437+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 msgr2=0x7f78b4153580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f78b4153580 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f78a4007290 tx=0x7f78a40075a0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 shutdown_connections 2024-03-29T09:12:41.438 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f789406acc0 0x7f789406d180 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78b40b3470 0x7f78b40b3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7898005210 0x7f78b4153580 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 --2- 172.21.15.99:0/971623590 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78b4154a00 0x7f78b4154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 >> 172.21.15.99:0/971623590 conn(0x7f78b409c490 msgr2=0x7f78b40b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 shutdown_connections 2024-03-29T09:12:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.438+0000 7f78b8a00700 1 -- 172.21.15.99:0/971623590 wait complete. 2024-03-29T09:12:41.439 INFO:teuthology.orchestra.run.smithi099.stderr:instructed osd(s) 7 to deep-scrub 2024-03-29T09:12:41.456 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:12:41.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.647+0000 7fb748400700 1 Processor -- start 2024-03-29T09:12:41.647 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.647+0000 7fb748400700 1 -- start start 2024-03-29T09:12:41.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.648+0000 7fb748400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb74015d690 0x7fb7400cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.648+0000 7fb748400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7400d1740 0x7fb7400d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.648 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.648+0000 7fb748400700 1 -- --> v1:172.21.15.99:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400afde0 con 0x7fb74015da70 2024-03-29T09:12:41.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.648+0000 7fb748400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400d2130 con 0x7fb7400d1740 2024-03-29T09:12:41.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.648+0000 7fb748400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400d01d0 con 0x7fb74015d690 2024-03-29T09:12:41.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.648+0000 7fb745000700 1 --1- >> v1:172.21.15.99:6789/0 conn(0x7fb74015da70 0x7fb7400cf120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.99:6789/0 says I am v1:172.21.15.99:42574/0 (socket says 172.21.15.99:42574) 2024-03-29T09:12:41.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.648+0000 7fb745000700 1 -- 172.21.15.99:0/856136955 learned_addr learned my addr 172.21.15.99:0/856136955 (peer_addr_for_me v1:172.21.15.99:0/0) 2024-03-29T09:12:41.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb745a00700 1 --2- 172.21.15.99:0/856136955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb74015d690 0x7fb7400cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.649 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb746400700 1 --2- 172.21.15.99:0/856136955 >> v2:172.21.15.133:3300/0 conn(0x7fb7400d1740 0x7fb7400d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb73fe00700 1 -- 172.21.15.99:0/856136955 <== mon.0 v1:172.21.15.99:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26194671 0 0) 0x7fb7400afde0 con 0x7fb74015da70 2024-03-29T09:12:41.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb73fe00700 1 -- 172.21.15.99:0/856136955 --> v1:172.21.15.99:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7280035b0 con 0x7fb74015da70 2024-03-29T09:12:41.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb745a00700 1 -- 172.21.15.99:0/856136955 >> v2:172.21.15.133:3300/0 conn(0x7fb7400d1740 msgr2=0x7fb7400d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:12:41.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb745a00700 1 --2- 172.21.15.99:0/856136955 >> v2:172.21.15.133:3300/0 conn(0x7fb7400d1740 0x7fb7400d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.651 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb745a00700 1 -- 172.21.15.99:0/856136955 >> v1:172.21.15.99:6789/0 conn(0x7fb74015da70 legacy=0x7fb7400cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb745a00700 1 -- 172.21.15.99:0/856136955 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400d04b0 con 0x7fb74015d690 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb746400700 1 --2- 172.21.15.99:0/856136955 >> v2:172.21.15.133:3300/0 conn(0x7fb7400d1740 0x7fb7400d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.649+0000 7fb745a00700 1 --2- 172.21.15.99:0/856136955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb74015d690 0x7fb7400cfb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb734009840 tx=0x7fb734009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aaff7db42b45b92 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb73fe00700 1 -- 172.21.15.99:0/856136955 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734011830 con 0x7fb74015d690 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb73fe00700 1 -- 172.21.15.99:0/856136955 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb734011990 con 0x7fb74015d690 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb73fe00700 1 -- 172.21.15.99:0/856136955 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb734011d20 con 0x7fb74015d690 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb748400700 1 -- 172.21.15.99:0/856136955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb74015d690 msgr2=0x7fb7400cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.652 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb748400700 1 --2- 172.21.15.99:0/856136955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb74015d690 0x7fb7400cfb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb734009840 tx=0x7fb734009b50 comp rx=0 tx=0).stop 2024-03-29T09:12:41.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb748400700 1 -- 172.21.15.99:0/856136955 shutdown_connections 2024-03-29T09:12:41.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb748400700 1 --2- 172.21.15.99:0/856136955 >> v2:172.21.15.133:3300/0 conn(0x7fb7400d1740 0x7fb7400d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb748400700 1 --2- 172.21.15.99:0/856136955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb74015d690 0x7fb7400cfb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.650+0000 7fb748400700 1 -- 172.21.15.99:0/856136955 >> 172.21.15.99:0/856136955 conn(0x7fb74009c480 msgr2=0x7fb7400d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:41.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.651+0000 7fb748400700 1 -- 172.21.15.99:0/856136955 shutdown_connections 2024-03-29T09:12:41.653 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.651+0000 7fb748400700 1 -- 172.21.15.99:0/856136955 wait complete. 2024-03-29T09:12:41.655 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.653+0000 7fb748400700 1 Processor -- start 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.653+0000 7fb748400700 1 -- start start 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.654+0000 7fb748400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7400d1740 0x7fb74014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.654+0000 7fb748400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74015d690 0x7fb74006be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.654+0000 7fb748400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74015da70 0x7fb74006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.654+0000 7fb748400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b8be0 con 0x7fb74015d690 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.654+0000 7fb748400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b7b80 con 0x7fb74015da70 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.654+0000 7fb748400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b7e20 con 0x7fb7400d1740 2024-03-29T09:12:41.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74015d690 0x7fb74006be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74015d690 0x7fb74006be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:45110/0 (socket says 172.21.15.99:45110) 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 -- 172.21.15.99:0/636261106 learned_addr learned my addr 172.21.15.99:0/636261106 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745a00700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7400d1740 0x7fb74014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb746400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74015da70 0x7fb74006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 -- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7400d1740 msgr2=0x7fb74014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7400d1740 0x7fb74014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 -- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74015da70 msgr2=0x7fb74006c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74015da70 0x7fb74006c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 -- 172.21.15.99:0/636261106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb738009a70 con 0x7fb74015d690 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb746400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74015da70 0x7fb74006c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb745000700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74015d690 0x7fb74006be20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb72c001eb0 tx=0x7fb72c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb73d600700 1 -- 172.21.15.99:0/636261106 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c005d70 con 0x7fb74015d690 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb73d600700 1 -- 172.21.15.99:0/636261106 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c0070d0 con 0x7fb74015d690 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb73d600700 1 -- 172.21.15.99:0/636261106 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c007460 con 0x7fb74015d690 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.655+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7340094a0 con 0x7fb74015d690 2024-03-29T09:12:41.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.656+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7400b83e0 con 0x7fb74015d690 2024-03-29T09:12:41.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.657+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7400677a0 con 0x7fb74015d690 2024-03-29T09:12:41.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.658+0000 7fb73d600700 1 -- 172.21.15.99:0/636261106 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7fb72c0075c0 con 0x7fb74015d690 2024-03-29T09:12:41.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.658+0000 7fb73d600700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72006ac90 0x7fb72006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:12:41.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.658+0000 7fb73d600700 1 -- 172.21.15.99:0/636261106 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7fb72c0885f0 con 0x7fb74015d690 2024-03-29T09:12:41.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.659+0000 7fb745a00700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72006ac90 0x7fb72006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:12:41.660 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.659+0000 7fb745a00700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72006ac90 0x7fb72006d150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb734000c00 tx=0x7fb734008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:12:41.662 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.663+0000 7fb73d600700 1 -- 172.21.15.99:0/636261106 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7fb72c053980 con 0x7fb74015d690 2024-03-29T09:12:41.863 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.863+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7400b0310 con 0x7fb72006ac90 2024-03-29T09:12:41.865 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.865+0000 7fb73d600700 1 -- 172.21.15.99:0/636261106 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50605 (secure 0 0 0) 0x7fb734011630 con 0x7fb72006ac90 2024-03-29T09:12:41.866 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:12:41.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72006ac90 msgr2=0x7fb72006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72006ac90 0x7fb72006d150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb734000c00 tx=0x7fb734008030 comp rx=0 tx=0).stop 2024-03-29T09:12:41.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74015d690 msgr2=0x7fb74006be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:12:41.868 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74015d690 0x7fb74006be20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb72c001eb0 tx=0x7fb72c00c7b0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 shutdown_connections 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7fb72006ac90 0x7fb72006d150 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb74015da70 0x7fb74006c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb7400d1740 0x7fb74014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 --2- 172.21.15.99:0/636261106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb74015d690 0x7fb74006be20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 >> 172.21.15.99:0/636261106 conn(0x7fb74009c480 msgr2=0x7fb74006b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 shutdown_connections 2024-03-29T09:12:41.869 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:12:41.868+0000 7fb748400700 1 -- 172.21.15.99:0/636261106 wait complete. 2024-03-29T09:12:41.870 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:12:41.886 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-03-29T09:12:40.497798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222856,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751864,"statfs":{"total":773094113280,"available":772865908736,"internally_reserved":0,"allocated":6823936,"data_stored":3680592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001938"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-03-29T09:12:30.082749+0000","last_change":"2024-03-29T09:08:47.900244+0000","last_active":"2024-03-29T09:12:30.082749+0000","last_peered":"2024-03-29T09:12:30.082749+0000","last_clean":"2024-03-29T09:12:30.082749+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:12:30.082749+0000","last_undegraded":"2024-03-29T09:12:30.082749+0000","last_fullsized":"2024-03-29T09:12:30.082749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T09:08:46.879940+0000","last_clean_scrub_stamp":"2024-03-29T09:08:46.879940+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:26.640662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67600000000000005,"15min":0.71399999999999997},"min":{"1min":0.374,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85999999999999999,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61699999999999999,"15min":0.61099999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.77700000000000002,"5min":1.639,"15min":1.639},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":0.86399999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.77600000000000002,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.71699999999999997,"15min":0.76400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.72499999999999998,"5min":1.992,"15min":1.992},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.75900000000000001,"15min":0.81000000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82899999999999996,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.81899999999999995,"15min":0.86899999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85099999999999998,"5min":2.883,"15min":2.883},"last":0.81399999999999995}]},{"osd":3,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68799999999999994,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72799999999999998,"15min":0.748},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.87,"5min":1.387,"15min":1.387},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.68500000000000005,"15min":0.69799999999999995},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.89200000000000002,"5min":1.367,"15min":1.367},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.499,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.458,"15min":1.458},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.88700000000000001,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.753,"15min":0.77200000000000002},"min":{"1min":0.56000000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85599999999999998,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.92200000000000004,"5min":1.413,"15min":1.413},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.438,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82299999999999995,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.60199999999999998},"min":{"1min":0.47799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.65100000000000002,"15min":0.63100000000000001},"min":{"1min":0.53600000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.75900000000000001,"15min":0.81200000000000006},"min":{"1min":0.49099999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.84799999999999998,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.44600000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.85899999999999999,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.77600000000000002,"15min":0.82599999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.91000000000000003,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89500000000000002,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91800000000000004,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Mar 29 09:11:55 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.78200000000000003,"15min":0.79800000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.90100000000000002,"5min":1.026,"15min":1.026},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94499999999999995,"5min":1.111,"15min":1.111},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":902850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.55300000000000005,"15min":0.61599999999999999},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.755,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.52700000000000002,"15min":0.55100000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67200000000000004,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.40699999999999997}]},{"osd":2,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.59999999999999998,"15min":0.66600000000000004},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72199999999999998,"5min":1,"15min":1},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.52900000000000003}]},{"osd":4,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.96199999999999997,"15min":0.93400000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94799999999999995,"5min":12.356,"15min":12.356},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94999999999999996,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.018,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":0.92000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.034,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.80200000000000005,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91800000000000004,"5min":1.99,"15min":1.99},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":1.0109999999999999,"15min":0.97699999999999998},"min":{"1min":0.57799999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99199999999999999,"5min":12.669,"15min":12.669},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.97899999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84999999999999998,"15min":0.872},"min":{"1min":0.628,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97499999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.69199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57699999999999996,"5min":0.751,"15min":0.751},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59899999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.61799999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45300000000000001,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.54100000000000004,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.61499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.52100000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.022,"5min":1.093,"15min":1.093},"last":0.97099999999999997}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.84099999999999997}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.147,"15min":1.147},"last":1.012},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96699999999999997,"5min":1.069,"15min":1.069},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85099999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.755},"min":{"1min":0.58199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97899999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.92100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.45700000000000002,"15min":0.48199999999999998},"min":{"1min":0.25600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.59399999999999997,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.49099999999999999,"15min":0.52300000000000002},"min":{"1min":0.317,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.55400000000000005,"5min":1.252,"15min":1.252},"last":0.26300000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.51600000000000001,"15min":0.55500000000000005},"min":{"1min":0.31,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.53800000000000003,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.318},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.58299999999999996,"15min":0.629},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.57799999999999996,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.56799999999999995,"15min":0.61799999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55000000000000004,"5min":2.415,"15min":2.415},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.625,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.66600000000000004,"5min":2.056,"15min":2.056},"last":0.40000000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84099999999999997,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.77500000000000002,"15min":0.81699999999999995},"min":{"1min":0.50600000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88900000000000001,"5min":2.073,"15min":2.073},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.81000000000000005,"15min":0.85399999999999998},"min":{"1min":0.58299999999999996,"5min":0.496,"15min":0.496},"max":{"1min":0.87,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.80500000000000005},"min":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89600000000000002,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.83599999999999997,"15min":0.88900000000000001},"min":{"1min":0.57099999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.92000000000000004,"5min":2.387,"15min":2.387},"last":0.62},{"interface":"front","average":{"1min":0.746,"5min":0.873,"15min":0.93300000000000005},"min":{"1min":0.60599999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.90200000000000002,"5min":2.996,"15min":2.996},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.86099999999999999},"min":{"1min":0.51700000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90700000000000003,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.81699999999999995,"15min":0.86299999999999999},"min":{"1min":0.53600000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.89000000000000001,"5min":2.165,"15min":2.165},"last":0.53000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.498,"15min":0.54000000000000004},"min":{"1min":0.29099999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53000000000000003,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.52300000000000002,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.61499999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.502,"15min":0.54200000000000004},"min":{"1min":0.32300000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57499999999999996,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.52200000000000002,"15min":0.55900000000000005},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.57799999999999996,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.45300000000000001}]},{"osd":2,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.55000000000000004,"15min":0.59299999999999997},"min":{"1min":0.311,"5min":0.307,"15min":0.307},"max":{"1min":0.58999999999999997,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.65200000000000002},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.58899999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93300000000000005,"5min":2.278,"15min":2.278},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.84899999999999998,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.95299999999999996,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96599999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.77600000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83099999999999996,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.78900000000000003,"15min":0.81699999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96599999999999997,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.83999999999999997,"15min":0.88700000000000001},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.014,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.62}]},{"osd":7,"last update":"Fri Mar 29 09:11:52 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.84899999999999998,"15min":0.89600000000000002},"min":{"1min":0.503,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.89000000000000001,"5min":2.649,"15min":2.649},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.86199999999999999,"15min":0.90200000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.024,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.65000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69399999999999995,"15min":0.70299999999999996},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.45900000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.83099999999999996,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.66900000000000004},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.91000000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.67700000000000005,"15min":0.70699999999999996},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.51700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.77600000000000002,"15min":0.80400000000000005},"min":{"1min":0.60899999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0089999999999999,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.66700000000000004,"15min":0.66800000000000004},"min":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95499999999999996,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.751,"15min":0.80500000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.80100000000000005,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80400000000000005,"15min":0.83799999999999997},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.056,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98699999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.82999999999999996,"15min":0.88700000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93600000000000005,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.90400000000000003,"15min":0.96199999999999997},"min":{"1min":0.623,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.0289999999999999,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.71999999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1298432,"data_stored":902850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.63100000000000001,"15min":0.64700000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.442,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.02,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.64000000000000001,"15min":0.64700000000000002},"min":{"1min":0.47599999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66800000000000004,"15min":0.68000000000000005},"min":{"1min":0.45100000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71299999999999997,"15min":0.73299999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.98499999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.495,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.86099999999999999,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.748,"15min":0.77800000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0780000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.621},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.57799999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.67800000000000005}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64600000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91200000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0329999999999999,"5min":1.05,"15min":1.05},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69499999999999995,"15min":0.69099999999999995},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.96899999999999997,"5min":1.079,"15min":1.079},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0580000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.503},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.56000000000000005,"5min":0.497,"15min":0.497},"max":{"1min":1.022,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.63200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:12:41.890 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-29T09:08:46.879940+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=9, tm_min=8, tm_sec=46, tm_wday=4, tm_yday=89, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=9, tm_min=12, tm_sec=35, tm_wday=4, tm_yday=89, tm_isdst=0) 2024-03-29T09:12:41.890 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-29T09:12:53.414 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:12:53.420 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:13:01.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T09:13:02.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.094+0000 7f4148400700 1 Processor -- start 2024-03-29T09:13:02.094 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.094+0000 7f4148400700 1 -- start start 2024-03-29T09:13:02.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.095+0000 7f4148400700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4140145d70 0x7f4140161a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.096 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.095+0000 7f4148400700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4140146150 0x7f41400b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.095+0000 7f4148400700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41400b3990 con 0x7f41400b3480 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.095+0000 7f4148400700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41400b3c70 con 0x7f4140146150 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.095+0000 7f4148400700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4140160f40 con 0x7f4140145d70 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4140146150 0x7f41400b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f4140146150 0x7f41400b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47638/0 (socket says 172.21.15.99:47638) 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 -- 172.21.15.99:0/1109679839 learned_addr learned my addr 172.21.15.99:0/1109679839 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145a00700 1 --2- 172.21.15.99:0/1109679839 >> v2:172.21.15.133:3300/0 conn(0x7f4140145d70 0x7f4140161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.097 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 -- 172.21.15.99:0/1109679839 >> v2:172.21.15.133:3300/0 conn(0x7f4140145d70 msgr2=0x7f4140161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:13:02.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 --2- 172.21.15.99:0/1109679839 >> v2:172.21.15.133:3300/0 conn(0x7f4140145d70 0x7f4140161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 -- 172.21.15.99:0/1109679839 >> v1:172.21.15.133:6789/0 conn(0x7f41400b3480 legacy=0x7f414006a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 -- 172.21.15.99:0/1109679839 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4140161220 con 0x7f4140146150 2024-03-29T09:13:02.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.096+0000 7f4145000700 1 --2- 172.21.15.99:0/1109679839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140146150 0x7f41400b2f40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f41400d0fa0 tx=0x7f412c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0c750adb80d161 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f413fe00700 1 -- 172.21.15.99:0/1109679839 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c00b870 con 0x7f4140146150 2024-03-29T09:13:02.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f413fe00700 1 -- 172.21.15.99:0/1109679839 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c00b9d0 con 0x7f4140146150 2024-03-29T09:13:02.098 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f413fe00700 1 -- 172.21.15.99:0/1109679839 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c00bd60 con 0x7f4140146150 2024-03-29T09:13:02.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 -- 172.21.15.99:0/1109679839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140146150 msgr2=0x7f41400b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 --2- 172.21.15.99:0/1109679839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140146150 0x7f41400b2f40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f41400d0fa0 tx=0x7f412c009bd0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.101 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 -- 172.21.15.99:0/1109679839 shutdown_connections 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 --2- 172.21.15.99:0/1109679839 >> v2:172.21.15.133:3300/0 conn(0x7f4140145d70 0x7f4140161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 --2- 172.21.15.99:0/1109679839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140146150 0x7f41400b2f40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 -- 172.21.15.99:0/1109679839 >> 172.21.15.99:0/1109679839 conn(0x7f4140066df0 msgr2=0x7f414006d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 -- 172.21.15.99:0/1109679839 shutdown_connections 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.097+0000 7f4148400700 1 -- 172.21.15.99:0/1109679839 wait complete. 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.099+0000 7f4148400700 1 Processor -- start 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.099+0000 7f4148400700 1 -- start start 2024-03-29T09:13:02.102 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4148400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41400b3480 0x7f414015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4148400700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140145d70 0x7f41400afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4148400700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41400b0530 0x7f41400cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4148400700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400b0990 con 0x7f4140145d70 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4148400700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400ae0e0 con 0x7f41400b0530 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4148400700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400acbb0 con 0x7f41400b3480 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4145000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140145d70 0x7f41400afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4145000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140145d70 0x7f41400afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47648/0 (socket says 172.21.15.99:47648) 2024-03-29T09:13:02.103 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4145000700 1 -- 172.21.15.99:0/84626530 learned_addr learned my addr 172.21.15.99:0/84626530 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4146400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41400b0530 0x7f41400cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.100+0000 7f4145a00700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41400b3480 0x7f414015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4145000700 1 -- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41400b3480 msgr2=0x7f414015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4145000700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41400b3480 0x7f414015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4145000700 1 -- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41400b0530 msgr2=0x7f41400cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4145000700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41400b0530 0x7f41400cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4145000700 1 -- 172.21.15.99:0/84626530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c009550 con 0x7f4140145d70 2024-03-29T09:13:02.104 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4146400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41400b0530 0x7f41400cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:13:02.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4145000700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140145d70 0x7f41400afff0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f412c000c00 tx=0x7f412c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f413d600700 1 -- 172.21.15.99:0/84626530 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c004520 con 0x7f4140145d70 2024-03-29T09:13:02.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f413d600700 1 -- 172.21.15.99:0/84626530 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c01e070 con 0x7f4140145d70 2024-03-29T09:13:02.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41400b0b70 con 0x7f4140145d70 2024-03-29T09:13:02.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f413d600700 1 -- 172.21.15.99:0/84626530 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0038a0 con 0x7f4140145d70 2024-03-29T09:13:02.105 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.101+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41400b1130 con 0x7f4140145d70 2024-03-29T09:13:02.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.102+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41400ae270 con 0x7f4140145d70 2024-03-29T09:13:02.106 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.103+0000 7f413d600700 1 -- 172.21.15.99:0/84626530 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f412c003a00 con 0x7f4140145d70 2024-03-29T09:13:02.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.104+0000 7f413d600700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f412006ac90 0x7f412006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.104+0000 7f413d600700 1 -- 172.21.15.99:0/84626530 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f412c020070 con 0x7f4140145d70 2024-03-29T09:13:02.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.104+0000 7f4145a00700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f412006ac90 0x7f412006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.107 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.105+0000 7f4145a00700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f412006ac90 0x7f412006d150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4134009930 tx=0x7f4134008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.108 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.109+0000 7f413d600700 1 -- 172.21.15.99:0/84626530 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f412c055ad0 con 0x7f4140145d70 2024-03-29T09:13:02.307 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.307+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 --> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f414006aee0 con 0x7f412006ac90 2024-03-29T09:13:02.309 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.310+0000 7f413d600700 1 -- 172.21.15.99:0/84626530 <== mgr.4104 v2:172.21.15.99:6832/41898 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50567 (secure 0 0 0) 0x7f4134009e00 con 0x7f412006ac90 2024-03-29T09:13:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:13:02.312 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f412006ac90 msgr2=0x7f412006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f412006ac90 0x7f412006d150 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4134009930 tx=0x7f4134008040 comp rx=0 tx=0).stop 2024-03-29T09:13:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140145d70 msgr2=0x7f41400afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140145d70 0x7f41400afff0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f412c000c00 tx=0x7f412c004750 comp rx=0 tx=0).stop 2024-03-29T09:13:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 shutdown_connections 2024-03-29T09:13:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f412006ac90 0x7f412006d150 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41400b0530 0x7f41400cfe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.313 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41400b3480 0x7f414015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 --2- 172.21.15.99:0/84626530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4140145d70 0x7f41400afff0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.312+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 >> 172.21.15.99:0/84626530 conn(0x7f4140066df0 msgr2=0x7f4140144410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:13:02.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.313+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 shutdown_connections 2024-03-29T09:13:02.314 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.313+0000 7f4148400700 1 -- 172.21.15.99:0/84626530 wait complete. 2024-03-29T09:13:02.314 INFO:teuthology.orchestra.run.smithi099.stderr:dumped all 2024-03-29T09:13:02.331 INFO:teuthology.orchestra.run.smithi099.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-03-29T09:13:00.501906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222840,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751880,"statfs":{"total":773094113280,"available":772865925120,"internally_reserved":0,"allocated":6823936,"data_stored":3680592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":221236792},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002680"},"pg_stats":[{"pgid":"1.0","version":"58'119","reported_seq":214,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-03-29T09:12:37.890672+0000","last_change":"2024-03-29T09:12:37.890672+0000","last_active":"2024-03-29T09:12:37.890672+0000","last_peered":"2024-03-29T09:12:37.890672+0000","last_clean":"2024-03-29T09:12:37.890672+0000","last_became_active":"2024-03-29T09:08:47.898931+0000","last_became_peered":"2024-03-29T09:08:47.898931+0000","last_unstale":"2024-03-29T09:12:37.890672+0000","last_undegraded":"2024-03-29T09:12:37.890672+0000","last_fullsized":"2024-03-29T09:12:37.890672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"58'119","last_scrub_stamp":"2024-03-29T09:12:37.890560+0000","last_deep_scrub":"58'119","last_deep_scrub_stamp":"2024-03-29T09:12:37.890560+0000","last_clean_scrub_stamp":"2024-03-29T09:12:37.890560+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:32:11.220269+0000","scrub_duration":0.0057734559999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:12:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.65100000000000002,"15min":0.70499999999999996},"min":{"1min":0.39700000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84199999999999997,"5min":2.8380000000000001,"15min":2.8380000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.622,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91700000000000004,"5min":1.639,"15min":1.639},"last":0.71899999999999997}]},{"osd":1,"last update":"Fri Mar 29 09:12:52 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76300000000000001,"15min":0.85199999999999998},"min":{"1min":0.47199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97199999999999998,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.75600000000000001},"min":{"1min":0.52000000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89500000000000002,"5min":1.992,"15min":1.992},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:12:52 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.73099999999999998,"15min":0.80100000000000005},"min":{"1min":0.45900000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.91200000000000003,"5min":2.8740000000000001,"15min":2.8740000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78700000000000003,"15min":0.85799999999999998},"min":{"1min":0.51000000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96299999999999997,"5min":2.883,"15min":2.883},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Mar 29 09:12:52 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.53900000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94099999999999995,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70999999999999996,"15min":0.74199999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.84999999999999998,"5min":1.387,"15min":1.387},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Mar 29 09:12:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66600000000000004,"15min":0.69199999999999995},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82099999999999995,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.755,"15min":0.77300000000000002},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.87,"5min":1.367,"15min":1.367},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Mar 29 09:12:52 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.71699999999999997,"15min":0.75600000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89900000000000002,"5min":1.458,"15min":1.458},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.68799999999999994,"15min":0.72599999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81999999999999995,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Mar 29 09:12:52 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73299999999999998,"15min":0.76500000000000001},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91800000000000004,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73699999999999999,"15min":0.76400000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.876,"5min":1.413,"15min":1.413},"last":0.81000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65700000000000003,"15min":0.66900000000000004},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.96399999999999997,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.64800000000000002,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.80900000000000005}]},{"osd":1,"last update":"Fri Mar 29 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.67100000000000004,"15min":0.63800000000000001},"min":{"1min":0.46300000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91400000000000003,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.745,"15min":0.80700000000000005},"min":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.76,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Mar 29 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69899999999999995,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88800000000000001,"5min":3.2599999999999998,"15min":3.2599999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.55000000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.97699999999999998,"5min":1,"15min":1},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Mar 29 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.61599999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76800000000000002,"15min":0.82299999999999995},"min":{"1min":0.51400000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.1360000000000001,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.78200000000000003}]},{"osd":4,"last update":"Fri Mar 29 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.68700000000000006,"15min":0.67300000000000004},"min":{"1min":0.47799999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.581,"5min":2.581,"15min":2.581},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.72599999999999998,"15min":0.70399999999999996},"min":{"1min":0.499,"5min":0.434,"15min":0.434},"max":{"1min":3.3130000000000002,"5min":3.3130000000000002,"15min":3.3130000000000002},"last":0.625}]},{"osd":5,"last update":"Fri Mar 29 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.68799999999999994,"15min":0.66900000000000004},"min":{"1min":0.432,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.8839999999999999,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.76900000000000002,"15min":0.751},"min":{"1min":0.46999999999999997,"5min":0.438,"15min":0.438},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Mar 29 09:12:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53100000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":3.242,"5min":3.242,"15min":3.242},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.508,"5min":2.508,"15min":2.508},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1298432,"data_stored":902850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.55300000000000005,"15min":0.61599999999999999},"min":{"1min":0.308,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.755,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.52700000000000002,"15min":0.55100000000000005},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.67200000000000004,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.62,"15min":0.66600000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.59999999999999998,"15min":0.66600000000000004},"min":{"1min":0.34000000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60499999999999998,"5min":4.2050000000000001,"15min":4.2050000000000001},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54100000000000004,"15min":0.55500000000000005},"min":{"1min":0.374,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72199999999999998,"5min":1,"15min":1},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.58599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70299999999999996,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.45900000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.96199999999999997,"15min":0.93400000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94799999999999995,"5min":12.356,"15min":12.356},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.57099999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.94999999999999996,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.438,"15min":0.438},"max":{"1min":1.018,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":0.92000000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.034,"5min":3.9529999999999998,"15min":3.9529999999999998},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.80200000000000005,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.91800000000000004,"5min":1.99,"15min":1.99},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":1.0109999999999999,"15min":0.97699999999999998},"min":{"1min":0.57799999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.99199999999999999,"5min":12.669,"15min":12.669},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:11:56 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.97899999999999998,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.84999999999999998,"15min":0.872},"min":{"1min":0.628,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97499999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.73399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41499999999999998,"15min":0.42299999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.46200000000000002,"15min":0.47999999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57699999999999996,"5min":0.751,"15min":0.751},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.48099999999999998,"15min":0.48999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59899999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.50700000000000001,"15min":0.52700000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.61799999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.45300000000000001,"15min":0.46400000000000002},"min":{"1min":0.29799999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.52800000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.54100000000000004,"15min":0.56399999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.61499999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.89600000000000002,"5min":1.028,"15min":1.028},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.52100000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.022,"5min":1.093,"15min":1.093},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71899999999999997,"15min":0.73099999999999998},"min":{"1min":0.496,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.99399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73599999999999999,"15min":0.749},"min":{"1min":0.53100000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92600000000000005,"5min":1.147,"15min":1.147},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76400000000000001,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.96699999999999997,"5min":1.069,"15min":1.069},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:11:53 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77500000000000002,"15min":0.79700000000000004},"min":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85099999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.879},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.749,"15min":0.755},"min":{"1min":0.58199999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97899999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.443,"15min":0.47699999999999998},"min":{"1min":0.32600000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.55200000000000005,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.47299999999999998,"15min":0.51700000000000002},"min":{"1min":0.26300000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.62,"5min":1.252,"15min":1.252},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.48699999999999999,"15min":0.54500000000000004},"min":{"1min":0.318,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.51200000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.55200000000000005,"15min":0.61799999999999999},"min":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.623,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.53000000000000003,"15min":0.60599999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55000000000000004,"5min":2.415,"15min":2.415},"last":0.5},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58999999999999997,"15min":0.66400000000000003},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.70899999999999996,"5min":2.056,"15min":2.056},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72799999999999998,"15min":0.78600000000000003},"min":{"1min":0.54400000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.872,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.755,"15min":0.81000000000000005},"min":{"1min":0.54700000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93100000000000005,"5min":2.073,"15min":2.073},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.84799999999999998},"min":{"1min":0.56799999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.0549999999999999,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.873},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76500000000000001,"15min":0.80000000000000004},"min":{"1min":0.54600000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.042,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.749}]},{"osd":6,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.88},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96799999999999997,"5min":2.387,"15min":2.387},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84299999999999997,"15min":0.92300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0960000000000001,"5min":2.996,"15min":2.996},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.78900000000000003,"15min":0.85199999999999998},"min":{"1min":0.59099999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.79000000000000004,"15min":0.85399999999999998},"min":{"1min":0.53000000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.96599999999999997,"5min":2.165,"15min":2.165},"last":0.84899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:12:53 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47399999999999998,"15min":0.53200000000000003},"min":{"1min":0.28999999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.0049999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50700000000000001,"15min":0.54500000000000004},"min":{"1min":0.35699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.64200000000000002,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:12:53 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47699999999999998,"15min":0.53300000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.996,"5min":2.2959999999999998,"15min":2.2959999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.55200000000000005},"min":{"1min":0.32100000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.96399999999999997,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.436}]},{"osd":2,"last update":"Fri Mar 29 09:12:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52400000000000002,"15min":0.58399999999999996},"min":{"1min":0.32400000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.69299999999999995,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.58099999999999996,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3240000000000001,"5min":2.5009999999999999,"15min":2.5009999999999999},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Mar 29 09:12:53 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.81200000000000006},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90900000000000003,"5min":2.278,"15min":2.278},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.81999999999999995,"15min":0.875},"min":{"1min":0.61499999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":0.93700000000000006,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":0.65400000000000003}]},{"osd":5,"last update":"Fri Mar 29 09:12:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.76100000000000001,"15min":0.80300000000000005},"min":{"1min":0.57399999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.96799999999999997,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.753,"15min":0.79800000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1040000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":0.624}]},{"osd":6,"last update":"Fri Mar 29 09:12:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.76900000000000002,"15min":0.81000000000000005},"min":{"1min":0.57399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92400000000000004,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.80300000000000005,"15min":0.875},"min":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.038,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Mar 29 09:12:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.81100000000000005,"15min":0.88300000000000001},"min":{"1min":0.54300000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1559999999999999,"5min":2.649,"15min":2.649},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.82299999999999995,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.92000000000000004,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":312482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68799999999999994,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.91500000000000004,"5min":2.0569999999999999,"15min":2.0569999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67600000000000005,"15min":0.70499999999999996},"min":{"1min":0.45800000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.88700000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.88700000000000001}]},{"osd":1,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66800000000000004,"15min":0.66900000000000004},"min":{"1min":0.51500000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.83499999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66800000000000004,"15min":0.70399999999999996},"min":{"1min":0.42999999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95599999999999996,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84999999999999998}]},{"osd":2,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.65500000000000003,"15min":0.66200000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77800000000000002,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.70099999999999996,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84999999999999998,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71699999999999997,"15min":0.73699999999999999},"min":{"1min":0.499,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.87,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.75700000000000001,"15min":0.79700000000000004},"min":{"1min":0.54700000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":0.754}]},{"osd":5,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.498},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.70799999999999996,"15min":0.79000000000000004},"min":{"1min":0.435,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83899999999999997,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.76400000000000001,"15min":0.82499999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.997,"5min":2.2309999999999999,"15min":2.2309999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.76700000000000002,"15min":0.78900000000000003},"min":{"1min":0.53100000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.89400000000000002,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.622}]},{"osd":7,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.77600000000000002,"15min":0.86899999999999999},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.85499999999999998,"15min":0.94599999999999995},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.97199999999999998,"5min":3.5790000000000002,"15min":3.5790000000000002},"last":0.70799999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1298432,"data_stored":902850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63,"15min":0.64700000000000002},"min":{"1min":0.497,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84799999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.67600000000000005},"min":{"1min":0.44500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91100000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.878,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66400000000000003,"15min":0.67900000000000005},"min":{"1min":0.44800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88900000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.70099999999999996,"15min":0.72899999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99199999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67000000000000004,"15min":0.66500000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72999999999999998,"15min":0.77200000000000002},"min":{"1min":0.53000000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95099999999999996,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71699999999999997,"15min":0.71599999999999997},"min":{"1min":0.55000000000000004,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.96999999999999997,"5min":1.036,"15min":1.036},"last":0.879}]},{"osd":4,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.65000000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.71999999999999997,"15min":0.73899999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.87,"5min":1.05,"15min":1.05},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84399999999999997,"5min":0.998,"15min":0.998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.72699999999999998,"15min":0.75},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88200000000000001,"5min":1.079,"15min":1.079},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Mar 29 09:12:54 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.79400000000000004},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.89800000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78000000000000003,"15min":0.79400000000000004},"min":{"1min":0.50700000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.95499999999999996,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T09:13:02.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-29T09:13:02.518 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.518+0000 7f3538000700 1 Processor -- start 2024-03-29T09:13:02.519 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.519+0000 7f3538000700 1 -- start start 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3538000700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3530160770 0x7f35300b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3538000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35300d0d60 0x7f35300d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3538000700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35300d3810 con 0x7f35300d3380 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3538000700 1 -- --> v2:172.21.15.99:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35300d1520 con 0x7f3530160770 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3538000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35300d1800 con 0x7f35300d0d60 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3535600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3530160770 0x7f35300b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3535600700 1 --2- >> v2:172.21.15.99:3300/0 conn(0x7f3530160770 0x7f35300b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47652/0 (socket says 172.21.15.99:47652) 2024-03-29T09:13:02.521 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3535600700 1 -- 172.21.15.99:0/1311919911 learned_addr learned my addr 172.21.15.99:0/1311919911 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3534c00700 1 --2- 172.21.15.99:0/1311919911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35300d0d60 0x7f35300d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3535600700 1 -- 172.21.15.99:0/1311919911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35300d0d60 msgr2=0x7f35300d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3535600700 1 --2- 172.21.15.99:0/1311919911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35300d0d60 0x7f35300d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.520+0000 7f3535600700 1 -- 172.21.15.99:0/1311919911 >> v1:172.21.15.133:6789/0 conn(0x7f35300d3380 legacy=0x7f35300cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.521+0000 7f3535600700 1 -- 172.21.15.99:0/1311919911 --> v2:172.21.15.99:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35300d1ae0 con 0x7f3530160770 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.521+0000 7f3535600700 1 --2- 172.21.15.99:0/1311919911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3530160770 0x7f35300b3cc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f35240098f0 tx=0x7f3524009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accb5c518309227e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.521+0000 7f352fe00700 1 -- 172.21.15.99:0/1311919911 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352400b8a0 con 0x7f3530160770 2024-03-29T09:13:02.522 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.521+0000 7f352fe00700 1 -- 172.21.15.99:0/1311919911 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352400ba00 con 0x7f3530160770 2024-03-29T09:13:02.523 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.521+0000 7f352fe00700 1 -- 172.21.15.99:0/1311919911 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352400bd90 con 0x7f3530160770 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.521+0000 7f3538000700 1 -- 172.21.15.99:0/1311919911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3530160770 msgr2=0x7f35300b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.521+0000 7f3538000700 1 --2- 172.21.15.99:0/1311919911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3530160770 0x7f35300b3cc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f35240098f0 tx=0x7f3524009c00 comp rx=0 tx=0).stop 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.522+0000 7f3538000700 1 -- 172.21.15.99:0/1311919911 shutdown_connections 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.522+0000 7f3538000700 1 --2- 172.21.15.99:0/1311919911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3530160770 0x7f35300b3cc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.522+0000 7f3538000700 1 --2- 172.21.15.99:0/1311919911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35300d0d60 0x7f35300d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.522+0000 7f3538000700 1 -- 172.21.15.99:0/1311919911 >> 172.21.15.99:0/1311919911 conn(0x7f353008afd0 msgr2=0x7f353009c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.522+0000 7f3538000700 1 -- 172.21.15.99:0/1311919911 shutdown_connections 2024-03-29T09:13:02.526 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.522+0000 7f3538000700 1 -- 172.21.15.99:0/1311919911 wait complete. 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.523+0000 7f3538000700 1 Processor -- start 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.523+0000 7f3538000700 1 -- start start 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.524+0000 7f3538000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35300d0d60 0x7f35300cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.524+0000 7f3538000700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35300d3380 0x7f35300ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.524+0000 7f3538000700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3530160770 0x7f35301515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.524+0000 7f3538000700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353015abe0 con 0x7f35300d0d60 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.524+0000 7f3538000700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300c80c0 con 0x7f35300d3380 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.524+0000 7f3538000700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35300c8310 con 0x7f3530160770 2024-03-29T09:13:02.527 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35300d0d60 0x7f35300cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35300d0d60 0x7f35300cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47666/0 (socket says 172.21.15.99:47666) 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 -- 172.21.15.99:0/1671640690 learned_addr learned my addr 172.21.15.99:0/1671640690 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3534c00700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35300d3380 0x7f35300ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3536000700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3530160770 0x7f35301515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 -- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3530160770 msgr2=0x7f35301515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3530160770 0x7f35301515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 -- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35300d3380 msgr2=0x7f35300ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.528 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35300d3380 0x7f35300ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 -- 172.21.15.99:0/1671640690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3524009550 con 0x7f35300d0d60 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3534c00700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35300d3380 0x7f35300ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f3535600700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35300d0d60 0x7f35300cd020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3524009c40 tx=0x7f3524004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.525+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3524004590 con 0x7f35300d0d60 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.526+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352401e070 con 0x7f35300d0d60 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.526+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35300c85f0 con 0x7f35300d0d60 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.526+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35240038e0 con 0x7f35300d0d60 2024-03-29T09:13:02.529 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.526+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35300c8bb0 con 0x7f35300d0d60 2024-03-29T09:13:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.527+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35300678b0 con 0x7f35300d0d60 2024-03-29T09:13:02.530 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.527+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f3524003a40 con 0x7f35300d0d60 2024-03-29T09:13:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.529+0000 7f352d600700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f351006ac90 0x7f351006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.529+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f3524020070 con 0x7f35300d0d60 2024-03-29T09:13:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.529+0000 7f3534c00700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f351006ac90 0x7f351006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.531 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.530+0000 7f3534c00700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f351006ac90 0x7f351006d150 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f35300c8050 tx=0x7f351c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.533 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.533+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f3524055960 con 0x7f35300d0d60 2024-03-29T09:13:02.742 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.742+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f353015fac0 con 0x7f35300d0d60 2024-03-29T09:13:02.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.746+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f3524055960 con 0x7f35300d0d60 2024-03-29T09:13:02.747 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.747+0000 7f352d600700 1 -- 172.21.15.99:0/1671640690 <== mon.0 v2:172.21.15.99:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f3524058f80 con 0x7f35300d0d60 2024-03-29T09:13:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f351006ac90 msgr2=0x7f351006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f351006ac90 0x7f351006d150 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f35300c8050 tx=0x7f351c008040 comp rx=0 tx=0).stop 2024-03-29T09:13:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35300d0d60 msgr2=0x7f35300cd020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35300d0d60 0x7f35300cd020 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3524009c40 tx=0x7f3524004750 comp rx=0 tx=0).stop 2024-03-29T09:13:02.750 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 shutdown_connections 2024-03-29T09:13:02.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f351006ac90 0x7f351006d150 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3530160770 0x7f35301515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f35300d0d60 0x7f35300cd020 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 --2- 172.21.15.99:0/1671640690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35300d3380 0x7f35300ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 >> 172.21.15.99:0/1671640690 conn(0x7f353008afd0 msgr2=0x7f3530073120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:13:02.751 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 shutdown_connections 2024-03-29T09:13:02.752 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.750+0000 7f3538000700 1 -- 172.21.15.99:0/1671640690 wait complete. 2024-03-29T09:13:02.768 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T09:13:02.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T09:13:02.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.958+0000 7f7c69c00700 1 Processor -- start 2024-03-29T09:13:02.958 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.958+0000 7f7c69c00700 1 -- start start 2024-03-29T09:13:02.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.959+0000 7f7c69c00700 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c6415d980 0x7f7c640cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.959+0000 7f7c69c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c640b1780 0x7f7c640ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.959+0000 7f7c69c00700 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c640d1a50 con 0x7f7c640cfd70 2024-03-29T09:13:02.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.959+0000 7f7c69c00700 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c640d2260 con 0x7f7c6415d980 2024-03-29T09:13:02.959 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.959+0000 7f7c69c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640d1ca0 con 0x7f7c640b1780 2024-03-29T09:13:02.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c640b1780 0x7f7c640ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c640b1780 0x7f7c640ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.99:35756/0 (socket says 172.21.15.99:35756) 2024-03-29T09:13:02.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 -- 172.21.15.99:0/727270729 learned_addr learned my addr 172.21.15.99:0/727270729 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:13:02.961 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c62000700 1 --2- 172.21.15.99:0/727270729 >> v2:172.21.15.133:3300/0 conn(0x7f7c6415d980 0x7f7c640cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 -- 172.21.15.99:0/727270729 >> v2:172.21.15.133:3300/0 conn(0x7f7c6415d980 msgr2=0x7f7c640cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 --2- 172.21.15.99:0/727270729 >> v2:172.21.15.133:3300/0 conn(0x7f7c6415d980 0x7f7c640cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 -- 172.21.15.99:0/727270729 >> v1:172.21.15.133:6789/0 conn(0x7f7c640cfd70 legacy=0x7f7c6415d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 -- 172.21.15.99:0/727270729 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640d1f80 con 0x7f7c640b1780 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c61600700 1 -- 172.21.15.99:0/727270729 <== mon.1 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507913384 0 0) 0x7f7c640d1a50 con 0x7f7c640cfd70 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.960+0000 7f7c63400700 1 --2- 172.21.15.99:0/727270729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c640b1780 0x7f7c640ce270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009e50 tx=0x7f7c5c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceef21eb2f7e73ae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c61600700 1 -- 172.21.15.99:0/727270729 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00e070 con 0x7f7c640b1780 2024-03-29T09:13:02.962 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c61600700 1 -- 172.21.15.99:0/727270729 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7c5c00b910 con 0x7f7c640b1780 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c61600700 1 -- 172.21.15.99:0/727270729 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c00bce0 con 0x7f7c640b1780 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 -- 172.21.15.99:0/727270729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c640b1780 msgr2=0x7f7c640ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 --2- 172.21.15.99:0/727270729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c640b1780 0x7f7c640ce270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009e50 tx=0x7f7c5c00b330 comp rx=0 tx=0).stop 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 -- 172.21.15.99:0/727270729 shutdown_connections 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 --2- 172.21.15.99:0/727270729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c640b1780 0x7f7c640ce270 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 --2- 172.21.15.99:0/727270729 >> v2:172.21.15.133:3300/0 conn(0x7f7c6415d980 0x7f7c640cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 -- 172.21.15.99:0/727270729 >> 172.21.15.99:0/727270729 conn(0x7f7c6409c470 msgr2=0x7f7c640d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 -- 172.21.15.99:0/727270729 shutdown_connections 2024-03-29T09:13:02.963 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.961+0000 7f7c69c00700 1 -- 172.21.15.99:0/727270729 wait complete. 2024-03-29T09:13:02.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.963+0000 7f7c69c00700 1 Processor -- start 2024-03-29T09:13:02.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.963+0000 7f7c69c00700 1 -- start start 2024-03-29T09:13:02.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c69c00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c640b1780 0x7f7c640cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c69c00700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c640cfd70 0x7f7c6415a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c69c00700 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c6415d980 0x7f7c640b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c69c00700 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640ca8d0 con 0x7f7c640cfd70 2024-03-29T09:13:02.966 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c69c00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640cd640 con 0x7f7c640b1780 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c69c00700 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c640ca5f0 con 0x7f7c6415d980 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c62000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c640cfd70 0x7f7c6415a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c62000700 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c640cfd70 0x7f7c6415a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:47678/0 (socket says 172.21.15.99:47678) 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.964+0000 7f7c62000700 1 -- 172.21.15.99:0/2046570123 learned_addr learned my addr 172.21.15.99:0/2046570123 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c63400700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c6415d980 0x7f7c640b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c62a00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c640b1780 0x7f7c640cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c62000700 1 -- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c6415d980 msgr2=0x7f7c640b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c62000700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c6415d980 0x7f7c640b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.967 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c62000700 1 -- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c640b1780 msgr2=0x7f7c640cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c62000700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c640b1780 0x7f7c640cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c62000700 1 -- 172.21.15.99:0/2046570123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5c009b30 con 0x7f7c640cfd70 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c62000700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c640cfd70 0x7f7c6415a6a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e870 tx=0x7f7c5000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c5aa00700 1 -- 172.21.15.99:0/2046570123 <== mon.0 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50009590 con 0x7f7c640cfd70 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c5aa00700 1 -- 172.21.15.99:0/2046570123 <== mon.0 v2:172.21.15.99:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7c50013680 con 0x7f7c640cfd70 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c5aa00700 1 -- 172.21.15.99:0/2046570123 <== mon.0 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50013a10 con 0x7f7c640cfd70 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.965+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c641f3e40 con 0x7f7c640cfd70 2024-03-29T09:13:02.968 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.966+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c641f4400 con 0x7f7c640cfd70 2024-03-29T09:13:02.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.966+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c640b4a30 con 0x7f7c640cfd70 2024-03-29T09:13:02.969 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.967+0000 7f7c5aa00700 1 -- 172.21.15.99:0/2046570123 <== mon.0 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 5) v1 ==== 89137+0+0 (secure 0 0 0) 0x7f7c50013b70 con 0x7f7c640cfd70 2024-03-29T09:13:02.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.968+0000 7f7c5aa00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c4406ac90 0x7f7c4406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T09:13:02.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.968+0000 7f7c5aa00700 1 -- 172.21.15.99:0/2046570123 <== mon.0 v2:172.21.15.99:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f7c50089160 con 0x7f7c640cfd70 2024-03-29T09:13:02.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.968+0000 7f7c62a00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c4406ac90 0x7f7c4406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T09:13:02.970 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.969+0000 7f7c62a00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c4406ac90 0x7f7c4406d150 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c54007fd0 tx=0x7f7c54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T09:13:02.972 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:02.973+0000 7f7c5aa00700 1 -- 172.21.15.99:0/2046570123 <== mon.0 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181614 (secure 0 0 0) 0x7f7c500543d0 con 0x7f7c640cfd70 2024-03-29T09:13:03.219 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.219+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c64067d80 con 0x7f7c640cfd70 2024-03-29T09:13:03.220 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.220+0000 7f7c5aa00700 1 -- 172.21.15.99:0/2046570123 <== mon.0 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f7c50017de0 con 0x7f7c640cfd70 2024-03-29T09:13:03.220 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-03-29T09:13:03.220 INFO:teuthology.orchestra.run.smithi099.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5432,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/790352630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":790352630},{"type":"v1","addr":"172.21.15.133:6833","nonce":790352630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5435,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1265449168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1265449168},{"type":"v1","addr":"172.21.15.133:6835","nonce":1265449168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6102,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6836/1161439746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":1161439746},{"type":"v1","addr":"172.21.15.99:6836","nonce":1161439746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6576,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/4025583999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6835","nonce":4025583999},{"type":"v1","addr":"172.21.15.99:6837","nonce":4025583999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2024-03-29T09:13:03.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c4406ac90 msgr2=0x7f7c4406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:03.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c4406ac90 0x7f7c4406d150 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c54007fd0 tx=0x7f7c54009040 comp rx=0 tx=0).stop 2024-03-29T09:13:03.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c640cfd70 msgr2=0x7f7c6415a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T09:13:03.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c640cfd70 0x7f7c6415a6a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000e870 tx=0x7f7c5000eb80 comp rx=0 tx=0).stop 2024-03-29T09:13:03.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 shutdown_connections 2024-03-29T09:13:03.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:6832/41898,v1:172.21.15.99:6833/41898] conn(0x7f7c4406ac90 0x7f7c4406d150 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:03.223 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c6415d980 0x7f7c640b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:03.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c640b1780 0x7f7c640cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:03.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 --2- 172.21.15.99:0/2046570123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7c640cfd70 0x7f7c6415a6a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T09:13:03.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 >> 172.21.15.99:0/2046570123 conn(0x7f7c6409c470 msgr2=0x7f7c6406bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T09:13:03.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.222+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 shutdown_connections 2024-03-29T09:13:03.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-03-29T09:13:03.223+0000 7f7c69c00700 1 -- 172.21.15.99:0/2046570123 wait complete. 2024-03-29T09:13:03.224 INFO:teuthology.orchestra.run.smithi099.stderr:dumped fsmap epoch 27 2024-03-29T09:13:03.240 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-29T09:13:03.240 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-29T09:13:03.241 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T09:13:03.241 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.255 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T09:13:03.255 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T09:13:03.255 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.309 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T09:13:03.309 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T09:13:03.309 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.326 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T09:13:03.326 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T09:13:03.327 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.383 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T09:13:03.383 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-29T09:13:03.384 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-29T09:13:03.384 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.424 INFO:tasks.ceph.osd.0:Stopped 2024-03-29T09:13:03.424 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-29T09:13:03.424 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.507 INFO:tasks.ceph.osd.1:Stopped 2024-03-29T09:13:03.508 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-29T09:13:03.508 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.553 INFO:tasks.ceph.osd.2:Stopped 2024-03-29T09:13:03.553 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-29T09:13:03.553 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.597 INFO:tasks.ceph.osd.3:Stopped 2024-03-29T09:13:03.598 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-29T09:13:03.598 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.640 INFO:tasks.ceph.osd.4:Stopped 2024-03-29T09:13:03.641 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-29T09:13:03.641 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.689 INFO:tasks.ceph.osd.5:Stopped 2024-03-29T09:13:03.690 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-29T09:13:03.690 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.740 INFO:tasks.ceph.osd.6:Stopped 2024-03-29T09:13:03.740 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-29T09:13:03.740 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.792 INFO:tasks.ceph.osd.7:Stopped 2024-03-29T09:13:03.793 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-29T09:13:03.793 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-29T09:13:03.793 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.859 INFO:tasks.ceph.mgr.y:Stopped 2024-03-29T09:13:03.859 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-29T09:13:03.859 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.921 INFO:tasks.ceph.mgr.x:Stopped 2024-03-29T09:13:03.922 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-29T09:13:03.922 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-29T09:13:03.922 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.943 INFO:tasks.ceph.mon.a:Stopped 2024-03-29T09:13:03.944 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-29T09:13:03.944 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:03.961 INFO:tasks.ceph.mon.b:Stopped 2024-03-29T09:13:03.962 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-29T09:13:03.962 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T09:13:04.021 INFO:tasks.ceph.mon.c:Stopped 2024-03-29T09:13:04.021 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-29T09:13:04.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-29T09:13:04.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:13:04.101 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-29T09:13:04.367 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:13:04.367 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-29T09:13:04.512 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:13:04.513 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-29T09:13:04.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:13:04.627 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-29T09:13:04.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:13:04.778 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-29T09:13:05.081 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:13:05.081 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-29T09:13:05.222 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:13:05.223 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-29T09:13:05.338 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:13:05.338 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-29T09:13:05.478 INFO:tasks.ceph:Archiving mon data... 2024-03-29T09:13:05.480 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/data/mon.a.tgz 2024-03-29T09:13:05.480 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-03-29T09:13:05.523 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.DxHcbxP3RY 2024-03-29T09:13:05.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.DxHcbxP3RY 2024-03-29T09:13:05.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.DxHcbxP3RY 2024-03-29T09:13:05.816 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.DxHcbxP3RY is 369KB 2024-03-29T09:13:05.912 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.DxHcbxP3RY 2024-03-29T09:13:05.952 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/data/mon.b.tgz 2024-03-29T09:13:05.952 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-03-29T09:13:05.994 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.9CZ7evmc1M 2024-03-29T09:13:05.995 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.9CZ7evmc1M 2024-03-29T09:13:06.245 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.9CZ7evmc1M 2024-03-29T09:13:06.353 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.9CZ7evmc1M is 487KB 2024-03-29T09:13:06.449 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.9CZ7evmc1M 2024-03-29T09:13:06.489 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/data/mon.c.tgz 2024-03-29T09:13:06.489 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-03-29T09:13:06.569 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.typNjhiTxQ 2024-03-29T09:13:06.569 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.typNjhiTxQ 2024-03-29T09:13:06.769 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.typNjhiTxQ 2024-03-29T09:13:06.877 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.typNjhiTxQ is 487KB 2024-03-29T09:13:06.974 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.typNjhiTxQ 2024-03-29T09:13:07.015 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-29T09:13:07.015 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T09:13:07.019 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T09:13:07.119 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-29T09:13:07.119 DEBUG:teuthology.orchestra.run.smithi099:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-29T09:13:07.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-29T09:13:07.200 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-29T09:13:07.207 INFO:teuthology.orchestra.run.smithi133.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-29T09:13:07.222 INFO:tasks.ceph:Archiving crash dumps... 2024-03-29T09:13:07.224 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/remote/smithi099/crash 2024-03-29T09:13:07.225 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T09:13:07.289 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/remote/smithi133/crash 2024-03-29T09:13:07.290 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T09:13:07.353 INFO:tasks.ceph:Shutting down logrotate 2024-03-29T09:13:07.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:13:07.357 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T09:13:07.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T09:13:07.517 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T09:13:07.580 INFO:tasks.ceph:Compressing logs... 2024-03-29T09:13:07.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T09:13:07.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T09:13:11.731 INFO:tasks.ceph:Archiving logs... 2024-03-29T09:13:11.732 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/remote/smithi099/log 2024-03-29T09:13:11.733 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T09:13:12.526 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/remote/smithi133/log 2024-03-29T09:13:12.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T09:13:13.007 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T09:13:13.017 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T09:13:13.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T09:13:13.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T09:13:13.143 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T09:13:13.153 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T09:13:13.153 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T09:13:13.158 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T09:13:13.193 INFO:teuthology.orchestra.run.smithi099.stderr:bash: ntpq: command not found 2024-03-29T09:13:13.198 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2024-03-29T09:13:13.200 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T09:13:13.200 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-03-29T09:13:13.200 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 +460us[ +460us] +/- 75ms 2024-03-29T09:13:13.201 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -527us[ -525us] +/- 56ms 2024-03-29T09:13:13.201 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +197us[ +197us] +/- 70ms 2024-03-29T09:13:13.201 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:13:13.206 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-29T09:13:13.206 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-03-29T09:13:13.206 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 53 +504us[ +504us] +/- 75ms 2024-03-29T09:13:13.206 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -486us[ -476us] +/- 56ms 2024-03-29T09:13:13.206 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +221us[ +232us] +/- 70ms 2024-03-29T09:13:13.206 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-29T09:13:13.207 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T09:13:13.217 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T09:13:13.217 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T09:13:13.240 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T09:13:13.380 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-29T09:13:13.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T09:13:13.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:13:13.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-29T09:13:13.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:13:13.643 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T09:13:13.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T09:13:13.683 INFO:teuthology.task.internal:Duration was 1189.205676 seconds 2024-03-29T09:13:13.684 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T09:13:13.716 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T09:13:13.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T09:13:13.720 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T09:13:13.826 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T09:13:13.831 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-29T09:13:14.219 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T09:13:14.219 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-03-29T09:13:14.219 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T09:13:14.276 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-03-29T09:13:14.277 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T09:13:14.333 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T09:13:14.333 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T09:13:14.338 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T09:13:14.423 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T09:13:14.423 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T09:13:14.427 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T09:13:14.557 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T09:13:14.570 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T09:13:14.571 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T09:13:14.636 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T09:13:14.702 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T09:13:14.712 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:13:14.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:13:14.775 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-03-29T09:13:14.780 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-03-29T09:13:14.813 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:13:14.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:13:14.853 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T09:13:14.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T09:13:14.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T09:13:14.905 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T09:13:14.905 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/remote/smithi099 2024-03-29T09:13:14.906 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T09:13:14.989 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169/remote/smithi133 2024-03-29T09:13:14.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T09:13:15.064 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T09:13:15.064 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T09:13:15.069 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T09:13:15.109 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T09:13:15.121 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T09:13:15.122 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T09:13:15.143 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T09:13:15.144 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T09:13:15.151 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T09:13:15.188 INFO:teuthology.orchestra.run.smithi099.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 09:13 /home/ubuntu/cephtest 2024-03-29T09:13:15.192 INFO:teuthology.orchestra.run.smithi133.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 09:13 /home/ubuntu/cephtest 2024-03-29T09:13:15.196 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T09:13:15.207 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T09:13:15.267 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T09:13:15.293 DEBUG:teuthology.nuke:shortname: smithi099 2024-03-29T09:13:15.294 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T09:13:15.318 DEBUG:teuthology.nuke:shortname: smithi133 2024-03-29T09:13:15.318 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T09:13:15.322 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 08:41:48.058747', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHjuiw4ET3PmkH1wWnugZbpBQ11jBn6EVX0hktynHK1DwKhuOOn59FfDYFbktPv+hJf2YQy+VF9dgCgM4ZmInxo='} 2024-03-29T09:13:15.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628169', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 08:41:48.056934', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEAdC+qenmuQZ2Ai2w2X9Q44TgT72N3dd0UgBMDYJpKaq1BCq99ERVhZj4/lvVfPK/WygeKTtsZP/EXl4Fppfs0='} 2024-03-29T09:13:15.380 INFO:teuthology.orchestra.console.smithi099:Power off 2024-03-29T09:13:15.380 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T09:13:15.403 DEBUG:teuthology.orchestra.console.smithi099:power off output: Chassis Power Control: Down/Off 2024-03-29T09:13:15.403 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:13:15.421 INFO:teuthology.orchestra.console.smithi133:Power off 2024-03-29T09:13:15.421 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T09:13:15.430 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-03-29T09:13:15.442 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2024-03-29T09:13:15.442 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:13:15.465 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-03-29T09:13:19.432 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:13:19.466 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:13:19.542 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-03-29T09:13:19.577 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-03-29T09:13:23.545 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:13:23.577 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:13:23.656 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is on 2024-03-29T09:13:23.688 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2024-03-29T09:13:23.788 INFO:teuthology.orchestra.console.smithi133:Power off completed 2024-03-29T09:13:27.660 DEBUG:teuthology.orchestra.console.smithi099:pexpect command: ipmitool -H smithi099.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T09:13:27.770 DEBUG:teuthology.orchestra.console.smithi099:check power output: Chassis Power is off 2024-03-29T09:13:27.871 INFO:teuthology.orchestra.console.smithi099:Power off completed 2024-03-29T09:13:27.976 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/pool-perm} duration: 1189.205676317215 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=ecf4333faa9447eeafbed6cde0cc1976 status: fail success: false 2024-03-29T09:13:27.976 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T09:13:28.057 INFO:teuthology.run:FAIL